Paper & Final Exam Comments

Final Exam Comments

Good job on the final exam, folks! Most people in my sections improved their % of correct answers between the midterm and final, and actually, the mean change was +3 percentage points, even though the class median % went down. Two students in my sections actually improved their scores by more than 30 percentage points! (Wow!) I'd like to take credit for your achievements, of course, but I know that you are the ones who put in the time and effort to study for this class, so this was your achievement. Great job!

Overall Paper Comments

Overall, I was also really impressed with your papers. First of all, I was really happy about the quality of your writing (for the most part :] ). Based on other precomm papers I've graded in the last few years, I was expecting a lot more trouble understanding what you had to say, but it seems that most of you really picked up on the straightforward social-science style. I'd recommend that you try the same style in your other comm papers! Secondly, most of you did a really good job at following the assignment prompt and including most, if not all, of the elements you needed. Good for you!

On the flip side, that means that papers that didn't include everything pretty much automatically looked worse than the papers that did. Remember: paper grades are based on the median paper, so your papers weren't competing against some arbitrary standard that I thought would be adequate, but they were rather competing against other papers, so your grades are relative to how much time, effort, knowledge, understanding, etc. other students put into their papers, and in this case, it seems everyone else did put in a substantial amount of time, effort, etc. If your paper got above around a 55/80, then you got the main point of the assignment. If you got below that score, that probably means you ere missing something major, and I'd welcome your meeting with me to discuss what you can do to improve for your next precomm class. And If you got a "check-minus" or below on the "Writing style" criterion, I'd STRONGLY recommend that you visit CLAS before turning in your next comm paper -- it means that I had to spend a significant amount of time struggling to decode your writing before I could understand what you were trying to say (if I understood it at all).

As you probably noticed, I didn't have the time to put written summary comments on most of your papers. Instead, most of your feedback was in the form of written comments on the paper, plus that half-sheet at the end. But before I explain what my codes mean, here's a reminder of what I said about the paper in section (taken directly from the slides)...

Section Slides

Paper objectives

• Learn how to conduct research
• Understand the research process & criteria for evaluation
• Understand your own study
• Learn how to follow procedure & directions
• Learn how to report research
• Understand what should be included in each section
• Learn to write well in a scientific manner & in APA style

General paper-writing tips

• 1. Put in time & effort.
• 2. Follow directions (esp pp.11-12 of reader).
• 3. Evaluate your study & writing based on the “steps” criteria.
• 4. Use course terminology & show what you’ve learned.
• 5. Focus on content.
• Note: I will follow pp. 11-12 of the reader & my handout CLOSELY when grading

Specific paper-writing tips

• “Introduction” is a section, not a paragraph
• Relate your results to your hypotheses
• APA style
• See p. 17
• Paraphrase, don’t quote
• Cite everything that’s not your group’s idea
• Group section of this class basically over
• Still OK to confer re: specific things, but
• DON’T READ ANY PORTION OF ANY GROUP MEMBER’S WRITING
• Important: Write your name on the back of your back page & nowhere else
• P.11

Grading Scheme

So how did I decide how well you'd completed each of the parts of this assignment, relative to other students? Just as I said I would -- by seeing to what degree you followed directions, included everything you were supposed to, showed an ability to discuss your study using correct course terminology & other knowledge gained from the class, and otherwise met the paper objectives I showed you. And remember how, when I said, "I will follow pp. 11-12 of the reader & my handout CLOSELY when grading," I threatened to make a checklist and check your papers to make sure you'd included everything? Well, as you probably noticed, I (eventually) decided to do just that. So here's what the codes mean:

This means that you gave a brilliant, knowledgeable, thorough, clear, nuanced, insightful answer; you demonstrated a thorough understanding of course material and its relationship to your study; you clearly met the criteria in the handout for good completion of applicable research steps.
This means that you gave something more than the basic requirement -- you gave more detail; you were able to link important, relevant outside class knowledge to your discussion; you noticed important things about your study; etc.
This means that you completed the task at about the median level. You "did the job," but nothing more.
This means that something important was missing from your answer, or your answer was very vague, or you got something a little wrong.
This means that either you didn't include this element in your paper, or your discussion showed significant misunderstanding of the element.

Here are some examples of probably the simplest element of the paper, the "Overall design."

"We used the survey method, because after we randomly assigned people to groups and showed them websites with different levels of interactivity, we had them fill out a questionnaire."
"We used an experiment with a survey."
"The overall design of our study was an experiment."
"The overall design of our study was an experiment, and in particular, a posttest-only control group design."
"Because this study's hypotheses predicted a cause-and-effect relationship between interactivity and clicking behavior, the methodology of this study was an experiment -- we manipulated our independent variable, website interactivity; we flipped a coin to randomly assign participants to our experimental and control groups; and we attempted to control all other extraneous variables by treating all participants equally. The particular design of this study was a posttest-only control group design, because we observed clicking behavior in our randomly assigned experimental and control groups only after participants viewed the stimulus webpage."

People's paper scores were based on how far above or below the median their paper was overall, balancing all the sections and weighting more heavily the "most important section!" (p.12 reader), the discussion section. So if your paper got more minuses than pluses, you probably got below the median score, and if your paper had a lot more pluses than minuses, you probably got a lot higher than the median score. The student with the top paper has generously given me a copy of her paper, so if you'd like to see what it would've taken you to get the top score, you can come to my office and I can show the paper to you.

Written Comments in Paper

Comments within the paper can help you figure out why you got the pluses & minuses, but a lot of the written comments in the paper were mainly aimed at helping you improve your paper-writing skills. A lot of you probably read my comments and thought, "Wow -- that's really nitpicky!" If that's the case, then that probably means I thought highly enough of your abilities that I thought you could handle detailed critiques. My goal with the nitpicky stuff wasn't to be overly harsh with your grade, it was to push your writing to the next level. You can get a better idea of what your grade was based on by looking at the half-sheet than by looking at the detailed comments.

FYI, stuff that was underlined or put in parentheses were either wrong or awkwardly phrased. Comments/changes of my own that are in parentheses are recommendations to improve your writing; they don't necessarily mean that you got the thing wrong, they're only suggestions that you might want to consider. Arrows generally mean that your subject-verb agreement or subject-pronoun agreement was off, but sometimes they're just arrows. Circles were for my own reference; I usually used them to identify your hypotheses/RQ's, if you didn't clearly identify them yourself.

Remaining Points of Confusion

Here are some concepts that a lot of you still seem confused about...

1. Social science does not answer philosophical or moral questions (re: values, appropriateness, social policy, etc.); social science doesn't say what should or shouldn't be, it just shows what is. What is can inform what you think should be, but doesn't itself recommend anything.

2. The only way you can generalize to a larger population than your sample is if you get a representative sample of that population. The only way you can get a representative sample of a population is through random sampling techniques. Methodology has little to do with external validity (ability to generalize) -- it's not true that all surveys have external validity and all experiments don't; it's only true that most survey/correlational surveys use representative sampling techniques, and most experiments do not, and it's those sampling techniques that determine generalizability. You can't generalize results to a larger population with an improperly conducted survey with no random sampling, while you can properly generalize results to a larger population with an experiment that used random sampling techniques. To what population can you properly generalize? To the population from which you randomly selected your sample.

3. Internal validity is not the same thing as the ability to make causal claims. "Internal validity" refers to the ability to conclude that the results of your study accurately reflect what went on in your study; it refers to how well you conducted your study. So experiments and surveys both have some level of internal validity (just as they both have some level of external validity), but it's not true that "experiments" automatically have internal validity, any more than it's true that "surveys" automatically have external validity. BUT it is true that if an experiment has been performed correctly, i.e., with true random assignment to conditions based on a manipulated independent variable, and with all variables other than the manipulated IV held constant across groups, then it has internal validity, and if an experiment has been performed correctly, then causal claims can be made. So if and only if an experiment has been performed correctly, then (a) the experiment has internal validity and (b) causal claims can be made.

4. Random sampling and random selection are not the same thing as random assignment (see Final FAQ #2, Q10), so you can't use representative sampling methods to randomly assign people to conditions. That means that any sort of systematic assignment to groups is just that -- systematic (so it'll produce systematic error), not random.

5. Hypotheses are the basis for everything else that goes on in your study, so (a) your rationale for making your predictions should be clear and well thought out, and (b) everything should be linked to your hypotheses. Hypotheses say what variables you're interested in, which variables (if any) you need to manipulate and which you need to measure, what relationship among variables is predicted, what type of research methodology you need to use, what type of sample you need to collect, what type of statistical analyses you need to perform, and what conclusions you can make. So your results section should be focussed on testing your predictions, which means that the analyses you perform should be based on the variables in your hypotheses, not on the questions in your questionnaire. Because results should show relationships among variables, rather than people's responses to particular questions, if you've used multiple questions to measure a variable (for triangulation of measurement), then you need to create a scale of those questions to turn multiple items into one variable.

Most Commmon Writing Errors

Here are the most common writing errors...

1. Errors with amount vs. number terms. Use number terms (e.g., "many," "few,") when quantities are measured as individual, discrete items, and amount terms (e.g., "much," "little,") when quantities are massed and collective. People are always referred to with number terms (e.g., "a large number of people preferred interactive to noninteractive websites") and not with amount terms. You can think of the difference in this way (warning: this example is gruesome, but probably memorable): in order to talk about people in amount terms, you'd need to make them all massed together, so you'd need to e.g., put a bunch of people into a wood mulcher; only then could you select "a little amount" or "a large amount" of people -- you could select e.g., 100 oz of people, or 1 gallon of people, etc. Otherwise, you can only measure people as discrete, complete individuals.

On a related note, "between" is used when entities are distinct individuals (a traitor stood between the two patriots"), and "among" is used when items are a mass or collective group (e.g., "there was a traitor among them"), and in most cases, "between" is used for two entities, and "among" is used for more than two entities or masses.

On another related note, people are "whos" and not "whats," so you should say, "Participants who looked at the interactive website..." rather than "Participants that looked at the interactive website..." That's especially true in social science -- it's important for ethical reasons to treat participants as people rather than lab rats (which is the reason that they're now called "participants" instead of "subjects").

2. Errors with comparison terms. If you use comparison terms, you need to specify what's being compared, so you need to say e.g., more than what, less than what, as ______ as what, different from what (also notice the preposition -- "different" always takes "from" and not "than"), etc.

3. Errors with homonyms, homophones, and plurals, e.g., its (possessive form of "it") and it's (contraction of "it is"), whose (possesive form of who) and who's (contraction of "who is"), effects (noun) vs. affects (verb), media (modes of communication) vs. mediums (people who communicate with the dead), etc.

Wow, that was long! :] In the next few days, I plan to make one last blog entry about how you can use the knowledge you've gained in this class.