Midterm FAQ, part 2

Q: What's the difference between inter-coder reliability and intra-coder reliability?

A: There's inter-coder reliability if all observers who are coding something (e.g., assigning a grade to your group project) code the same way (e.g., all 4 TA's assign you an "A" ). There's intra-coder reliability if I read your paper 4 times and assign an "A" every time. There's inter-item reliability if I break up your paper into sections (e.g., literature review, hypotheses, results) and grade every section separately (so I break the paper into separate "items"), and assign every section an "A".

inter-coder: all coders code the same as each other
intra-coder: one coder codes the same every time
inter-item: each "item" (in, e.g., a self-esteem questionnaire) gives the same result as every other item (in the questionnaire)

Q: For midterm practice qu. #10, why is (b) the correct answer?

A: (a) is true because it was a representative sample, so you can generalize to the population you drew the sample from.

(b) is NOT true, because the only way you can make a causal claim is with the experimental method. In other words, to make a causal claim, you'd need to manipulate a variable (e.g., manipulate beverage type by giving one group beer and the other water), randomly assign participants to one group for each "level" of the variable (1 group to the beer condition and 1 group to the water condition), and control as many outside factors as possible (e.g., it would be bad/uncontrolled it experimenters were extra nice to one of the groups but not the other). The study in qu. #10 just asks everyone the same questions (there are no separate conditions), so this is the correlational/survey method and you therefore can't make causal claims.

(c) is true. You've measured 2 variables (drinking habits & support for alcohol on campus), so you can assess the relationship. As long as you measure 2 variables (regardless of whether the design is a content analysis, survey/correlational study, or experiment), you can see how the 2 variables are related. (But if you assessed the relationship with an experimental design, you could assess the _causal_ relationship, and not just the "relationship.")

So (a) and (c) are both true, which means that (b) is the only one that is NOT an advantage of the design.

Q: For midterm practice qu. #12, what's a "longitudinal design"?

A: A longitudinal design just involves collecting measures over time (over the course of days, months, or years). By contrast, a cross-sectional design involves measuring all variables in one shot, on one day, because they're all variables from the present or past.

The 2 variables in (a) (education level, support for drug legalization) can both be measured at the same time -- they're _current_ levels of the variables.

The 2 variables in (b) (era during which one was raised, trust for government) can both be measured at the same time -- they're current (trust for government) or past (when you were raised).

The 2 variables in (d) (level of income, preference for newspapers over TV) can both be measured at the same time -- they're both current.

The 2 variables in (c) (awareness of advertised products, progression of the season) CANNOT both be measured at the same time. Current awareness of advertised products is in the present. But "progression" isn't in the present or past (in which case you'd be able to measure it at the same time as awareness and be done with it), it's in the present and _future_. To find levels of awareness as the season progresses, you'd need to measure awareness several times _as the season progresses_, i.e., over time, i.e., longitudinally. So the answer is (c).

Q: For midterm practice qu. #14, why is (b) the correct answer?

A: Self-esteem and eye contact refer to the same variable -- eye contact is the _measure_ (operationalization) of the variable "self-esteem". If that measure were used to _predict_ something, e.g., GPA in college, then you could criticize the measure's _predictive validity_ if you found that eye contact does not actually predict GPA in college. (Predictive validity isn't about how well a measure "predicts" its own variable, it's about how well the measure predicts another variable.) But that's not what's being criticized here.

The criticism given in this question is just about how well the measure does (or in this case, doesn't) "fit" the variable -- it criticizes the validity of the measure. It doesn't give any concrete info about why the variable is bad. It doesn't say that eye contact is a bad measure
because, e.g.:
1. eye contact doesn't capture the full range of self-esteem (content validity is low)
2. though self-esteem is related to the number of friends a person has, eye contact isn't related to your # of friends (construct validity is low)
3. self-esteem doesn't predict GPA in college (predictive/criterion validity is low)

It just gives one person's opinion that it doesn't seem, on the surface of things, that eye contact is a good measure of self-esteem. So the answer is (b), face validity.

Q: For midterm practice qu. #20, why is the sample not a stratified sample? Didn't stratification occur because people with and without phones were automatically different?

A: In stratification, you divide people into strata, and then select people from each stratum (randomly, for random stratification). The number of people you select from each stratum is determined by the proportion of each stratum in the population. E.g., if you wanted to do a stratified sample of men and women in Comm 88, you'd need to get the course roster, divide everyone into a "male" or "female" category, and determine what % of the class is male. If you found that the class is 50% male, and you wanted a sample size of 10, then you'd select 5 people from the "male" list and 5 people from the "female" list.

Stratification (a) is purposeful (if you just happened to get 50% males and 50% females, then that's chance, not stratification), (b) requires that you have a complete list of everyone in each stratum and know which stratum everyone belongs to, (c) requires previous knowledge about proportions in the population (e.g., you'd need to know what % of people don't have phones), and (d) involves collecting data from _all_ strata (so in the case of this question, it would involve collecting data from people without phones as well as people with phones).

In this Gallup study, there was no purposeful prior division into groups, there was only 1 group that was sampled (people whose phone numbers were randomly selected), and nothing was mentioned about people without phones. (Careful you don't add anything into the question that's not there!) So it's not stratified. Instead, it's a "simple random sample" because people were randomly selected from a population: Californian adults with phones.

(OK, you could technically say that e.g., people with multiple phone #'s have a greater chance of being contacted than people with 1 phone # for the entire family, but that's getting too technical. This is basically a simple random sample.)

Good luck on the exam, everyone!