- William M. K. Trochim, “Quasi-Experimental Design”, “Relationships Among Pre-Post Designs”, “Qualitative Measures”, and “Unobtrusive Measures”, Research Methods Knowledge Base (October 20, 2006 Edition) <http://www.socialresearchmethods.net>
- A possible confound in regression-discontinuity designs is a Hawthorne effect. If performance on a post-test improves for the bottom 50% of scorers on a pre-test, this is a confounding explanation if the upper 50% of pre-test scorers are given no treatment. A solution is to introduce a placebo-like (hypothesized to be ineffectual) treatment for the high scorers, so that they have an experience of being studied that is similar to the low scorers.
- Trochim’s recoding of the checkmarks into 1s and 0s for the appearance of different themes in an example in the section on “The Qualitative Debate” may not lead to meaningful totals in the addition of 1s and 0s, e.g. if the texts or qualitative data being coded are not a random or representative sample of anything, but are instead chosen to illustrate variety, as is often the case in qualitative analysis.
- OPTIONAL: Norm Scheiber, “Freaks and Geeks: How Freakonomics Is Ruining the Dismal Science”, The New Republic, April 2, 2007 (with a follow-up exchange between Steven Levitt and Norm Scheiber)
Slides: http://www.stanford.edu/class/symsys130/SymSys130-5-8-2013.ppt.pdf. A few points:
- The “example scenarios” can be used to illustrate the appropriateness of different methods of observation, e.g. a student who wants to know whether studying with music improves test performance might do a regression-discontinuity study in which the bottom half of scorers on a test are told to listen to music while studying for the next test. If the regression line shifts up or down for the bottom testers relative to the upper range ones, this is evidence the manipulation proved effective or harmful, depending on the result. Mike Krieger, a SSP alum who co-founded Instagram, recommends to startup entrepreneurs that they undertake informal ethnographic observations to get an idea of how consumers shop.