May 1 – Testing and Surveying

Readings:

  • Gerianne de Klerk, “Classical Test Theory (CTT)”, In M. Born, C.D. Foxcroft & R. Butter (Eds.), Online Readings in Testing and Assessment, International Test Commission, 2008
    • deKlerk says that squaring a validity coefficient and multiplying it by 100 yields “the explained variance in a relationship. To explain this more fully,” he says, “the following example may assist: when the predictive validity coefficient of an ability test in relation to the criterion of job retention is 0.70 (i.e.the number of years the person will stay in the job) it means that 49% ((0.70)²*100 = 49%) of the differences in job retention (the criterion) can be explained (or predicted) by differences in the test achievements.” This latter statement is suspect. The coefficient explains 49% of the variance, but this is not the same as 49% of the differences in test achievements. See the Wikipedia article on “variance”.
  • Jon A. Krosnick, “Survey Research”, Annual Review of Psychology 50:537-567, 1999
    • This is an excellent survey (no pun intended) of the research on surveying, but what Krosnick calls “optimizing” versus “satisficing” behavior is imposing some spin on the analysis. It seems to assume that when people are doing less than the optimal in responding, they are doing something that is good enough for some purpose (a.k.a. “satisficing”, a term popularized by Herbert Simon). A more neutral perspective would be to map “satisficing” in his analysis onto the division of mental processes into Kahneman’s System 1/”thinking fast” and to map “optimizing”onto System 2/”thinking slow”.

Slides: http://www.stanford.edu/class/symsys130/SymSys130-5-1-2013.ppt.pdf. A few points:

  • In addition to Classical Test Theory, which we discussed in class and which was introduced in the deKlerk reading, you might also want to look at Item Response Theory.
  • Test theory was developed for intelligence testing, but it can be applied to lots of different types of tests, e.g. those measuring attitudes, personality traits, beliefs, etc.
Advertisements
This entry was posted in Class Sessions. Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s