April 22 – Prediction

Readings:

  • Daniel Kahneman and Amos Tversky, “On the Psychology of Prediction”, Psychological Review 80(4):237-251, 1973
    • ‘Prediction’ as a task of judgment or scientific modeling has a slightly different meaning from the common one. It does not always have to be about future events.
    • The nominal prediction examples in this paper are illustrations of what became  known as “base rate neglect” – ignoring or dramatically underweighting prior probabilities in subjective probability updating under Bayes’s Theorem.
    • Both base rate neglect and nonregressive prediction are manifestations of the general tendency of intuitive predictions to underweight background knowledge. Intuitive judges also pay insufficient attention to the reliability of evidence for predicting an outcome.
    • The paper is important for scientific prediction both because it tells us something about the biases of scientists, and because it showcases the difference between theoretical models (probalistic, regression) and intuitive judgment. In the problems discussed in this paper, the former are normative/more accurate, and the latter are descriptive.
  • Daniel Kahneman and Gary Klein, “Conditions for Intuitive Expertise: A Failure to Disagree”, American Psychologist 64(6):515-526, 2009
    • This paper shows the difference between the heuristics and biases (HB) approach to judgment, and the naturalistic decision making (NDM) approach. It also shows how they can be joined in a synthesis, which is summarized in the last section.
    • The authors discuss numerous studies that have compared intuitive prediction by experts with algorithmic procedures (p. 523). Studies favoring algorithms vastly outnumber those favoring human experts. But there is a sweet spot for human expertise when the environment has highly valid cues that can be learned by a decision maker with enough practice.
    • The premortem method (p. 524) is an example of a debiasing procedure for human judgment.

Slides: http://www.stanford.edu/class/symsys130/SymSys130-4-22-2013.ppt.pdf. An additional note:

  • Regression toward the mean is often stated as a mere consequence of imperfect  correlation.   But it also depends on the assumption of a linear estimate, i.e. a linear model as defined  above.  For nonlinear models (e.g. cubic models), an unbiased estimate of one variable based on  another may not imply regression to the mean.  Regression to the mean is therefore an empirical  phenomenon which is implied by the linear model.  It may not occur in reality if a nonlinear model  better fits observations than a linear one.  See http://www.stanford.edu/class/symbsys170/2010/Induction.pdf.
Advertisements
This entry was posted in Class Sessions. Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s