The Missing Value Problem?

In statistics, artificial intelligence, and many computational models of mind, missing values are perceived as a nuisance which must be replaced or eliminated before proceeding with the business of inference. In contrast, in interactions between minds and natural environments, patterns of missing knowledge carry information which can serve as the basis for intelligent inference. Consider the eating habits of wild rats which come to recognize foods from their own diet, or from smelling food traces on the breath of conspecifics (Barnett, 1963). These animals have been shown to exhibit a strong neophobia, that is, a reluctance to eat foods they do not recognize. This preference can be so strong that rats may prefer recognized foods over novel ones even if they recognize them from the breath of neighbors who appear to be suffering from food poisoning at the time (Galef, 1987; Galef, McQuoid, & Whiskin, 1990). While this behavior seems like a bias, recognition-based food choice helps rats avoid poisoning: any food a living rat or its living neighbors has chosen was clearly not deadly. Missing knowledge, specifically, a lack of recognition, informs food choice in rats. In this paper, I shift focus from animal behavior to human inference and examine the prevalence and consequences of reasoning by recognition in our own species.
We live surrounded by proper names, some which we recognize, and some of which we do not. The names we recognize are not a haphazard collection, they tend to stand out on various criteria which people find interesting. We may notice that the cities whose names we recognize are often larger than those we do not. The corporations we recognize tend to generate more revenue, and possess a greater market share, than those we do not. The scholars we recognize tend to draw larger audiences than those we do not. How do the names we recognize come to reflect such a variety of variables in the world? Names which are outstanding on criteria that people find interesting are more likely than others to be spoken and written about, and are subsequently more likely than others to become recognized. The probabilistic inference can be made that the recognized names are outstanding and unrecognized ones less so.

Figure 1:
To make inferences about unknown features of the environment, we may exploit the relationship between the presence of items in recognition memory and the criterion. This relationship comes about through mediators in our accessible environment which reflect the criterion

see Figure 1 here

Berlin
Hamburg
Munich
Cologne
Frankfurt
Essen
Dortmund
Stuttgart
Düsseldorf
Bremen
Duisburg
Hannover

Measuring the Relationship Between Memory, Mediator, and Environment
This paper deals with a model domain for inference: the set of cities in Germany with more than 100,000 inhabitants (83 cities in 1994). To measure the strengths of the various correlations in this domain I compared the populations of these cities to the number of articles in which they were mentioned in the last 12 years of the Chicago Tribune, and the percentage of 67 University of Chicago students who recognized them (also in 1994). Table 1 shows the 12 largest cities in Germany (in order of 1994 population) taken from the complete data. Computing Spearman correlations over all 83 cities, the surrogate correlation was .79, the ecological correlation was .70, and the recognition correlation was .60. Recognition memory is more in tune with how frequently cities are mentioned in media than with the actual populations of cities. This is to be expected since the newspapers, and not city populations, are a part of our everyday environment. Given that there are solid relationships between recognition memory and socially-interesting criteria, how might the mind exploit this fact to make inferences?

The Recognition Heuristic

I turn now to a precise model of how people draw inferences from recognition memory. The task is inferring which subset of a class of objects scores highest on some criterion. For sets of two objects, this amounts to the two-alternative forced-choice paradigm. An example question would be: Which river is longer? A) The Nile B) The Isar. For questions of this sort, the recognition heuristic is simply stated:

If only one of the alternatives is recognized, then choose it.

Naturally, this heuristic is only sensible in domains where recognition is correlated with the criterion. If the correlation is negative (as it would be in the task of inferring which of two cities is smaller, for instance), then the unrecognized alternative should be chosen. The recognition heuristic seems like a bias or overly simple strategy. Is it?

Accuracy
The accuracy attainable from the recognition heuristic depends on two variables: how often the heuristic can be applied, and the degree of correlation between recognition and the criterion of interest. As a thought experiment, consider the case of an American who learns, one by one, to recognize the 100 largest cities in France. Every time she learns a new city, she is given a test consisting of all possible pairs of cities drawn from the 100 largest, and her task is to pick the larger city in each pair. Before she has learned any of the cities, she will have to guess on every pair, and thus score 50% correct, as represented by the leftmost point in Figure 2.

Figure 2:
Applying the recognition heuristic may lead considerable accuracy, as well as a less-is-more effect

see Figure 2 here

A Simple Test of the Recognition Heuristic
This simple test asks how often unprompted people will use the recognition heuristic (Goldstein, 1996). I quizzed people on all pairs of cities drawn from the 25 (n=6) or 30 (n=16) largest in Germany (300 or 435 questions) and asked them to choose the more populous city in each case. Either before or after the test, the participants were asked to check off from a list which of these cities they recognized (order, however, had no effect). From this recognition information, I calculated how often participants had an opportunity to choose in accordance with the recognition heuristic, and compared it to how often they actually did. Figure 3 shows the results for 22 individual participants.

Figure 3:
How often participants made choices in accordance with the recognition principle

see Figure 3 here

A Tougher Test of the Recognition Heuristic
In this experiment, participants were taught useful information that offered an alternative to following the recognition heuristic (Goldstein, 1996). The information was about the presence of major league soccer teams, which are powerful predictors of city population in Germany. The objective was to see which people would choose as larger: an unrecognized city, or a recognized city that they just learned has no soccer team.
The experiment began with a training session during which participants were instructed to write down all information that would follow. They were first told that they would be quizzed on the populations of the 30 largest cities in Germany. Next they were taught i) that nine of the 30 largest cities in Germany have soccer teams, ii) that the nine cities with teams are larger than the 21 cities without teams in 78% of all possible pairs, and iii) the names of four well-known cities that have soccer teams, as well as the names of four well-known cities that do not. Participants were then tested to make sure they could reproduce all of this information exactly and could not proceed with the experiment until they did so. Either before or after the main task, participants were shown a list of German cities and asked to mark those that they recognized before coming to the experiment.
With their notes beside them, participants were then presented pairs of cities and asked to choose the larger city in each pair. To motivate them to take the task seriously, they were offered a chance of winning 15 dollars if they scored more than 80% correct. To reiterate, the point of the experiment was to see which participants would choose as larger: a city they have never heard of before, or one which they recognized beforehand but just learned had no soccer team. From the information presented in the training session (which made no mention of recognition), one would expect the participants to choose the unrecognized city. Why? An unrecognized city either has or does not have a soccer team. If it does (a 5 in 22 chance from the information presented), then there is a 78% chance that it is larger. If it does not, then there is an equal chance it is larger. The unrecognized city should be favored because any chance of it having a soccer team suggests that it is probably larger. Figure 4 shows the results.
The graph reads the same as Figure 3. The left-hand bars are of different heights because individual participants recognized different cities before the experiment, so the number of cases where the recognition heuristic applied varied. Twelve of 21 participants made choices in accordance with the recognition heuristic without exception, most others deviated on only one or two items. All in all, participants followed recognition in 273 of the 296 total critical pairs. The median proportion of inferences agreeing with the heuristic was 100% (mean 92%), despite conflicting knowledge. It appears that the additional information was not integrated into the inferences, consistent with the recognition heuristic.

Figure 4:
Recognition heuristic adherence despite training that encouraged the use of information conflicticting with recognition

see Figure 4 here

The Recognition Heuristic As a Prototype of Fast and Frugal Heuristics

The recognition heuristic is one of many fast and frugal heuristics which organisms can use under limited time, knowledge, and computational might (Gigerenzer & Goldstein, 1996). Since it uses recognition memory, a fundamental psychological mechanism, and profits from missing knowledge, the recognition heuristic is perhaps the simplest of these adaptive tools. Reasoning by recognition is a form of one-reason decision making (Goldstein, 1996) because it bases complex inferences on recognition alone. This conflict avoiding strategy eliminates the need to make trade-offs between cues pointing in opposing directions, a well documented desire of human decision makers (e.g., Baron, 1990; Hogarth, 1987; Payne, Bettman, & Johnson, 1993). In the interaction between minds and real-world environments, patterns of missing knowledge carry important information which organisms can exploit to make inferences. As missing knowledge is filled in, the usefulness of recognition is diluted, and the accuracy and efficiency of inferences may decline.

Acknowledgments

I wish to thank Valerie Chase, Jean Czerlinski, and Gerd Gigerenzer for their comments, and The University of Chicago, The Max Planck Institute for Psychological Research, and The Max Planck Institute for Human Development for financial support.

References

Barnett, S. A. (1963). The rat: A study in behavior. Chicago: Aldine.

Baron, J. (1990). Thinking and deciding. Cambridge, UK: Cambridge University Press.

Brunswik, E. (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62, 193-217.

Galef, B. G., Jr. (1987). Social influences on the identification of toxic foods by Norway rats. Animal Learning & Behavior, 18, 199-205.

Galef, B. G., Jr., McQuoid, L. M., & Whiskin, E. E. (1990). Further evidence that Norway rats do not socially transmit learned aversions to toxic baits. Animal Learning & Behavior, 18, 199-205.

Gigerenzer, G. & Goldstein, D. G. (1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 104, 650-669.

Goldstein, D. G. (1996). Models of bounded rationality for inference. Doctoral dissertation, Department of Psychology, The University of Chicago.

Goldstein, D. G. & Gigerenzer, G. (1998). Recognition: How to exploit a lack of knowledge. In G. Gigerenzer & P. Todd (Eds.), Ecological rationality: Simple heuristics that make us smart. New York: Oxford University Press.

Hogarth, R. M. (1987). Judgement and choice: The psychology of decision. 2nd ed. Chichester: Wiley.

Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge: Cambridge University Press.

Daniel G. Goldstein
Stanford University
Department of Engineering-Economic Systems & Operations Research
Terman Engineering Center
Stanford, CA 94305-4027 USA
dang1@leland.stanford.edu