Extended Prolepsis 4.3: Gigernezer to the rescue, continued further

[UPDATE: I decided to add Turkey, which ended up making things look even worse for G&G.]

Perhaps stocks aren’t the best place to use the recognition heuristic.  Gigerenzer & Goldstein (1996, p. 651) refer to the cities task as their “drosophila,” so if the recognition heuristic works anywhere, it should work here.  Despite their impressive results, however, there is cause for concern about the fruit fly’s health.  As Kelman (2011) points out, the recognition heuristic may not work as well when the cities are not in North America or Western Europe.  Goldstein & Gigerenzer (2002, p. 86) somewhat implausibly claim that the fact that they’ve replicated their results for cities in the USA and Germany means that the results “stand up” in “different culture[s].”  In an attempt to see whether this is actually the case, I imitated their methodology for determining ecological correlations for some non-WEIRD countries: in addition to Germany, I looked at cities in Turkey, Argentina, Nigeria, and Thailand.  Ecological correlations reported by Goldstein & Gigerenzer (2002) were .72 for Die Zeit’s mentions of American cities and .70 for the Chicago Tribune’s mentions of German cities.  Instead of the Chicago Tribune, which has a shockingly useless web search function, I used the New York Times, limiting my search to articles published in the first decade of the 21st century.  Otherwise, I followed their methodology exactly.  I first created a list of every city in the relevant country with a population of at least 100,000.  Next, I searched the Times for articles that mentioned the city and the country by name.  I then computed the ecological correlations for each country, as well as a worldwide ecological correlation, in which all cities were included.  The initial results, along with comparative data from Goldstein & Gigerenzer (2002, p. 86) are presented in Table XXX

Country

# of applicable cities

Ecological correlation

Germany

81

.83

Turkey

67

.41

Argentina

42

.77

Thailand

11

.98

Nigeria

73

.86

World

280

.19

Table XXX: Worldwide ecological correlations

A few remarks on these data are in order.  First, I replicated Goldstein & Gigerenzer’s (2002) strong ecological correlation for German cities.  Second, despite the fact that I chose countries from multiple continents, the ecological correlations remained fairly high, with the Thai ecological correlation coming in at a whopping .98.  Third, despite these impressive data, the worldwide ecological correlation was much lower, in large part because goings on in Germany receive much more coverage than those in the Middle East, Asia, Africa, and South America.

The almost absurdly high ecological correlation for Thailand, along with the low worldwide correlation led me to delve a bit deeper into the data.  One thing that became immediately apparent was that much of the strength of these correlations is due to the top few cities’ receiving the lion’s share of media attention.  Turkish cities received 13,634 mentions, of which 3090 went to Istanbul.

Thai cities received 1973 mentions, of which 1510 went to Bangkok.

Nigerian cities received 2657 mentions, of which 1150 went to Lagos.

Argentine cities received 5695 mentions, of which 3320 went to Buenos Aires.

German cities received 172,488 mentions, of which 135,000 went to Berlin.

Could it be that these big cities carried most of the weight of the correlation?  To explore this question, I re-ran the correlations, excluding first the most populous, then the two most populous, then the three, four, and five most populous cities in each zone.  The ecological correlations did not stand up too well to this outlier-removal exercise, as illustrated in Table XXX.

Country

# of applicable cities

Ecological correlation

EC-1

EC-2

EC-3

EC-4

EC-5

Germany

81

.83

.64

.62

.48

.45

.32

Turkey

67

.41

.19

.05

.02

.03

.04

Argentina

42

.77

.50

.43

.44

.31

.15

Thailand

11

.98

-.16

-.04

.14

.35

.56

Nigeria

73

.86

.35

.29

.34

.32

.27

World

280

.19

.25

.30

.35

.38

.41

Table XXX: Worldwide ecological correlations, ex top five cities

A few more remarks are now in order.  First, it appears that, even in the German case, most of the ecological correlation is driven by the top few cities.  The same trend held for all other countries.  Second, this trend was actually reversed for the worldwide ecological correlation, presumably because the methodology removed Istanbul, Lagos, Bangkok, Ankara, and Izmir, which were covered much less than Berlin (#6 in population) despite their somewhat similar size.  Third, although the ecological correlations dissipated, in all but a few cases (Thailand EC-1 and EC-2), they remained positive, though much more modest.

One thing that should now be evident is how very sensitive the drosophila is to slight perturbations.  Is the recognition heuristic a reliable guide to which of two cities is larger?  The answer is that it depends.  It depends on whether the cities are in the same country.  It depends on whether the cities are in the USA or Western Europe, on the one hand, or the rest of the world on the other.  It depends on whether one of the cities is the most populous (or second most populous, or third…) in the entire country.  But if Goldstein & Gigerenzer (2002) are right, people do not take these caveats into account when applying the recognition heuristic.  Results like these should make us wary of accepting Fairweather & Montemayor’s (forthcoming) account of frugal virtues.