Two researchers, Edward Vul and Harold Pashler, had the idea of asking people to answer this question (and many similar ones) not once but twice. The subjects were not told the first time that they would have to guess again. Vul and Pashler’s hypothesis was that the average of the two answers would be more accurate than either of the answers on its own. The data proved them right. In general, the first guess was closer to the truth than the second, but the best estimate came from averaging the two guesses. Vul and Pashler drew inspiration from the well-known phenomenon known as the wisdom-of-crowds effect: averaging the independent judgments of different people generally improves accuracy. In 1907, Francis Galton, a cousin of Darwin and a famous polymath, asked 787 villagers at a country fair to estimate the weight of a prize ox. None of the villagers guessed the actual weight of the ox, which was 1,198 pounds, but the mean of their guesses was 1,200, just 2 pounds off, and the median (1,207) was also very close. The villagers were a “wise crowd” in the sense that although their individual estimates were quite noisy, they were unbiased. Galton’s demonstration surprised him: he had little respect for the judgment of ordinary people, and despite himself, he urged that his results were “more creditable to the trustworthiness of a democratic judgment than might have been expected.” Similar results have been found in hundreds of situations. Of course, if questions are so difficult that only experts can come close to the answer, crowds will not necessarily be very accurate. But when, for instance, people are asked to guess the number of jelly beans in a transparent jar, to predict the temperature in their city one week out, or to estimate the distance between two cities in a state, the average answer of a large number of people is likely to be close to the truth. The reason is basic statistics: averaging several independent judgments (or measurements) yields a new judgment, which is less noisy, albeit not less biased, than the individual judgments. Vul and Pashler wanted to find out if the same effect extends to occasion noise: can you get closer to the truth by combining two guesses from the same person, just as you do when you combine the guesses of different people? As they discovered, the answer is yes. Vul and Pashler gave this finding an evocative name: the crowd within. Averaging two guesses by the same person does not improve judgments as much as does seeking out an independent second opinion. As Vul and Pashler put it, “You can gain about 1/10th as much from asking yourself the same question twice as you can from getting a second opinion from someone else.” This is not a large improvement. But you can make the effect much larger by waiting to make a second guess. When Vul and Pashler let three weeks pass before asking their subjects the same question again, the benefit rose to one-third the value of a second opinion. Not bad for a technique that does not require any additional information or outside help. And this result certainly provides a rationale for the age-old advice to decision makers: “Sleep on it, and think again in the morning.”
The wisdom of crowds. So long as we do not fall for the herd effect and allow opinions to cascade and let them remain independent, aggregated opinions usually work better.
In fact, what this excerpt says is that being able to generate a “second opinion” just by your own helps you improve your probability of a right guess. That’s when advice such as “sleep on it” makes sense. When making important and non time-critical decisions, give yourself some time to reset and make another guess.
Key thing is to know when to rely on the wisdom of crowds versus the opinions of experts. For niche topics that require advanced technical knowledge, it perhaps makes more sense to aggregate guesses from a crowd of experts rather than your everyday person.
