The regression-to-the-mean effect has also led to confusion in treating diseases whose severity fluctuates over time or from which people naturally recover in time. Doctors give treatments when symptoms are more severe, and if the severity fluctuates over time, then we should expect patients to improve without treatment, simply by waiting. Many quack treatments and pseudoscientific approaches capitalize on this. You wait until someone’s symptoms are bad and then give them the medicine. And, lo! The symptoms ease up, and the quack claims it’s all due to the medicine. This is why randomized controlled trials are so important. In such a trial, there are two equivalent groups of patients. One group receives the purported treatment and the other receives a placebo, or nothing at all, with neither patients nor researchers knowing which group received which. If the symptom alleviation is purely due to regression to the mean, and not to the treatment, then the two groups will recover at the same rate. An almost comic example of how regression to the mean can be misunderstood, and alternative explanations conjured up to explain something we should in fact expect to occur, is given by Arthur Koestler in his book The Roots of Coincidence. He wrote: “Even the most enthusiastic experimental subjects showed a marked decline in hits towards the end of each session, and after some weeks or months of intense experimenting most of them lost altogether their special gifts. Incidentally, this ‘decline effect’ (from the beginning to the end of a session) was considered as additional proof that there was some human factor at work influencing the scores, and not just chance.” The regression-to-the-mean effect is ubiquitous. Once you’ve been alerted to the phenomenon, you can see it everywhere. It occurs whenever the score, outcome, or response has a random component. Take performance, for example—in an examination, a test, a workplace, sports, or whatever. While performance clearly does depend partly on intrinsic ability, preparedness, and other factors, it also owes something to chance. Perhaps you were feeling particularly good on the day, or the questions on the exam just happened to be on the topics you’d anticipated, or the representatives from the prospective client turned out to be old high school friends. The chance aspect of your good performance is likely to fade away the next time, so that it looks as if you have deteriorated. Regression to the mean signals that caution must be exercised in taking the results at face value: an extreme score may well be so primarily because of chance. There’s also a flip side to this. If extremely good performance owes something to favorable chance, then particularly poor performance likewise owes something to unfavorable chance. All of this has obvious implications for just about any kind of ranking (of sports teams, surgeons, students, universities, you name it): if a high position owes much to chance, it’s likely to be followed by a lower position next time.
Regression to the mean. This probably is a concept that all you enlightened readers already know, but allow me to just expound on it anyways.
Truth is, most of the things we do in our careers are highly based on exogenous factors, or luck. At work, we’re reliant on finding the right mentors, having the right projects or being at the right place at the right time. Good performance in careers is not something as simple as say playing darts or bowling, where the number of variables are vastly lower and luck dosen’t play such a huge role.
Once luck is involved, then we have to expect regression to the mean. The top performer for 1 year might suddenly not able to perform when put in a different team or environment. The worst performer, would highly likely tend towards average performance. Adopting this into our mental models will help us double-check our initial judgements so we can be more accurate in our judgements, especially when you’re hiring or interviewing.
