Noise

Imagine you are a judge, a bank clerk or a doctor, and your decisions affect the lives of many people. Although you try to be as objective and fair as possible, your ability to make decisions is inevitably influenced by a variety of random factors that you may not even consciously perceive. These factors, known as "noise," represent random variations in judgment that can lead to unfair, inefficient, and unpredictable outcomes. In "Noise," authors Cass R. Sunstein, Daniel Kahneman, and Olivier Sibony explore this phenomenon and its far-reaching implications in various areas of life. In the following summary, we look at eight key ideas that reveal the nature of noise, its impact on human judgment, and possible ways to reduce it.

This phenomenon can be illustrated with the example of two high school students applying to the same prestigious university. Both have excellent grades and their interviews go smoothly. Yet only one of them is accepted. How is this possible? The answer may lie in the weather during the interviews. Research by Uri Simonsohn found that on cloudy days, admissions officers pay more attention to academic performance, while on sunny days they focus more on non-academic qualities such as athletic and artistic talents. Such unpredictable factors can also influence decisions in other areas. Bank clerks, referees in sports competitions, doctors and judges may be subject to the influence of external circumstances such as hunger, fatigue, weather or personal emotions. These factors cause variability in their decisions that has no apparent pattern or cause, and are known as opportunity noise. Decisions can also be influenced by previous experiences or events. For example, if an admissions officer has just seen several exceptional candidates, he or she may be less willing to accept another, even if he or she is equally qualified. This noise is unpredictable and can lead to unfair results. These examples show that even when we try to make objective decisions, our judgments can be influenced by a number of random and unrelated factors. This phenomenon has serious implications for systems that rely on human judgement and highlights the need to find ways to minimise noise and increase the consistency of decision making.

To better understand the differences between these two concepts, we will use an example from a shooting range at a carnival. Imagine that you and a friend are shooting an air rifle at targets. Your shots are scattered all over the target with no clear pattern, which represents noise - random errors with no systematic cause. On the other hand, your friend's shots are clustered, but they are all low and to the left of the target, indicating distortion - a systematic error that has a pattern. In psychology, bias is often used to describe the cognitive mechanisms that lead to systematic errors in judgment. Bias causes our judgments to be consistently wrong in a particular direction. For example, inference bias, which causes our judgments to lean toward a desired outcome, leads to interpreting evidence in a biased way. This kind of systematic error can lead to noise in the whole system. Imagine an immigration court where different judges have different biases. One judge may be stricter and another more lenient. If the decisions of these judges were displayed on a target, they would cluster in different places, introducing bias. However, when we consider all the judges on the court, their different distortions lead to overall noise that causes variability in asylum decisions. Noise in the system can be caused by the different biases of individuals. Each judge or decision maker has his or her own systematic errors that contribute to the overall variability of decisions in the system. Therefore, it is important to distinguish between noise and bias, as addressing one can help mitigate the other. By identifying and correcting systematic biases, we can reduce overall noise and improve the consistency and fairness of decision making.

This phenomenon manifests itself strongly in situations where we need to predict future behavior or outcomes based on limited information. The human mind tends to rely on patterns and stories that intuitively make sense, even when these judgements are often incorrect. An example would be a judge's decision about the amount of bail for a defendant. The judge has certain information, such as the defendant's criminal history and behavior, and must predict whether the defendant will comply with the conditions of release or flee. He often relies on his experience and intuition to make this decision. Research shows, however, that such intuitive decision making can be unreliable and influenced by personal biases or recent experiences. In 2018, a research team led by Sendhil Mullainathan developed an algorithm to predict bail risk. This algorithm, based on objective data, outperformed human judges by reducing the number of inmates and crime committed by released defendants by 24% to 42%. The algorithm was more successful because it was not influenced by intuitive judgments, but analyzed the data systematically and without emotional biases. People are often subject to so-called cognitive confirmation bias when they look for information that confirms their assumptions and ignore information that contradicts them. This bias can affect predictions because we rely on information that intuitively makes sense to us instead of taking into account all relevant facts. Predicting the future is inherently uncertain and relying on intuition can lead to inaccurate and variable results. It is therefore important to recognise the limitations of our intuitive judgements and to use methods and tools that help us make more accurate predictions. Rules and algorithms that are based on objective data can provide more reliable and consistent predictions by eliminating subjective biases and noise.

The human mind has a natural tendency to understand the world through stories. Information that works within a story is better remembered and understood. When we encounter events or decisions that don't form a clear, causal pattern, we tend to overlook or ignore them. Psychological studies have shown that people tend to attribute events to individuals and their characteristics, rather than considering the broader circumstances. This phenomenon, known as the fundamental attribution error, explains why we often create narratives that focus on personal traits and ignore random factors. For example, when we see judicial decisions that seem inconsistent, we tend to explain them based on the personal characteristics of the judges. We may think that one judge is strict while the other is lenient. In this way, we create stories that make sense, when in reality the difference may be due to noise - random and unpredictable variations in judgment. Noise resists our efforts to create stories because it has no clear cause or logic that we can understand. As a result, noise is often overlooked or simplistically explained as bias, a concept that fits more easily into the narratives we create. Noise can only be observed statistically when we analyze a large number of decisions and see variability that has no obvious pattern. Without statistical analysis, these random variations seem insignificant or we attribute them to the wrong causes. Therefore, it is important to acknowledge the existence of noise and its impact on decision-making, even if it does not fit into our stories. Ignoring noise can lead to unfair and inconsistent results, so we must look for ways to identify and minimize it to improve the accuracy and fairness of our decisions.

This principle is based on a phenomenon known as the wisdom of crowds effect, where the combination of many independent judgments leads to a more accurate result than individual judgments. To illustrate this principle, we can go back to a story from 1906, when the polymath Francis Galton visited the county fair in Plymouth. There was a competition where villagers had to guess the weight of an ox. None of them guessed correctly, but when Galton calculated the average of all the guesses, the result was almost exact - it was only one pound off the actual weight. This effect has been observed many times and shows that when we ask multiple people to make independent judgments, the noise in the individual judgments cancel out. Any one person can make a mistake, but the different mistakes made by individuals tend to compensate for each other. In this way, the overall noise is reduced and the resulting average is more accurate. For this principle to work, it is important that each judgment be independent. If people are influenced by the opinions of others, the effect of the wisdom of crowds is lost. Independence ensures that one person's errors and distortions do not affect others. Another important aspect is that all judges must evaluate exactly the same case or question. Only then can the results be correctly averaged and the effect of noise reduction be achieved. Using this principle can be very useful in a variety of fields, from medicine, where we can combine independent diagnoses from different doctors, to the financial sector, where we can average independent risk assessments. In this way, we achieve more accurate and reliable decisions that are less affected by random variation. Averaging independent judgments is a simple but effective way to reduce noise and improve the quality of decisions in many different contexts.

To successfully combat noise, we must first recognize and quantify it. This process starts with a systematic audit of the decisions that are made in different institutions. Marvin E. Frankel, a federal judge in the United States, was one of the first to notice the huge disparities in sentences for similar crimes. He realized that the variability in sentences had no logical justification and often depended on the personal opinions of judges. To better understand the extent of this problem, Frankel initiated research where he asked 208 federal judges to sentence defendants in 16 fictitious cases. The results showed shocking variability in the proposed sentences, confirming the presence of extensive noise. An audit, like the one Frankel conducted, can identify the extent of noise in decision-making. This process involves defining an acceptable level of variability and then mapping the differences between decisions. For example, insurers may analyze differences in claim payouts between different adjusters to determine if there is unwarranted variability. In an audit, it is important to provide arbitrators with consistent scenarios to consider. Each adjudicator should assess the same cases so that their responses can be compared and noise can be identified. After mapping these responses, we can pinpoint where unwarranted variation is occurring. Once the noise is identified, institutions can begin to take action to reduce it. This may include implementing standardized procedures, policies, and training programs to help improve consistency in decision making. For example, insurers can introduce guidelines that clearly define how different types of claims are to be evaluated and assessed, thereby reducing variability between different adjusters. Identifying and making noise visible are essential steps to effectively reduce it. Without a clear understanding of the extent and causes of noise, it is difficult to take effective action to eliminate it. A systematic approach to auditing decisions can reveal hidden patterns of variability and enable organizations to improve the accuracy and fairness of their decisions.

This concept involves establishing systematic procedures and rules that help minimize variability in decision making and ensure more consistent results. Imagine a surgeon who washes his hands thoroughly before surgery to reduce the risk of infection. Like hygienic practices in medicine, decision hygiene requires consistency and discipline. Before making any important decision, it is necessary to stop and think statistically about the issue at hand to avoid making rash and intuitive judgments. One of the key steps of decision hygiene is taking an outside perspective. Rather than delving into the details of a particular case, we should frame the situation in the context of a broader group of similar cases. For example, when deciding on a new CEO, instead of focusing on the personal traits and past achievements of the CEO, it is better to examine general statistics on the success of new CEOs in the industry. Another important aspect is resisting premature intuition. Our intuitions can be misleading and often lead us to the wrong conclusions. Therefore, it is essential to base our decisions on a solid statistical framework and only then add intuitive elements if they are supported by relevant data. Breaking down complex decisions into smaller, independent questions also helps to reduce noise. When different aspects of a problem are considered separately and independently, the errors and biases of one assessor are less likely to be reflected in the overall decision. For example, when evaluating a candidate for a job, different experts may independently assess the candidate's technical skills, experience, and personality traits. Decision hygiene is about establishing structured and systematic procedures that help minimize the influence of random and unpredictable factors on decision making. The implementation of such procedures requires discipline and prevention, much like hygienic practices in medicine. By consistently following these rules, we can reduce noise and achieve more consistent and fair decisions.

For noise reduction measures to be effective, they must be accepted and implemented by all those involved in the decision-making process. One example is the introduction of strict sentencing guidelines in the US federal courts in 1984. This law was intended to reduce the variability in sentences that were imposed for similar crimes. Judges were forced to consider only the crime and the defendant's criminal history, which greatly reduced subjective variability in sentencing. As a result, the disparity in sentence lengths between judges was rapidly reduced. However, many judges were unhappy with this loss of their discretion. They felt that their ability to consider the unique circumstances of each case was limited, leading to their resistance to the new rules. This resistance illustrates how important it is for all involved to accept and support the new noise reduction measures. When the Supreme Court struck down the law on technical grounds in 2005, sentencing variability increased again. An analysis of 4,000 criminal cases convicted after the law was repealed showed that the disparity between severe and lenient sentences doubled. This return to higher noise underscores the importance of the agreement and cooperation of all stakeholders in implementing rules to reduce noise. For noise reduction to become a permanent part of the decision-making process, it is imperative that judges and other decision-makers understand the importance of accuracy over personal statements and individuality. They need to see the extent and cost of noise in the form of inaccuracies and injustices that result. For example, in insurance companies, managers can conduct audits that show how large the differences in claims payouts are between different adjusters. When adjusters understand that such disparities can lead to losses in the hundreds of millions of dollars, they will be motivated to adopt uniform practices. Successful implementation of noise reduction rules requires that all stakeholders work together to formulate practical and system-specific rules that balance noise reduction with other costs and requirements. When all stakeholders feel part of the process, they are more likely to adopt the new rules and reduce noise in their decisions. Consensus and collaboration among all stakeholders are key to effectively reducing noise and improving consistency and fairness in the decision-making process.

The book "Noise" deals with random variations in human judgment that can have serious consequences on decision making. The authors emphasize that our decisions are often influenced by unrelated factors that lead to unpredictable and unfair outcomes. Identifying and making these biases visible are the first steps towards reducing them. The book also argues that establishing systematic procedures and discipline can help minimize variability in decision making. Averaging multiple independent judgments can increase the accuracy and reliability of our decisions. However, the key to successful noise reduction is the agreement and cooperation of all stakeholders so that new rules and procedures are implemented effectively. Overall, the book emphasizes the need to recognize and minimize noise in different areas of life in order to make fairer and more accurate decisions. Do you have feedback for us? We'd love it if you let us know your opinion on our content! Just send an email to feedback@rewio.app with the title of this book in the subject line and share your impressions!