Human Responses to Algorithmic Correction: Evidence from Centre Court

arXiv:2401.16754 · cs.LG, cs.CY, econ.GN, q-fin.EC · Submitted 2024-01-30 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Human Responses to Algorithmic Correction".

Jane: Here's the gist of the paper. The gist: AI oversight, specifically Hawk-Eye review in tennis umpires, has been shown to cause umpires to care 37% more about Type II errors after its introduction because being overruled by AI carries psychological costs <ref:2401.16754#pg21>.

Tom: How it works The study investigates the impact of Hawk-Eye review, an AI technology used to predict ball landing locations, on umpires' decision-making in top tennis tournaments <ref:2401.16754#pg4>.

Jane: The researchers found that after the introduction of Hawk-Eye review, umpires lowered their overall mistake rate by 8% (1.1 p.p.) for close calls <ref:2401.16754#pg4>.

Tom: However, this aggregate decrease masked heterogeneity in error types <ref:2401.16754#pg4>.

Paper summary: Tom: So we're looking at this paper by Almog, Gauriot, Page, and Martin titled "Human Responses to Algorithmic Correction: Evidence from Centre Court". The main idea here is they provide the first field evidence that when you use AI oversight, it actually changes how people make decisions in real life.

Jane: That's right. They look at Hawk-Eye review of tennis umpires in top tournaments and find that the introduction of this technology has a measurable impact on umpire behavior. The core claim is that umpires didn't just get better overall, but their specific types of errors actually shifted after the AI became present.

Lu: What's interesting is they found that while the overall mistake rate dropped, there was a clear trade-off in what kind of mistakes were being made, specifically a shift from calling balls out when they were in to calling balls in when they were out.

Meng: From a production standpoint, that sounds like the AI is making umpires more cautious about being wrong on the 'in' side, because the cost of being overruled by an AI prediction is higher than just making a simple human error.

Lalam: I see. They found that because of these costs, umpires cared thirty-seven percent more about Type II errors under AI oversight, which is calling a ball out when it was actually in > <ref:2401.16754#pg1,that because of these costs, umpires cared 37% more about Type II>

Tom: Exactly. That thirty-seven percent increase in care for those specific errors is what they attribute to the psychological costs of being overruled by the AI system > <ref:2401.16754#pg1,the psychological costs of being overruled by>

Jane: And that leads into how they modeled this. They used a model of attention-constrained umpires to structurally estimate these psychological costs, accounting for the trade-off between cognitive and psychological costs >

Lu: The methodology involves a two-stage approach where they look at decisions before and after Hawk-Eye was introduced to estimate those parameters, which is pretty rigorous work >

Meng: I wonder what the real cost of that attention constraint is when you're trying to deploy this on a massive scale. What breaks at scale if an umpire has that kind of cognitive load?

Lalam: The paper suggests that the optimal information structure was determined by the Invariant Likelihood Ratio conditions, which allowed them to estimate that AI oversight penalty >

Tom: So, it sounds like they're not just looking at accuracy numbers but trying to model the internal decision-making process of a human under pressure from an AI system >

Jane: They also looked at how this response varied based on skill, stakes, and the type of perceptual task involved, which showed that the error shift was more pronounced for balls closer to the line during matches in the early stages >

Lu: And they distinguished between serves and non-serves too; they found no evidence of AI oversight impacting umpire performance during serves but a two point three percentage point decrease in incorrect calls for non-serves >

Meng: That's interesting because it suggests that maybe the potential for improvement is higher on non-serves, which makes sense if the technology isn't as immediately dominant there >

Lalam: And they conclude that when balls landed very close to the line, umpires were more inclined to call them in as an attempt to minimize the occurrence of the more costly type of error >

Tom: So, it seems like when things are tight and close, umpires actually lean toward calling things in instead of out because they are trying to avoid that higher cost error >

Jane: That's a really important point for us to consider because it shows how the system can actually misalign human incentives when the goal is just minimizing a specific type of mistake >

Lu: The authors suggest two main reasons why we need to approach AI oversight with caution: the free-riding motive and an incentive misalignment motive >

Meng: Free-riding means umpires might stop trying hard because they think the AI will recognize their mistakes anyway, which is a big deployment risk if you can't monitor that behavior >

Lalam: The second reason is that implementation guidelines can shift incentives in a way that reduces overall decision quality, which is what we saw happen here when balls were close to the line >

Tom: So it boils down to whether the correction itself improves things or if the structure of the correction just pushes us toward a different kind of mistake >

Jane: And for anyone listening who isn't in sports, this paper really highlights how type I and type II errors can have very different consequences in real settings, like medical or judicial contexts >

Lu: They use this tennis example to build an initial understanding of the implications of AI oversight on humans, which is a starting point for more complex modeling >

Meng: I'm thinking about what this means for our own systems; if we introduce correction mechanisms that change human behavior this much, we have to build in safeguards against those incentive shifts >

Lalam: The paper itself suggests that the shift in error types can be highly detrimental in contexts like medical and judicial settings where Type I and Type II errors have widely differing implications >

Conclusion: Tom: So we’ve looked at how Hawk-Eye review in tennis changed umpire decisions—specifically that when things are close, umpires start calling balls in more often because of the penalty for being wrong on those tight calls.

Jane: That shift is really interesting because it shows that the correction itself can actually push people toward a different kind of mistake, which is something we don't always anticipate when we build these systems.

Lu: The authors are David Almog and Romain Gauriot, and they’re using Hawk-Eye data from professional tennis matches to test this idea about how human response changes after AI oversight is introduced.

Meng: I’m looking at the authors, and they seem to be focusing heavily on the psychological costs that umpires have when an AI system flags their call as wrong.

Lalam: And what they find is that because of those costs, umpires care thirty-seven percent more about Type II errors—calling a ball out when it’s actually in—after the AI is in play.

Tom: That thirty-seven percent figure is the number that really grabs you; it’s not just a general improvement, it’s a specific change in how they weigh different types of mistakes.

Jane: The authors are suggesting this isn't just about accuracy numbers; it’s about how the structure of the correction itself can misalign human incentives, which is something we need to think about when we deploy any kind of oversight system.

Lu: They conclude that this shift in error types can be a problem in fields like medicine or law where Type I and Type II errors have very different kinds of consequences.

Meng: It means if you design an AI correction system, you’re not just fixing mistakes; you’re changing the human's internal risk profile, which is a serious thing to model when scaling up production.

Lalam: So the paper suggests that we need to be careful about how we set those correction rules because they can actually push us toward less ideal decisions under specific conditions.

Tom: Exactly. It’s a reminder that the cost of being overruled by an AI isn't just technical; it’s psychological, and understanding that human factor is what makes this research so relevant for production engineers.

David Almog, Romain Gauriot, Lionel Page, Daniel Martin

Kellogg School of Management, Northwestern University · Deakin University · University of Queensland · University of California, Santa Barbara

cs.LG, cs.CY, econ.GN, q-fin.EC

Submitted: 2024-01-30

Updated: 2026-10-18

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 87/100

The gist: The gist: AI oversight, specifically Hawk-Eye review in tennis umpires, has been shown to cause umpires to care 37% more about Type II errors after its introduction because being overruled by AI

Key concepts

Hawk-Eye Review: This is an AI technology used to predict exactly where a tennis ball will land on the court before it bounces. Umpires use this data to check their calls, and when they disagree with the AI's prediction, they are subject to review. This system introduces a new layer of external judgment into the decision-making process.
Type II Error: This error occurs when an umpire incorrectly calls a ball 'out' (meaning it was actually 'in'). The study found that after Hawk-Eye review, umpires paid significantly more attention to avoiding this specific mistake. This is because being overruled by the AI feels psychologically costly to the umpire.
Incentive Misalignment Motive: This concept refers to situations where the rules or incentives of a system encourage people to behave in a way that reduces overall quality. In this study, it suggests that when umpires feel pressured by AI oversight, their focus shifts away from optimal decision-making toward simply minimizing the risk of being caught making an error.
Psychological Costs: These are the mental or emotional burdens an individual experiences when their actions are judged or corrected by a system, like Hawk-Eye. The researchers modeled how these costs affect umpires' attention and decision-making, showing that the fear of being wrong in front of AI influences how carefully they make subsequent calls.

Terminology

Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes

Summary

The gist: AI oversight, specifically Hawk-Eye review in tennis umpires, has been shown to cause umpires to care 37% more about Type II errors after its introduction because being overruled by AI carries psychological costs <ref:2401.16754#pg21>.

How it works

The study investigates the impact of Hawk-Eye review, an AI technology used to predict ball landing locations, on umpires' decision-making in top tennis tournaments <ref:2401.16754#pg4>. The researchers found that after the introduction of Hawk-Eye review, umpires lowered their overall mistake rate by 8% (1.1 p.p.) for close calls <ref:2401.16754#pg4>. However, this aggregate decrease masked heterogeneity in error types <ref:2401.16754#pg4>.

The key behavioral shift observed is a change from making Type II errors (calling a ball out when in) to Type I errors (calling a ball in when out) for the closest calls <ref:2401.16754#pg4>. This shift was driven by the asymmetric psychological costs of being overruled by AI <ref:2401.16754#pg4>. Specifically, improperly stopping a point (calling a ball out when it was actually in) became a new cause of concern with the introduction of Hawk-Eye <ref:2401.16754#pg4>.

Structural Estimation of Psychological Costs

The researchers structurally estimated the psychological costs of being overruled by AI using a model of attention-constrained umpires <ref:2401.16754#pg4>. This model accounts for the trade-off between cognitive costs and psychological costs <ref:2401.16754#pg4>. The analysis utilized a two-stage approach to estimate these parameters, using decisions before and after Hawk-Eye was introduced <ref:2401.16754#pg4>.

The results suggested that these psychological costs lead umpires to care 37% more about Type II errors (calling a ball out when in) after the introduction of Hawk-Eye review <ref:2401.16754#pg5>. This finding is supported by the estimation of the AI oversight penalty, which represents the umpire’s disutility from having been caught making a mistake by the AI system <ref:2401.16754#pg4>.

Heterogeneity in Response

The study examined three main sources of heterogeneity: skill, stakes, and type of perceptual task <ref:2401.16754#pg6>. The analysis showed that the shift in error types was more pronounced and only significant for matches in the early stages for balls closer to the line <ref:2401.16754#pg4>.

The analysis also distinguished between serves and non-serves, finding that while there was no evidence of AI oversight impacting umpire performance during serves, there was a 2.3 percentage point decrease in incorrect calls for non-serves <ref:2401.16754#pg29>. This suggests that non-serves may have a higher potential for wholesale improvement <ref:2401.16754#pg4>.

Implications and Conclusion

The findings suggest two central reasons to approach AI oversight with caution: the free-riding motive, where technology can dissuade effort under the belief that mistakes will be recognized by the technology anyway <ref:2401.16754#pg4>. The second reason is an incentive misalignment motive, where implementation guidelines shift incentives in a way that reduces overall decision quality <ref:2401.16754#pg4>. The analysis indicates that when balls landed very close to the line, umpires were more inclined to call them in as an attempt to minimize the occurrence of the more costly type of error <ref:2401.16754#pg4>.

The paper concludes by suggesting that this shift in error types can be highly detrimental in contexts like medical and judicial settings where Type I and Type II errors have widely differing implications <ref:2401.16754#pg4>. The research serves as an initial building block for understanding the implications of AI oversight on humans <ref:2401.16754#pg4>.

Data Sources

The study utilized a consolidated data set comprising 698 matches across 35 distinct tournaments, merging data from three sources <ref:2401.16754#pg11>. The primary source is the Hawk-Eye Base data set, which provides precise information on points and bounce locations for over 1,800 men’s singles matches <ref:2401.16754#pg23>. The second data set is the Challenge data set, tracking outcomes of challenges during tournaments following the introduction of Hawk-Eye <ref:2401.16754#pg24>. Finally, video auditing was used to identify ground truth for challenges in a subset of matches <ref:2401.16754#pg39>.

Merging Algorithm

The merging algorithm achieved a merge rate of 99.3% out of 144 challenges audited <ref:2401.16754#pg38>. The algorithm involved eight iterations to minimize false positives, with the final iteration achieving a 40% efficacy for challenges left unmerged <ref:2401.16754#pg39>. The criteria for each iteration were designed to progressively relax constraints based on factors like set, game, score, and distance difference <ref:2401.16754#pg38>.

Mistake Identification Criteria

The study identified potential mistakes using four criteria applied to the data <ref:2401.16754#pg40>. Criterion 1 selected strokes where the first player to hit a ball out in a given point also won the point, with an accuracy rate of 98% <ref:2401.16754#pg39>. Criterion 2 identified points where a player hit a ball out and the point continued after at least three more strokes, yielding an accuracy rate of 96% <ref:2401.16754#pg39>. Criterion 3 identified strokes where all strokes of a point were in and the last player to hit lost the point, with a 92% accuracy rate <ref:2401.16754#pg40>. Criterion 4 identified points where there was a second serve after the first serve was in, with an accuracy rate of 86% <ref:2401.16754#pg40>.

Attention Model

The researchers extended the canonical model of rational inattention by incorporating costs that scale linearly with Shannon mutual information <ref:2401.16754#pg37>. This extension allows for asymmetric costs of attention for different states, such as when the ball is in or out <ref:2401.16754#pg39>. The optimal information structure was determined by the Invariant Likelihood Ratio (ILR) conditions <ref:2401.16754#pg39>. This allowed for the estimation of the AI oversight penalty, which is the umpire’s disutility from having been caught making a mistake by the AI system <ref:2401.16754#pg40>.

Final Findings on Error Shift

The analysis showed that when balls landed very close to the line, umpires were more inclined to call them in as an attempt to minimize the occurrence of the more costly type of error <ref:2401.16754#pg32>. Overall, the effect of incentive misalignment outweighed incentives to improve performance, leading to an upsurge of incorrect calls during balls that were just out <ref:2401.16754#pg31>.

The analysis suggests that when balls landed very close to the line, umpires were more inclined to call them in, as an attempt to minimize the occurrence of the more costly type of error <ref:2401.16754#pg32>. The shift in types of errors can be highly detrimental in medical and judicial contexts where Type I and Type II errors have widely differing implications <ref:2401.16754#pg34>.

The paper views itself as an initial building block for understanding the implications of AI oversight on humans <ref:2401.16754#pg34>. Future research could be conducted in experimental settings to complement findings <ref:2401.16754#pg34>. The study is a rare example where AI outperforms even the best humans, so the only reasons humans continue to make line calls today are social, labor, and cost reasons <ref:2401.16754#pg4>.

The paper was written by David Almog, Romain Gauriot, Lionel Page, and Daniel Martin <ref:2401.16754#pg2>.

Improvements for AI systems

Increase umpires' focus on Type II errors by designing oversight systems that make improper stopping of a point a new cause of concern, as improperly stopping a point (calling a ball out when it was actually in) became a new cause of concern with the introduction of Hawk-Eye.
Implement dynamic threshold beliefs for calls based on distance from the line, allowing umpires to shift their call strategy based on whether a ball is near or far from the boundary, as the only region of the court where the introduction of Hawk-Eye increased the rate at which umpires make mistakes is the two closest bins on the outside of the line.
Develop oversight systems that explicitly account for asymmetric psychological costs, such as shame and embarrassment, by structuring AI feedback to manage these penalties, as this leads to a finding that the AI oversight penalty dominates any utility gain from having the correct call implemented for Type II errors.

Abstract

We study how algorithmic review changes the initial decisions that a system is designed to correct. Using Hawk-Eye data from professional tennis, we compare umpire decisions with and without review against a common accuracy benchmark. Under review, initial errors were 8.6 percentage points higher just outside the line and lower for comparably close balls inside. We find no evidence that average initial accuracy deteriorated. Within matches, reversals were followed by larger adjustments than upheld calls, consistent with adaptation to correction. Overall, correction offsets worse initial decisions for incorrect in calls and reinforces better initial decisions for incorrect out calls. Whether review lowers organizational costs depends jointly on these changes and the costs of remedying corrected errors. A conditional illustration shows how utilization spreads fixed review costs across calls.