When the game is on the line, umpires make the safe call, not the right call

TLDR: "In [the example below], the probability of having an umpire invoke the Infield Fly rule is over 50% when the outcome of the at bat is relatively inconsequential [...] Conversely, in a tight game, the same batted ball is significantly less likely (~5%) to be called an Infield Fly."

"[T]hese results are consistent with the concept of Defensive Decision Making: [...] umpires may be choosing the more defensible action of not signaling for the Infield Fly instead of the action they may think as more appropriate."

In a previous post, I discussed how Major League Baseball (MLB) umpires tend to make errors more often when they have called two strikes or two balls in a row. I likened this to the Gambler's Fallacy, wherein the umpires mentally assigns a falsely low probability of witnessing (e.g.) three consecutive strikes, and therefore think the third pitch 'must' be a ball.

I offered another explanation - Defensive Decision Making -  where the umpire may be more likely to make the call that is more defensible in place of the call they see as best. When the hitter is advantaged in the count at three balls and zero strikes, we saw the umpire is more likely to call a close pitch a strike instead of correctly calling it a ball. The umpire may be appealing to a more defensible perception of fairness rather than the true call.

Continuing with this theme, others have suggested is that Defensive Decision Making is more evident when the stakes of the game increase. Previous studies have pointed to the perceived change in the way fouls and penalties are called in the playoffs versus the regular season for the National Basketball Association and National Hockey League, respectively. It has already been shown that sports viewers have an omission bias - fans tend to think a bad call by an official is worse than a non-call - and the defensive decision in this scenario would be the absence of excessive fouls/penalties. Superficial analyses of these situations reveal there is no statistical change in the number of fouls/penalties per game. These results are not surprising, given that these studies fail to address the change in strategy and style of play of the teams in the playoffs.

I attempt to uncover evidence consistent with the notion that Defensive Decision Making would occur more often in situations of heightened importance. In order to avoid the pitfalls of previous studies, I look at a situation where the officials must make a judgement call that is independent of team strategy - this is to say that the events requiring a subjective decision are as best as random. Therefore, I have chosen MLB's Infield Fly Rule, which is summarized below :

If all of the following conditions are met and the umpire signals an Infield Fly, the batter is automatically called out, regardless of the outcome. The conditions are as follows: First and second must be occupied with less than two out (third may or may not be occupied). The batter must hit a fair fly ball, which is not a line drive nor bunt, that; In the umpire's judgement, can be caught by an infielder with ordinary effort. H/T to Close Call Sports which have some great examples of the rule if you are still confused.

Note that in (3) the focus is the umpire's judgement.

I have collected all the instances from the 2016 and 2017 MLB regular seasons (up to 3 May 2017) which meet the criteria of (1) and (2) as above from Baseball Savant's Statcast Search. This data provides the angle and distance each batted ball traveled under conditions (1) and (2) and whether the play was called an Infield Fly or otherwise. Below is a graphical depiction of the 7,001 such occurrences. Each point represents the batted ball of one at bat with red indicating an umpire signaled an Infield Fly.

I then assign each observation a value to indicate the importance of the at bat's outcome to the overall outcome of the game. This is known as the Leverage Index - it accounts for the inning, the number of outs, the runners on each base, and the difference in runs of each team. Higher values indicate the outcome of the at bat will have a larger effect on the outcome of the game.

Note that in criteria (2) above, the batted ball must be a "fair fly ball." Fly balls are determined by the height the ball travels during its flight. Also note that in criteria (3) the ball must be able to be "caught by an infielder with ordinary effort" which indicates it cannot travel too far from home plate (hence the name Infield Fly). If the umpires were acting truly independent of any biases, one would expect that only a ball's height and distance would have explanatory power of whether a batted ball is called an Infield Fly. Indeed it is true that these two characteristics explain most of the variation in the calls using a simple logit model:

prob(Infield Fly) = f(height, distance)

This model has a pseudo-R² of 0.84 and correctly classifies the Infield Fly calls 99.2% of the time (i.e. the model and the umpires agree 99.2% of the time).

Next I add in the Leverage Index to the model. After experimenting with several specifications, I find that there is a significant negative correlation between the probability of the umpire calling for an Infield Fly and the importance of the at bat.

Below is a graphical depiction of how the probability of having the Infield Fly called declines as the leverage of the at bat increases when holding the height and distance of the batted ball constant. In this example, the probability of having an umpire invoke the Infield Fly rule is over 50% when the outcome of the at bat is relatively inconsequential to the overall outcome of the game. Conversely, in a tight game, the same batted ball is significantly less likely (~5%) to be called an Infield Fly.

I find that these results are consistent with the concept of Defensive Decision Making: the defensive decision in this scenario is to force the fielder to make the catch to get the out. Under this hypothesis, umpires may be choosing the more defensible action of not signaling for the Infield Fly instead of the action they may think as more appropriate.

I have shown there exists some evidence that Defensive Decision Making in one aspect of baseball, however there is no indication that Defensive Decision Making is limited to just the Infield Fly Rule. In fact, Defensive Decision Making may be prevalence in all sorts aspects of each and every game yet we lack the ability to test it empirically. Until then, this proves once again that there are many human biases that present themselves in everyday life, and umpires are no different the rest of us.

Thanks for reading, please offer you questions and comments below!