Your Last Shot (Might) Matter: The Hot Hands Effect in Professional Tennis

The hot-hands effect is more-or-less synonymous with the concept of correlation. Is the outcome of one first serve correlated with the next?

Most people actually have a very good intuition for correlation, but may not know it yet. Some simple examples:

• You live in Los Angeles and it is sunny at 12 pm. The weather at 1pm will, probably, also be sun. We would say that the weather between 1 and 12 pm is highly correlated (in LA).

• You live in Denver, and it is sunny at 12 pm. But, the weather at 1pm could be sleet and snow. In Denver, the weather can be uncorrelated between one hour and the next.

• You see a someone walking down the street, they take one step forward. You can be pretty sure that their next step will be forward, because a pedestrian's steps are correlated.

• You see a drunkard stumbling down the street. They take one step forward, but you cannot be sure that their next step will be in the same direction as the first, because a drunk's steps are highly uncorrelated.

And one final example: you are defending Stephen Currie. He takes a step forward, and so you assume his next step will be in the same direction. You are wrong and he dribbles around you (and maybe your ankles get broken) because Stephen Currie's steps are highly uncorrelated.

We are interested in correlation between serves. Is the outcome of one first serve, correlated with the next? Formally we can encapsulate this with conditional probabilities. So imagine the following: you are watching a tennis match with your buddy, and one player is about to serve. You two like to gamble, so you make a bet on the outcomes of serves. You want to be careful about whether you bet that this player will fault (F) or whether their serve will be good (G). Because you are a careful better, you consider these probabilities:

• P(F) is the probability of a player faulting on this first serve. While P(G) is the probability this first serve goes in.

Again, for pros, P(F) is about 0.4 (40%), and P(G) is 0.6 (60%). It has to be the case that P(G) + P(F) = 1, because probabilities have to add to 1. If your buddy is putting up $1 even-money for your $1, then of course you bet on the serve going in because P(G) is larger than P(F). Unsurprisingly, the serve goes in and you win the bet. Your buddy is angry, and so they make another bet, this time more nuanced because they want to try and outsmart you. They ask you the following.

You've been beating me all day on these wagers, so I want to try something different. I know this player's average first serve percentage is 60% -- but their last serve went in, so I think they are running hot. I bet that there is a bigger than 60% chance that their next serve goes in too.

They are asking you to bet on a conditional probability. They are wagering on the probability of a Good serve given (conditional on) the fact that the previous/last serve was G also. We refer to this probability as P(G|LG) -- called the probability of a G serve given LG (LG for Last serve being G). The other probability in the mix also is P(G|LF): The probability of this serve being G, given that the previous serve was a Fault (LF). So in probability language, your buddy simply wants to bet on this being true:

P(G|LG) is larger than P(G)

In fact, this bet will be true only if the following is true as well.

P(G|LG) > P(G|LF)
Or in plain english: the chance is higher for a player's current serve to go in if their last serve was in also.

And that is the bet you settle on. And P(G|LG) > P(G|LF) is the hot hands effect!

If P(G|LG) > P(G|LF) , then hot hands exists. This model is sometimes referred to as a "causal model of hot handedness." A positive event can cause another, and vice-versa. The vice-versa is a very important nuance. Because if P(G|LG) > P(G|LF), then it is a truism that:

if you missed your last shot, then you are more likely to miss this one as well.

So you cannot have hot streaks without also having cold streaks. If your average (so P(G)) is extremely high, then you are going to have more good-streaks then cold ones simply because on average you make more serves in than out. So the corollary is really only entertaining in a perfectly mediocre player with blazing hot hands:

In this pathological example, let's say a player's chance of making a first serve is P(G) = 0.5, and that P(G|LG) is a whopping 1. This means that the instant the player makes their first serve, they cannot miss. The next serve will go in, and so the next after that as well, and so on... They have white-hot hot hands.
But, you can show that this also means that P(F|LF) is 100% as well! So if the player starts out the game with a miss, then they will continue to miss forever! This would be a player who has both the world's best player within them, as well as the world's absolute worst.

If we find that professional players have P(G|LG) > P(G|LF), then their chance of a fault also increases if they faulted on the last serve. If they can be hot, then they can be cold. Professionals train to be as consistent as possible, and so by necessity, training out the cold-streaks will also train out the hot-streaks. The duality of "hot-hands always implies cold-hands" is really this: if professionals train to give their game 100% of themselves all the time, yet they can frequently "flip" on and give 110% of themselves, then that implies that the 100% they were supposedly giving before is actually more like 91% (= 100/110). This intuition was behind the initial hypothesis that the hot-hands effect should be small in professionals. Because pros train to give 100% and, as Limmy put it:

"A person cannot give it any more than 100%"
- Limmy, Limmy's show.

Please see the acknowledgements section for the source of these data and the license information. Many thousands of hours of crowd-sourced effort (and not from me!) went into compiling these serve data.

The data come from crowdsourced shot-by-shot professional tennis data by The Tennis Abstract Match Charting Project . I will skip over the parsing and reading in of these massive files -- but the code to do so is included with the repository for this article.

These data contain a treasure trove of information, but important to us is whether or not a player's first serve went in. We have to however select a sample of players and matches that are sufficiently uniform. We make these important cuts:

1. We are only going to use match data between 2015 and 2020, to keep the number of years to a minimum yet still have a large enough sample size (about 100,000 serves) so that our uncertainties (how well we can estimate the hot hands effect) are small. We make this 5-year cut because players change over time. Players will be roughly consistent over a 2-3 year period (four- five years is pushing it, but still OK). If we go longer than that, we will have to split players into their different eras, because e.g., Federer from 2005 is not identical to Federer in 2020.

2. Different players have different first serve percentages. We do not want to bin together players as if they were one person. So we only keep players for whom we have more than 1000 serves on record, and we keep all players separate.

After these two cuts, we have a database of 110685 first serves, grouped by game. 110,000 serves total guarantees that we will be able to tease out a hot hands effect as small as 0.4%, averaging over all players. The serve sequences, grouped by game, will be lists like: [first serve was in, first serve was out, first serve was in,...] etc. for each player and for each 5-8 serve game they played.

These data come from the following players:

['Grigor Dimitrov', 'Jo Wilfried Tsonga', 'David Goffin', 'Marin Cilic', 'Stan Wawrinka', 'Stefanos Tsitsipas', 'Rafael Nadal', 'Richard Gasquet', 'Milos Raonic', 'Roberto Bautista Agut', 'Karen Khachanov', 'Roger Federer', 'Andrey Rublev', 'John Isner', 'Matteo Berrettini', 'Alexander Zverev', 'Feliciano Lopez', 'Daniil Medvedev', 'David Ferrer', 'Gael Monfils', 'Novak Djokovic', 'Pablo Carreno Busta', 'Tomas Berdych', 'Dominic Thiem', 'Diego Sebastian Schwartzman', 'Kevin Anderson', 'Benoit Paire', 'Gilles Simon', 'Fabio Fognini', 'Kei Nishikori', 'Denis Shapovalov', 'Andy Murray']

Some players are more highly sampled than others (you can guess who if you are familiar with tennis), and so those players will have the best estimate of their hot-hands effect. At the top of the ticket, we have roughly 11000 serves from Federer and 9000 from Djokovic. 10,000 serves for any one player means that we will be able to resolve a hot hands effect of 1% or larger, for these players individually.

This raises a very interesting bias: sampling bias. Why do Federer and Djokovic have the most serves on record? After all, all pros play a similar number of matches per year. Well:

• These are top five players, and these data come from televised tournaments. The top five players are typically going the last the longest in these tournaments, and so will play the most televised matches.

• This is crowd-sourced data. Individuals who love tennis had to watch each match and record what they saw. The top players have the biggest fan base, and so more people pay attention to their matches.

We might be affected by sampling bias if we grouped players together. But because we do not, we are safe from it.

We now have an enormous set of first serve data. Our goal is to see if P(G|LG) > P(G|LF), and in particular, by how much.

If P(G|LG) = P(G|LF) + h , then we say there is a hot-hands effect of magnitude h.

The goal is to find the value of h. If h = +10%, for instance, then a player is 10% more likely to make their first serve if their last serve went in also.

An interesting note is that h could, in principle, be negative. Then it would lead to an odd conclusion that a player is less likely to make their first serve if their last serve went in. This is confusing, but not so if you frame it the other way: a player is less likely to miss, if they missed their last serve also. So a negative hot hands effect could be interpreted as a "clutch-up" effect, whereby poor performance causes the player to "get their head in the game" and improve. More on this in the results section. (Spoiler: this is probably not happening for any of the players in this data set).

To calculate h, we need to analyze sequences of first serves and calculate empirically P(G|LG), P(G|LF). The best way to do this is sequence counting. One game of our crowdsourced data, after some formatting, looks like this:

Player: Roger Federer | Game of the match: 11
First serves: G F G G G

This means that in the 11th game of this particular match, Roger made his first first-serve (denoted by G, for in), missed the first-serve in the second point (F), and made the third, and so on. (In the code, the data are all converted to binary: G is denoted by 1 and F by 0.)

The probability of a fault, P(F), and the probability of making a serve P(G) is really simple: count all the occurrences of F and all the occurrences of G. We will call the number of faults N[F] and number of made-serves N[G]. So:

P(F) = N[F]/(N[F] + N[G]) and

P(G) = N[G]/(N[F] + N[G])

For Roger Federer, we get that P(G) = 0.67 ; so Roger has a 67% chance of making a first serve. This is close to his PGA stats for first serves during this same time period, so we have passed one easy check. John Isner, by contrast, has a first serve percentage of 73% across this 5 year period.

We are interested in the more interesting conditional probabilities of P(G|LG) and P(G|LF). To count these, we want to count sequences of two good serves. So how many times does G, G occur? And for P(G|LF), how many times does the sequence F G occur? In the previous example, (G F G G G) , we have 2 occurrences of G G and 1 of F G. We have roughly 10,000 serves for Roger across 2000 service games. So we simply need to count sequences, using each game independently, and add those numbers together. Formally, we want to count sequences of GG (call the number N[GG]) and divide it by the total count of any sequence with a Good last serve.

P(G|LG) = N[GG] / (N[GF] + N[GG])

P(G|LF) = N[FG] / (N[FF] + N[FG])

In plain english, P(G|LG) is equal to the number count of good serves where the last serve was also good, divided by all possible sequences where the last serve was good (so the present serve could be good or bad, hence the sum in the denominator).

This has been dense, so lets recap: we need to count pairs of consecutive good serves, and pairs of bad serves followed by good. This allows us to calculate P(G|LG) and P(G|LF). If P(G|LG) = P(G|LF) then there is no hot hands effect -- the current serve is independent of the last serve. But if say P(G|LG) = 0.68 (for Roger) and P(G|LF) = 0.66), then the hot hands effect is 2%. And simply put:

The hot hands magnitude = P(G|LG) - P(G|LF)

You can skip this section if you would like, but then you would be trusting that I did not make any mistakes (and I strongly recommend you do not assume that I made no mistakes). In this section, I verify the code and methods on "mock" (a.k.a fake) datasets with a known hot-hands effect.

Whenever you set out and create an analysis procedure or pipeline, it is necessary to test it thoroughly. Consider a machine learning video processing software that identifies cats. Suppose your end-goal is to have security cameras that tell the owners when a cat is at the water bowl in front of their house. There are two ways to test this invention:

1. Sell and sign contracts for your cat identifying video cameras, distribute them to homes, and collect money. Then when things don't work, you hastily try and push out software updates until the cameras reliably identify cats.

2. OR: before you even put the software on an actual video camera, create other software that creates CGI cat videos. Thousands of them. Hundreds of thousands of fake cat videos. And throw some dogs in there. This is what is called a "mock" dataset. In machine learning parlance, it is also commonly called a validation data set. You want to verify that your software can reasonably discern cats from non-cats, before you ship out your software to production.

Number 1. is an extreme example of what is called "testing in production." As you can guess, it is very bad practice. For instance, what if you were designing self driving car software that has to identify pedestrians? Testing in production could, literally, be fatal. 2. is a much better solution. And 2. is exactly what I have done with this code here . I created two mock datasets of fake tennis players: one with zero hot hands effect and one with a 10% effect.

Long story short, repeating the analysis outlined in Inferring conditional probabilities from real serve data, is successful. The 0% dataset yields a hot hands effect of 0.05% +- 0.14% (more on this +- below), and the 10\% case yields an estimate near 10%. See Figure 1 below. This Figure shows the inferred hot hands % for 30 fictional "players". The dot for each player is the hot hands % h, while the blue bars show the interval of confidence. The interval of confidence is for "1 sigma", and can be understood heuristically as follows. The dot (the observed data point) can fluctuate within that interval or bar just from random noise).

In the next section, we do this analysis on real data (which really just amounts to pressing Go on the code with the real data). But look at one crucial conclusion from the left panel of Figure 1. Every fictional player (with "names" 0 through 30) have a 0% underlying hot/cold hands effect. Emphatically 0. Yet, due to the finite number of samples, the inferred value (the dot in the plot) can be slightly above or below 0%. This is measurement error. It is unavoidable and can only be shrunk by having more serves in the dataset. The important corollary here is that we cannot look at player 29, for instance, and say oh, they have a hot hands effect of 1.5%. The error bar helps us avoid this, because they have 1.5% but only with what is called 1 sigma significance, which is to say its not very significant at all. You can see that the confidence is low because the bar on player 29's data point is very close to intersecting with 0. The odds in favor of player 29 having a real hot hands effect is only 3 to 1. And we know for a fact, that h is in fact 0%, because we made this dataset from scratch. This illustrates the danger of cherry-picking a dataset for outliers.

You are bound to find outliers in any dataset, but it's possible (and the case here) that there is no profound physical reason behind their outliery-ness. The reason may simply be unavoidable random scatter. It also tells us the importance of having accurately estimated uncertainties (the bars in the Figure), because you can assess how significantly a data point is offset from some value. None of the data points in Figure 1 (left panel) are said to be offset from 0% with high significance. Both figure panels are just random noise, perfectly expected, about the input values of 0% and 10%.

The same analysis on the mock data is applied to the 110,000 serves compiled by the Tennis Abstract Match Charting project. See the acknowledgements section for the data; the code is here. The result is summarized by figure 2 below. Note the intentional visual similarity between it and the mock data, Figure 1.

Across 110,000 professional tennis first serves, there is mild evidence in favor of a hot hands effect. The average hot hands effect is 0.91% with an uncertainty of 0.56%. Meaning that if professional tennis player makes a first serve, then they are (on average) about 1% more likely to make their next first serve.

Remember the bet that you and your buddy made:

the chance is higher for a player's current serve to go in if their last serve was in also.

So now you have data to back up your decision; and lucky for you, it is data that they do not have. So if they bet you that that statement is true, and give you better than 10:1 odds that it's true, then well that bet would be a favorable one to take (this is not investment advice or an encouragement to gamble).

Thanks for sticking with me, and hopefully you learned some things along the way.

We found in Figure 3a that John Isner's hot hands effect appears to be 5%. And that there was ostensibly 81 to 1 odds in favor of this (we also call this "2.5 sigma" evidence, in astrophysics). But we should take those 81-to-1 odds with a grain of salt, because of the following:

We arrived a set of hot hands magnitudes h, for a large number of players first; and then after, we picked a player with large h.

So we cherry picked an outlier. John Isner is roughly a 2.5 sigma outlier, and for a data set of 30 players, we expect 1 about half the time. So John could, in reality, have an underlying hot hands effect of 0% -- but because of measurement error (i.e., bad luck with how serves happened to land), he was just the 1 of 30. And in a set of 30 players, there will almost always be a 2-sigma outlier and sometimes a 2.5-sigma outlier. So there may not be any physical meaning to his 5%. This is also what is called a selection effect. The 81 to 1 odds were quoted without taking the selection effect into account. If we did take that subtle effect into account, those odds would probably be significantly lower.

I thank Mileta Cekovic for pointing me to the Tennis Abstract Match Charting data. This study would not be possible without those data.

This work is based on crowdsourced shot-by-shot professional tennis data by The Tennis Abstract Match Charting Project and is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The data in this article was obtained from https://github.com/JeffSackmann/tennis_MatchChartingProject , commit hash f82e786a931a1ed5b4fcb52a1abb172d90493efd