Pulling the lever – when to sack a Head Coach

This article uses the terms Manager and Head Coach interchangeably until near the end. 

“He’s got to go”, “he’s lost the dressing room”, “clueless, absolutely clueless”, “you don’t know what you’re doing”

Every football fan has heard these opinions at the ground, at the pub after, or in the media. 

Football management is a brutal business. It can take years of learning the trade, competing to get onto the FA courses, completing the licences, waiting for the opportunity to lead a team. You go from finally signing that Head Coach contract usually straight into a team in bad shape and low on confidence. Lose those first few games and your job life expectancy is low. You may never get another chance at the same level or higher.

Depressing isn’t it?

Yet the competition for these roles is high.

Do managers/head coaches actually make that much difference?

It depends what you ask and how you measure impact. I think the overall conclusion for research is that there are outliers in value add from coaches but most competent coaches get similar outcomes from similar resources.

This isn’t a statistically powerful argument but in my experience within clubs it makes a huge difference. They are probably the most significant individual within a club, the ultimate decision maker, and if they don’t fit in with what you are building as a club it can be a disaster. 

If you measure solely by short term results it might not make much difference if you are managed by Manager A or Manager B. However if you are a club with a vision, a plan, and an aligned view of how to get there it very much does matter who is leading the first team and their attitude to the project. 

Let’s start with a bit of basic maths.

The biggest problem with assessing football coaches is how random results are.

We’ll look at a team whose underlying data in every game is a third chance of a win, a third chance of a draw, and a third chance of a loss. We’d expect this of a team that gets roughly equal xG for and against. A proper midtable, exactly average team.

Let’s generate a random 46 game seasons with a simulated 0 (loss), 1 (draw) or 3 (win) outcome from each game.

Round 1: 1

Round 2: 3

Round 3: 0

Round 4: 0

Round 5: 0

Round 6: 1

Round 7: 0

Round 8: 1

Round 9: 3

Round 10: 0

Round 11: 3

Round 12: 1

Round 13: 3

Round 14: 0

Round 15: 1

Round 16: 3

Round 17: 3

Round 18: 0

Round 19: 1

Round 20: 0

Round 21: 1

Round 22: 1

Round 23: 3

Round 24: 3

Round 25: 0

Round 26: 1

Round 27: 1

Round 28: 0

Round 29: 0

Round 30: 0

Round 31: 3

Round 32: 0

Round 33: 0

Round 34: 1

Round 35: 1

Round 36: 0

Round 37: 1

Round 38: 3

Round 39: 0

Round 40: 1

Round 41: 1

Round 42: 3

Round 43: 0

Round 44: 3

Round 45: 3

Round 46: 1

Total: 55 points

A midtable season as we’d suspect would happen most of the time.

If we divide the season into rolling 10 game periods we started slowly with only 9 points from the first 10 games, then hit form averaging 1.8 points a game for the next 10 games. The next 20 games were tough, reaching a nadir in 0.6 points a game over fixtures 25-34 with 5 defeats in 6 games. Thankfully the lads turned things around and finished strongly with a flurry of wins to put as all in a positive frame of mind for a promotion push next season. 

Remember these are all random numbers with an equal chance of happening. Nothing changed through the great run or the poor run, it was just the joy of randomness. 

xG (expected goals) allows us to delve a bit deeper than outcome. I now made the team really good and increased the likelihood of winning massively.

I simulated 3 more seasons and got 95,94, and 90 point outcomes.

The best 10 game sequence was 2.8 points per game in a stunning 12 from 13 win streak the (imaginary) fans still speak about. 

Season 2 started with 13 points from the first 10 games.  Just 1.3 PPG.

Had the manager lost it? Were the players in need of fresh ideas? No it was randomness, the underlying odds of winning/drawing/losing were identical in each simulation.

But we don’t live in a world where people take Monte Carlo simulations seriously. I think I had personally underestimated the external pressure that comes with these fluctuations in form. It is all very well sitting in a meeting saying “well actually our xG is fairly equal in most games and we should therefore expect randomness and streaks” but if the owner has just had a thousand people chanting at him to sack the useless idiot in charge he’ll tell you to stick the xG where the sun doesn’t shine and neither you or the manager are likely to survive at the club for long.

How many managers survive low points runs? 

Perhaps the poster boy for surviving poor runs should be Sir Alex Ferguson. 

In March – May 1989 and November to Feburary 1990 he has two runs of 10 game where he averages 0.5 PPG or lower.

I doubt Man Utd were running Monte Carlo simulations or using underlying xG trends in 1989, so why did they choose to keep him?

Alex Ferguson was already a proven manager when he joined Manchester United. A manager who had won a European trophy and league title with Aberdeen is clearly a talent. 

When you have someone of rare ability and a track record of success you naturally give them more time. 

Even so It remains remarkable that he wasn’t sacked and went on to become an all time great. 

People still talk about a Mark Robins goal in the FA Cup that saved his job.

Survivorship bias

It is about this point in an article that internet statisticians start pulling up their diagrams of WWII plans riddled with bullets and finding their bookmarked article on “survivorship bias”. If you only look at the ones who survived and thrived you ignore all the ones who were rightlfully sacked because they were actually no good.

True, we should never use one off examples as proof of a concept. 

Alex Ferguson is obviously a massive outlier, and 1989 was a long time ago, without Champions League qualification and massive prize money losses to worry about.

What we wanted to research was whether managers could survive significant periods of poor form and turn things around (e.g. win enough to keep their job longer).

We looked for EFL Championship Head Coaches who had 10 game runs where 12 or fewer points were won, in any period of their time at the club. From the first of these runs at their club we then measured how long they remained in position.

Thomas Frank endured a terrible run on joining Brentford, before winning PL promotion. Other long serving managers like Mowbray, Farke, and Neil remained in charge for years after very poor 10 match spells.

There are plenty of examples of head coaches who lasted only a few months after their first bad run (if they even lasted that long) but with the average Championship manager lasting 280 days in charge this isn’t surprising. 

In the Premier League we can look at Mikel Arteta, and his main coaching influence, as examples of Head Coaches who have had terrible runs of form but survived to build teams that maximise resources.

Mikel Arteta

David Moyes

This type of analysis has one major flaw, and it isn’t just cherry picking the few examples of coaches that have survived and thrived.

We are dealing with a hypothesis “teams should probably not change head coach so often” that can’t really be tested. We can look at what happened teams after the coach was sacked (this has been done before and on average teams stay pretty much on the same trajectory) and we can look at teams who didn’t (and on average they pretty much stay on the same trajectory). But we can’t look at individual decisions and say for sure what would have happened.

What we can see is there are lots of examples where bad runs haven’t resulted in early exits and relative success has followed. So we can’t use poor runs of form as evidence coaches are bad at their job. 

Would Moyes at Mancheseter United have turned into a Premier League winning manager, would Nathan Jones’ Southampton be on the brink of European football? You many scoff and say it is impossible, they were clearly “out of their depth” but both got those jobs because they had already clearly proved they were good managers.

One Lever 

The issue is that as a football club in a poor run, outside of player transfer windows you effectively only have one lever you can pull to change things. Sacking the Head Coach. 

And again as a consulting data science team you can say “it wil likely make little difference on average” and refer to the below meme but you aren’t the ones under real intense pressure to change things.

So what can you do?

We need a deeper understanding of what makes a manager good. It is likely a combination of their ability to coach and how they connect to a club and group of players. 

It is a word used almost to the point of cliché nowadays but alignment, alignment, alignment.

If a club doesn’t have a well defined culture, and owners, sporting staff, and recruited players who fit together than a Head Coach won’t be just a Head Coach. 

They will have a different, much bigger job of defining the club culture, bringing his own style of play, signing players who fit his style, AND coaching. A Manager.

Those are different jobs and some people will be brilliant at all aspects, and some only one.

Knowing what the job actually is, a Manager or a Head Coach is important. We’ve been using the words interchangeably but they aren’t the same thing. 

Knowing what you are

The single biggest differentiator between clubs who over and under achieve is a shared identity. 

We are _________ FC and we do _________ really well. We are working hard on  __________ and in 5 years time the outcome of this will be _______________________.

If you can fill in the blanks above, and you have some degree of agreement across the club in the answers you receive back, you are doing much better than most clubs.

And this is where the alignment really comes in. If you know what you are, and what you do well, and what you are working on then choosing a Head Coach who fits the club from a cultural and football philosophy point of view is much simpler. 

You don’t want someone to shake things up, or tear things down. You want someone who can accelerate your progress towards a defined outcome. 

If you don’t know then maybe you need a Manager.

So how do you decide whether to sack a Head Coach?

If you already think that you have your own Alex Ferguson, Mikel Arteta, or Thomas Frank then how do you resist pulling the lever and sacking them when on a poor run of form. 

A simple decision tree can help. Every club will have different situations so these need to adapt to the circumstances the coach/manager is working in.

Firstly we need to look at the underlying data. Some teams are just plain bad, injury ravaged, playing against vastly superior players every game. 

If the underlying data (not just xG but a whole basket of metrics that monitor different phases of play) any good?

How are they coping with the huge pressure. It is something we don’t experience in every day life and some people do not cope well with it.

Are they open to help? If set pieces are awful do they allow specialists to help? If recruitment has been poor do they embrace new ideas even when under pressure?

“Monitor and support” may seem like a bit wooly advice but changing a Head Coach / Manager is expensive and as we’ve seen things can turn around pretty quickly.

The support will look different depending on the underlying data, if it looks good the support may be more emotional, if poorly it may be emotional and practical (additional specialists to support areas of weakness).

Avoiding expensive mistakes is the most important benefit of analytical thinking in running football clubs. It takes real strength of leadership not to bow to pressure and pull the lever but sometimes not doing something is the best decision you can make.