FPL: skill, not luck

I have been trying to figure out whether skill or luck plays a bigger role in FPL for a while now. Looking back on some of the freak results in the 2022-23 season — and feeling more than a tinge of regret at missed opportunities to score points — I was often inclined to throw in the towel and admit that luck plays a bigger role than skill.

How can you account for Son coming off the bench to score 19 points for your opponent? Why can KdB score 19 for a league manager you are desperately chasing, only to score 5 in the next 7 games — after you transfer him in?

I quelled my sense of rage and decided to take a journalistic/scientific approach to the problem: I crunched the numbers. What I discovered is that luck can ensure you have a good streak — even a good season. It doesn’t always even out in the course of 38 games — a considerable sample size. However, only experienced players can finish with a high overall ranking several seasons in a row.

I’m going to make my case in detail by focusing on historically successful players. I’ll then compare these successful managers (and ones like them) against others, picked at random. Let’s take a look at some of the grandmasters in question.

Paul Marshman finished top-1k during that period
Fabio Borges enjoys the best average OR since first signing up (by a distance)
Richard Clarke finished inside top 50k during all 17 seasons FPL have been gathering data
Mark Sutherns — aka The Scout — finished inside top-5k more than anyone else: 10 times

A short preface

I’ve been playing FPL for 7 seasons now. During my second campaign I combined it with another hobby of mine — Microsoft Excel. As a result, I’ve been relying on FPL big data for a while now. To do so I scraped the numbers from the official website. At one point there were so many stats to crunch that I had to buy another PC, just to process them all!

The method I applied

I’ve read several books going into the 2022-23 season. One of the books, written by Paul Rogers, led me to a bona fide research on the topic of skill in FPL. The study is called “Identification of skill in an online game: The case of Fantasy Premier League”.

It was enlightening in a way, but I noticed one major flaw: the authors analyzed the top 1M after the 2018-19 season. As you can guess, a lot of casuals got inside that 1M based on luck. So I decided to instead track those who’ve done it season in, season out — I’ll be calling them elite players from now on.

I then picked random managers, a large enough number for a representative sample size. And then I pinned the former (elite) against the latter (random managers) — here and, by default, in every other article. For the 2022-23 season I analyzed 6M managers: those who signed up before the first gameweek.

How I picked managers to compare the elite against

It’s not feasible to analyze 10m players every gameweek. So I used a stratified sample size method: I took the first 5M accounts, divided them into 5 even groups (1M apiece) and took a progressively smaller pool from each following strata. I picked random players inside each strata, so, naturally, some elite players made the cut too.

Why only 5M? Generally, the bigger an account number, the likelier it is we are dealing with a fake account — or a drifter, who’ll abandon his team at some point during the season.

Before we analyze managers’ behavior in-depth — we’ll do so in the next several pieces — consider the bigger picture in the 2022-23 season. Both pools (elite and random players) were picked ahead of GW1 and both contain 10k managers. Look how much better the elite players fared.

Criteria I used to form “The Elite”

I had to answer a bunch of questions to do that:

Should I take into account all seasons they’ve played in or just the recent ones?
Should I pick out the elite based on relative or absolute ratings?
Finally, how many players should I pick: 100, 1000, 10000?

This is where I have to say — and you’ve probably figured it out yourself by now — that a bunch of elite players are known quantities on many popular fantasy sites, such as livefpl.net, fplstatistics and fplreview. What matters here is the principle these sites apply when picking out elite players: their historical high rankings.

To avoid skewing the numbers in favor of players who have simply participated in more seasons, I’ve decided to focus only on the last 5 campaigns. I also disregarded players with just a couple of seasons under their belts — even if their final OR was high. During these 5 seasons the number of managers jumped from 5.9M to 9.1M — by a hefty 50%, in other words.

I then weighed up the relative vs absolute rating dilemma. A relative rating is, on one hand, a more honest reflection of skill: it’s much harder to make a certain percentile with 9M players as opposed to 6M. Absolute ratings, on the other hand, are splattered on every site: everyone knows what top-10k is, but would everyone instantly understand what top-1% is? I think relative ratings reflect managers’ skill better than making top-1k/5k/10k.

Finally, I decided to include 10000 players in my “elite” sample size.

Elite’s average is 2508 points, random players stopped at 2388. That’s 120 points right there, at a pace of 3+ points per game.

You had to pick up 2523 points to finish inside 100k in 2022-23. 4 out of 10 elite players did that. 9 out of 10 random players failed.

It’s clear the elite players are doing something which propels them to pick up more points. So what’s the X factor? We’ll delve deeper into that in the following pieces. We’ll go over their transfer habits, chip strategies, choosing the captain, balancing the bench and team dynamics over the course of the season. We’ll start with how the players choose their GW1 teams.

Alex Baguzin