Quantifying Wide Receiver Skill
I fixed the NFL's NGS Data so you didn't have to
Hey folks. It’s been a while, but I’ve been hard at work here making nothing of any value whatsoever. It’s like if, instead of having a boulder to push up a hill, Sisyphus was wrangling with stupid NFL statistics that didn’t quite work (and getting significantly less jacked in the process).
I’ve been trying, for a while now, to bridge the gap between inputs and outputs in the NFL. This is something that makes player evaluation tricky in football in ways that it really just isn’t in other sports. I’ve written about this idea before, but in hockey and baseball, and to a lesser extent basketball, your inputs map extremely cleanly onto your outputs.
Let me explain. If you want to know how good a player is at shooting a basketball, you have, like, 700 shots over the course of a season to figure it out. A good 3-point shooter will, believe it or not, make a high proportion of their 3-point shots. Obviously, their environment matters, but you could have dropped Steph Curry onto any team and I’m pretty sure he’d have figured it out. The same is (clearly) true of baseball royalty like Ohtani, who can succeed anywhere, and of hockey stars all over.
Football, as usual, has less tractable problems, especially for the non-QB positions. My favorite example is reigning OPOY Saquon Barkley, who spent years toiling away for a Giants team that’d struggle to win 9 games before being unlocked by the Philadelphia Eagles. Even the so-called advanced metrics, which you’d expect to be driven by the player rather than their environment, can’t seem to figure things out.
Notice Anything?
Based on the above data, there are two self-congruent world views you can have about Saquon Barkley. One is that he was a mediocre football player for the first six years of his NFL career before finally putting it together in Philly. The second is that he’s, more or less, the same player he’s always been, but being put in a vastly superior environment has unlocked his potential. I’ll let you decide which I think is true.
The issue in football is that the inputs are pretty far removed from the outputs, so evaluating players based on their outputs is silly. Let’s take a look at player tracking stats to really get the point across.
Wide Receiver Skill
When you think of what makes a wide receiver good (these are inputs), you probably think “they run routes well, catch the ball well, and run after the catch well”. The outputs are “they catch a lot of balls and generate a lot of yards”. Seems easy enough. What would be lovely would be to isolate these skills and just grade receivers on how good they are, not how good their production is.
I did my best.
Separation
The people at Next Gen Stats publish the average number of yards of separation between receivers and their nearest defender at the time of a catch. In theory, receivers who are better at running routes will generate more separation. And that’s probably true in a vacuum. But life doesn’t exist in a vacuum.
For one thing, not all separation is created equal. It’s easier to separate at shorter target distances than longer passes; this probably feels paradoxical, but hey, it’s true.
Furthermore, and this is the important bit, defenses are likely to attempt to limit separation for an opposing team’s best receivers, and this is not tracked basically anywhere but matters deeply. Look at this chart of separation vs WOPR (weighted opportunity rating, developed by Josh Hermsmeyer). Essentially, having a high WOPR (a combination of air yards share and target share) implies that a receiver is highly used in an offense, but those receivers are generating less separation than less utilized players.
This shouldn’t be surprising; just look at the attention teams pay to Justin Jefferson and Ja’Marr Chase every play. Obviously, you can correct for this, but the point stands that football, the ultimate team game, is incredibly context dependent, and how good receivers are at generating separation follows the same rules.
Important context for how well a receiver separates (not an exhaustive list by any means) is below:
How much attention is the defense devoting to him? (No public data available 🙁)
How far downfield are his routes typically thrown?
What is his route-diversity?
What is the QB’s time between the snap and the throw? (The longer a QB can keep a play alive, the more likely it is that a receiver will be open)
I believe that the most important piece is the first part, and that data simply doesn’t exist for free.
Catch Rate
Catch rate is relatively free from the context issues that plague grading receiver separation, so I won’t spend a lot of time dwelling here. A receiver is good at catching the football if he does so when open and when the ball is thrown well. The context that matters intuitively breaks down to the following:
Was the ball thrown well (i.e. was it thrown by an accurate QB)?
How close are the nearest defenders?
Correcting for these two things should give a relatively robust idea of how good a receiver is at catching the football. Not going to spill the digital ink too much here.
Yards after Catch
Yards after the catch (YAC) is the final piece of the puzzle. Given that a receiver made the reception, how far downfield did he run with the ball? Elite receivers should have the ability to make defensive backs miss in the open field and the acceleration and top-end speed to generate some serious distance when they do. There is a lot of shared “important context” between YAC ability and Catch Rate, and Separation. Basically, where are the receivers relative to the rest of the defenders on the field?
Putting it all together
So, here’s the deal. What you could do--and I did--is model the expectation for an average receiver for each of these skills given all available context, record which receivers are better than expectation at each skill, and then blend the performance above expectation numbers to get a sense for receiver talent. Boom. Easy-peasy.
There are two big issues here. The first is that we don’t have the context that actually really matters to correct for, so we can’t factor it into the expectation modeling. The second is that, with all of these metrics, comparing against expectation starts to punish players who have put themselves in good situations. Players who separate well should be expected to generate more yards after the catch. It follows that their YAC above expectation receives a punishment for being better at getting open in the first place. There’s some redundant information between the three skills.
That’s not to say there’s no value in an exercise like this (there is!), just that the current state of so-called NFL advanced stats is lacking even when you try to correct for everything that they don’t.
I’m being too hard on myself, I actually think this is pretty cool, just doesn’t quite get as far as I’d like it to. In particular, receivers who have harder assignments (even harder than their WOPR, which I did correct for, implies) are punished here. If you don’t like my tables, subscribe to get my attention before yelling at me.
My last caveat here is that Next Gen Stats only publishes separation values based on targets, so there is some obvious survivorship bias here. There are some receivers who are probably only getting targeted on gimmicky routes where they’re wide open. I have no way to correct for this, but boy do I wish I did.
The columns of note in the table below are the Separation, Catch, and YAC Z scores, which are the context adjusted Z scores for each of the skills.
Some Quick Nerdy Notes
The separation, catch, and yac z scores are all fairly normally distributed with very little correlation. This means we did our job reasonably well at isolating truly independent skills.
Having said that, the combined ranking only correlates with itself y/y with an r2 of ~0.2 so, obviously, this is lacking. Perhaps more descriptive than very predictive, which bites.
Haven’t I seen this before?
The fine people at ESPN analytics have already done a pretty good job measuring these inputs using (not publicly available) player tracking data, but I wanted to build my own model.