A year ago, I asked a couple questions. What makes a quarterback good? And better yet, is there a way to predict which quarterback prospects will both get drafted and find success once they make the leap from college to the NFL?
The short answer is that there is a way to make such predictions. But how? There’s widespread disagreement every single year as to which top quarterback prospects are good, bad, busts, or the best. How do we predict the future for top signal callers, and how do we measure whether they’re truly “good” or not?
If you want the long version and explanation of the initial predictive model I built, you can find my deep dive into initial peak pass efficiency and rushing yard market share here. I was also able to identify six of the top seven quarterbacks drafted this year as top candidates to be drafted a full year in advance. However, I wanted to take things a step further and improve my model (which I’ll break down simply here shortly) to help answer that question.
What makes a quarterback good?
In short, in today’s NFL, teams value two things in quarterback play.
- Highly efficient, accurate passing with limited turnovers.
- Sufficient mobility that forces defenses to account for the quarterback as a potential rushing threat on any given play.
The first one is obvious. The second one gets more pushback. No, quarterbacks don’t need to be Lamar Jackson or Kyler Murray, but they do need to be a viable rushing threat, or at least not a complete liability. Don’t believe me? How many first-round quarterback selections since 2017 have logged negative rushing yards in their peak pass efficiency season in college (in college, sack yards count against rushing total)? One. Just one. And it was Josh Rosen. The only other quarterbacks selected inside the Top 100 picks with peak pass efficiency season negative rushing yards in that span? Will Grier and Davis Webb. Rushing ability matters for modern NFL quarterback prospects.
However, rushing ability is a bit easier to see when watching the game or just checking the box score. Pass efficiency that truly translates to “this quarterback is ready to play in the NFL” isn’t so simple.
In my initial pass efficiency model, I only adjusted for one schematic variable: play action percentage. But if we adjust for additional variables, the pass efficiency model even be more accurate at identifying “good” quarterbacks.
Which variables artificially manipulate quarterback efficiency?
I ended up adding in both average depth of target (aDOT) and jet motion percentage (essentially any horizontal motion of receiving options across the backfield pre-snap). Average depth of target had a greater impact on efficiency than jet motion did, but both variables combined with play action percentage yielded even more meaningful statistical relevance in terms of explaining the variance in pass efficiency measured in two ways: Adjusted Yards Per Pass Attempt (AY/A) Over Expectation and Completion Percentage Over Expectation.
In short, I adjusted for play action percentage, aDOT, and jet motion percentage and built a “combined AY/A and Completion Rate Over Expected pass efficiency model” to figure out which passers were truly good. The resulting metric: Scheme-Adjusted Pass Efficiency or “SAPE.” As I continue to improve this model in the coming years, the basis for the efficiency model remains the same and we can keep the name, even if I add a variable or two.
