The Post-Draft 2017 QB Success Model

Earlier this offseason I created the new and improved version of my QB success model. The QB success model uses several machine learning techniques to find the important variables in predicting QB success and in the implementation of the success classification itself. I’ve updated the results with the actual draft positions of the QBs from the 2017 NFL draft.

In the pre-draft version of my model, I projected the following draft positions. I’ve also highlighted their actual draft positions, so you can easily see the change. Draft position was one of the top two factors in predicting QB success, along with the QB’s final college year adjusted yards per attempt (AYA).

QB DPos.Est DPos
Deshaun Watson 5.6 12
Mitch Trubisky 15.5 2
DeShone Kizer 20.4 52
Pat Mahomes 68.6 10
Davis Webb 167.9 87
Nathan Peterman 189 171

We can see only Deshaun Watson and DeShone Kizer fell from my pre-draft projections, which means we should expect the two of them to drop from their pre-draft projection. Everyone else we should expect to increase.

Well, that’s pretty much what happened, except I made a couple of tiny errors in my pre-draft model. I accidentally grabbed the yards per attempt instead of the adjusted yards per attempt of both Kizer and Pat Mahomes which will mitigate some of Kizer’s drop in success thanks to his draft position drop, and add a small boost to Mahomes already improved projection from his skyrocketing draft position.

Here are the updated results for the 2017 QBs.1

Updated 2017 QB Success Model Results

QB Post-Draft Pre-Draft Diff
Mitch Trubisky 74.4% 73.8% 0.6%
DeShone Kizer 39.2% 43.9% -4.7%
Pat Mahomes 35.7% 27.4% 8.3%
DeShaun Watson 19.8% 20.4% -0.6%
Davis Webb 10.8% 2.9% 7.9%
Nathan Peterman 8.3% 7.0% 1.3%

There weren’t any dramatic shifts in success rates, and there shouldn’t be. Each of the QBs himself didn’t change as a player pre-draft to post-draft. There’s five other variables that go into predicting QB success, so draft slot shouldn’t be a massive game changer. But certainly it strongly correlates with success, even if it only means being drafted earlier gives a QB more shots at being successful.2

So what changed the most? Mahomes certainly got a massive boost not only from his leap in draft position from where he was projected a couple months ago to now, but also that tiny tick up from yards per attempt to AYA. It’s important to note that Mahomes now has a similar success probability to Jared Goff who was selected first overall in 2016’s NFL Draft. Mahomes career trajectory has been encouraging, and now that he’s going to be a full-time football player with elite development resources, he probably has a high ceiling.

Davis Webb sports the next highest jump in absolute success probability, and the largest gain in relative success probability. Webb’s AYA numbers hurt him, and his lack of improvement over three collegiate seasons is certainly a cause for concern. However, going 80 places higher in the NFL Draft than my pre-draft forecast, there’s probably more urgency to get him on the field in the near future than if he went at the projected 167th overall pick. Aside from his ball velocity, the rest of his profile is concerning. I mentioned the low AYA (6.9), but he also has a small hand size (9.25 inches) and a poor film grade (70).

Kizer’s fall was the largest of any QB, and that’s reflected in his updated success probability. However, he only drops 4.7 percent thanks to the little error I made in his AYA stat. Even with the 52nd overall pick, he’s still projected to have a success probability nearly double that of Watson, despite Watson’s 12th overall status. That’s because Watson has a poor ball velocity (49)3 and a worse final year AYA than Kizer (8.0 vs. 8.4).

Finally, Mitchell Trubisky and Nathan Peterman see only moderate gains. Trubisky moved up 13.5 draft slots from projected. So while that is the second biggest factor and while it’s near the top of the draft, it turns out the top of the draft actually has a flatter effect on success probability than a similar change in draft slot later in the draft. Intuitively it might not make sense at first, but let’s think about this. Would Trubisky really gain that much more in opportunity and/or evaluation by going from 13th to second in the draft? Probably not. He’s probably going to get his fair shot either way. But QBs who go from the fourth to the third round might stand a bit better chance of gaining opportunity, moving from just off the fringe to maybe slightly on the green.

Subscribe for a constant stream of league-beating articles available only with a Premium Pass.

  1. Note: this only includes QBs who have data for all the statistically significant predictors.  (back)
  2. There’s also probably a subjective scouting component to it that should boost the likelihood of success as well. Both of these are rolled into draft position.  (back)
  3. And 45 as measured by Ourlads.  (back)
By RotoDoc | @RotoDoc | Archive

Comments   Add comment