RotoViz vs. the Machine: Pitting Staff against App Projections
RotoViz has a great new Screener App, that helps you do some sophisticated analysis and regression work in just a few clicks. Here’s an exercise I did, using the Screener to create a running back projection model, which I then compared to our composite staff projections. There are some interesting takeaways.
First, here’s how I set up the model.
I’m looking only at RBs, and using data from 2010-2015. I’m also using per-game rather than per-season stats. I did that in part because fantasy is a weekly game, and weekly scoring is an important factor. But also that way players who were good on a per game basis, but missed games due to injury, get better consideration.
After some experimentation,1 I settled on these variables.
My three inputs are Year N receiving yards, rushing yards, and weight; my response variable, or the thing I’m trying to project, is Year N+1 PPR points. I spent some time experimenting with different combinations of explanatory variables. These three yielded the best r-squared and p-values, without exposing the model to too much multicollinearity or overfitting.
As a brief aside, both draft position and rushing TDs were predictive if I extend the timeline back to 2000. But both lose their prediction power in more recent seasons. That seems to speak to the changing nature of the game, and suggests that NFL teams are less inclined to overvalue early-drafted RBs. It also suggests that TDs are less important for RB value, which makes sense, as I’m predicting PPR scoring. The final interesting tidbit is that the weight estimate is negative, which means that heavier is worse than lighter, when controlling for the other variables in the model.
The table below shows our staff composite rank for each RB, then the rank as predicted by the Screener App’s model. I also included the player’s per-game scoring from last season and predicted scoring for 2016. There is some regression toward the mean,2 but overall it’s useful for understanding a ballpark pecking order. Play around with the table yourself; I included a few comments of my own below.
|STAFF||SCREENER||DIFF||PLAYER||PPR PREV||PPR PRED||PT DIFF|
The model I built with the Screener App doesn’t know that players have switched teams or will have new roles in 2016, so a lot of the ranking differences can be taken with a grain of salt. But I think there’s still some utility in looking at some of the differences. For example:
- Lamar Miller doesn’t fare very well in this exercise. Remember, the app doesn’t know he changed teams, but the app does predict that – all else being equal – he’d score around four points per game less this year than last because of the aforementioned regression to the mean. That doesn’t sway me from my bullish projection for Miller…but maybe it should give me pause. Miller has been really efficient with this touches, so some decline there could be expected. But I’m putting my faith in the Texans giving him
a shitload of cashenough opportunity to make him one of the top RBs.
- We’re also much higher on Eddie Lacy than the screener is. But unlike Miller, Lacy isn’t playing for a new team. All that’s changed is that he may – or may not – be in better shape. Brian Malone breaks down the risk inherent in his recent ADP.
- David Johnson looks terrible in the Screener model, mainly because he played all 16 games, but most of them as a backup. He comes out with just the 38th-best projection, vs. the staff composite of No. 5. That type of disparity could drive you to question your assumptions, without knowledge of his situation. Mike Braude makes the pro-Johnson argument, while Josh Hermsmeyer gives the reasons to fade Johnson.
- Thomas Rawls is another polarizing player. The Screener seems to be a fan of his per-game work last year, and projects him much higher than we do. Kevin Zatloukal has four reasons to like Rawls; Justin Winn would rather have C.J. Prosise.
- There are a handful of backs about whom the Screener is waving a big caution flag: Jeremy Hill, Duke Johnson, Shane Vereen, C.J. Anderson, Melvin Gordon, and Ameer Abdullah. All come in much lower (15 to 30 spots) in the screener’s projection. And it’s hard to say that 2016 looks more favorable than 2015 for any of them. They’re all on the same teams, and Vereen, Anderson, and Abdullah have new competition for touches. I’m not saying they’ll be busts. Gordon has literally nowhere to go but up, and it makes sense to think that Johnson could build on a solid rookie year. But the Screener definitely has me re-thinking these guys.
- It’s not all doom and gloom. Doug Martin, LeSean McCoy, and Jonathan Stewart all look undervalued in the Screener’s model. Another player the Screener likes a lot is Frank Gore.
- We’re projecting Justin Forsett as RB39, which exactly matches his current best ball ADP. But the screener sees him as RB15. Yes, I know the backfield situation in Baltimore is murky. But isn’t “Forsett is the lead back” one of the possible outcomes? That means he’s priced at his floor, with a lot of upside potential.
Fantasy Douche wrote a great article about the importance of aggregating ideas. I put a lot of stock in our staff projections: a lot of great guys put a lot of deep thought into them. But it’s also important to stay open to other ideas. This exercise will have me re-thinking a few players. Try out the Screener App (or, for a more casual approach, the Sim Apps) to build your own projection model.