On the Relationship Between Scouting and Advanced Stats (Part I)
Below I’ve embedded a tweet from last night where I jokingly pointed out a problem that I think exists with the idea that tape grinding is the highest form of football evaluation. Just to be clear, I wasn’t really trying to bust Steve’s chops that much as he’s an all around good dude and I think that we would probably agree on a bunch of the big picture issues related to eval (and I’ll get there before this piece is done.. and before you even read the rest of this piece you should follow Steve on Twitter). But it was an interesting thing to think about anyway, so let’s break this down.
The primary problem that I think the tape grinding crew has when it comes to eval is that eventually we’re going to need a way to settle the basic question as to which football players are good, and which football players are bad, and how are we going to do that? We could use our individual opinions about players, that’s one way to skin the cat. I say Brandon Lloyd sucks, I’ve seen some tape grinders say he’s good. How do we decide who’s right (I mean other than simply dismissing my opinion because it came from a spreadsheet)? Let’s forget about that issue for a minute. Let’s say we disagree about which teams are good and bad… except that’s not likely to happen is it? Why? We have win/loss records to settle the dispute. And while win/loss records are subject to all sorts of issues with variance (turnovers, wins in close games, etc), they are generally a good indication of who is good and who isn’t right? But that creates a problem because now we’ve settled the issue not by watching tape, or through a subjective assessment. We have a pretty concrete way to end the argument over which teams are good and which teams are bad. And if we return for a minute to the issue of Brandon Lloyd, how long do you think it would take before the straw-man tape grinder that I’ve created would offer Lloyd’s 2010 statistically good season in defense of his opinion? It would happen pretty quickly right? And there’s the issue for the tape grinders. Eventually we’re going to be talking about some kind of objective measure, and it will probably be a simple counting stat like yards or touchdowns.
Before I piss anyone off any more than I already have, let me offer that in some sense grinding tape is just a reaction to the very real problem which is that the box score does not contain enough data. On that point, I think I speak for all numbers focused watchers of football when I say that we would all love to have more data rather than less. So all of the things that are showing up in tape sessions – like whether a QB throws from his back foot, or whether a WR rounds off his routes – we’d love to see numbers on all of that stuff. As I’ve said to Eric Stoner on twitter before, I think that the advanced stats crowd would find a lot of common ground with the tape grinding crowd if we could all just figure out how to record what’s being watched. In other words “All of that is great. Now let’s write it down.” Because in the end, all the advanced stat crowd wants is the ability to backtest the effectiveness of a process/model.
The need to backtest the effectiveness of your process is central to the idea that I’ll talk about tomorrow in Part II of this series, which is the notion that over time your evaluations should improve. Today I just wanted to establish that at least on a very important part of the player evaluation discussion, it’s likely that in the end we’ll be talking about numbers. So if numbers will be a key part of the end of the discussion, it seems odd to dismiss their usefulness at the start of the discussion.