[Ed. Note: This is the first post by my good friend and stat guru Chase Stuart. For at least the rest of this season, Chase will be contributing his unique perspective to the site. Chase has previously contributed to the New York Times Fifth Down and the Pro-Football Reference Blog. You can also follow Chase on twitter.]
The last two seasons, I have published college football ratings using the Simple Rating System. Before explaining how the Simple Rating System (SRS) works, allow me to first explain what the SRS is trying to do (and just as importantly, what it’s not trying to do).
Most rating systems fall into one of two categories. A rating system could simply replicate the standings in any particular league; such a rating system would best be described as retrodoctive or explanatory. A retrodictive rating system fits the data to explain what happened in the past. The BCS computer ratings are mostly retrodictive; so are player or team ratings that give significant weight to high-leverage plays that tend to be highly random (clutch play, fumble recovery rates, etc.). An explanatory rating system tries to measure how much a team or player has accomplished in the past; it does not attempt to answer the question “what will happen next?” When Bill Parcells said “You are what your record says you are,” he’s championing retrodictive ratings. So was Rich Kotite when, coaching the 7-2 Eagles in 1994, he said to the media: “Judge me by my record.” An explanatory rating system would say that Kotite and his Eagles were doing well; but it would never have predicted that Kotite would go 4-35 over the next — and final — 39 games of his career.
The other type of rating system is a predictive system, which works as they name implies: it tries to predict the future. Here is a useful chart detailing some of the differences between the two in college football rating systems. Predictive rating systems are not very concerned with wins and losses; instead, they focus on more granular pieces of data. The best and most obvious example of a predictive rating system would be the formulas used by the folks in Vegas. Those who make point spreads aren’t disturbed if their rankings place Team A, which has “accomplished less” than Team B, higher up in their rankings. This weekend produced a useful example. No purely retrodictive rating system would put the Oklahoma Sooners ahead of the Kansas State Wildcats. Oklahoma was 6-1 but lost to a mediocre Texas Tech team; Kansas State was undefeated and had beaten some solid teams, albeit in less than thrilling fashion. Both the BCS ratings and the Associated Press’ rankings had Kansas State over Oklahoma, because those systems are designed to acknowledge accomplishments. But despite being the higher ranked team and playing at home, Kansas State was a 14-point underdog to the Sooners. And Oklahoma promptly went into Manhattan and blew out the Wildcats, 58-17.