The Simple Rating System: Bringing order (kinda) to chaos

[Ed. Note: This is the first post by my good friend and stat guru Chase Stuart. For at least the rest of this season, Chase will be contributing his unique perspective to the site. Chase has previously contributed to the New York Times Fifth Down and the Pro-Football Reference Blog. You can also follow Chase on twitter.]

The last two seasons, I have published college football ratings using the Simple Rating System. Before explaining how the Simple Rating System (SRS) works, allow me to first explain what the SRS is trying to do (and just as importantly, what it’s not trying to do).

A want and enjoyment of numerosity

Most rating systems fall into one of two categories. A rating system could simply replicate the standings in any particular league; such a rating system would best be described as retrodoctive or explanatory. A retrodictive rating system fits the data to explain what happened in the past. The BCS computer ratings are mostly retrodictive; so are player or team ratings that give significant weight to high-leverage plays that tend to be highly random (clutch play, fumble recovery rates, etc.). An explanatory rating system tries to measure how much a team or player has accomplished in the past; it does not attempt to answer the question “what will happen next?” When Bill Parcells said “You are what your record says you are,” he’s championing retrodictive ratings. So was Rich Kotite when, coaching the 7-2 Eagles in 1994, he said to the media: “Judge me by my record.” An explanatory rating system would say that Kotite and his Eagles were doing well; but it would never have predicted that Kotite would go 4-35 over the next — and final — 39 games of his career.

The other type of rating system is a predictive system, which works as they name implies: it tries to predict the future. Here is a useful chart detailing some of the differences between the two in college football rating systems. Predictive rating systems are not very concerned with wins and losses; instead, they focus on more granular pieces of data. The best and most obvious example of a predictive rating system would be the formulas used by the folks in Vegas. Those who make point spreads aren’t disturbed if their rankings place Team A, which has “accomplished less” than Team B, higher up in their rankings. This weekend produced a useful example. No purely retrodictive rating system would put the Oklahoma Sooners ahead of the Kansas State Wildcats. Oklahoma was 6-1 but lost to a mediocre Texas Tech team; Kansas State was undefeated and had beaten some solid teams, albeit in less than thrilling fashion. Both the BCS ratings and the Associated Press’ rankings had Kansas State over Oklahoma, because those systems are designed to acknowledge accomplishments. But despite being the higher ranked team and playing at home, Kansas State was a 14-point underdog to the Sooners. And Oklahoma promptly went into Manhattan and blew out the Wildcats, 58-17.

The SRS is a predictive system, which means it could theoretically place a 3-5 team ahead of a 7-1 team. As a sanity check, it usually tends to correlate pretty well with the point spread in most games (and it’s worth trying to understand the deviation when the lines do not match up). But as the name implies, it’s simple. The SRS does not factor in the thousands of pieces of data one could place into a rating system, trading precision for elegance and ease of understanding. Here’s how it works:

The SRS takes only two factors into account: strength of schedule and adjusted margin of victory. Each game is given equal weight. Therefore, the sum of a team’s SOS and MOV rating is its SRS rating. A team could have an SRS rating of 60 by having an MOV of 30 and an SOS of 30, or an MOV rating of 40 and an SOS of 40. Once you have the SRS scores for each team, it’s very simple to understand how the system arrived at those ratings. Further, the numbers the system spits out are easy to understand: if Team A has a rating of 55 and Team B has a rating of 44, it means that Team A is predicted to be 11 points better than Team B. The units here mean exactly what you think they mean.

It’s complicated to create these ratings, but I’ve done that in Excel for you. The tricky part is that each team’s strength of schedule is dependent on the ratings of each of their opponents, which is dependent on the ratings of each of their opponents, which includes the original team we’re trying to rate. If you adjust each team’s rating over hundreds of thousands of iterations, eventually the ratings converge, and we’re left with “true” ratings.

One last note: I wrote about that the SRS uses adjusted margin of victory. What does that mean? For starters, the road team is given 3 points for each game (but there is no home or road team for neutral site games). After that adjustment, all wins and losses of between 7 and 24 points as exactly that. So a 24-10 road win goes down as +17 for the road team, -17 for the home team. Wins of 7 or fewer points are scored as 7-point wins and losses of 7 or fewer points are scored as 7 point losses (except that road losses of 3 or fewer/home wins of 3 or fewer are graded as 0 point ties). This gives a very minor boost to teams that win by a couple of points. Finally, wins of more than 24 points/losses of more than 24 points are scored as the average between the actual number and 24. This is to avoid giving undue credit to teams that run up the score.

Now, the college football SRS ratings after week 9:

Rk  Team                 Con   G    MOV      SOS      SRS      W-L
1.  LSU                  SEC   8    24.5     43.9     68.4     8-0
2.  Alabama              SEC   8    27.4     40.7     68.1     8-0
3.  Oklahoma St          B12   8    22.0     45.2     67.2     8-0
4.  Stanford             P12   8    27.4     39.2     66.5     8-0
5.  Oklahoma             B12   8    21.5     44.1     65.6     7-1
6.  Boise St             MWC   7    22.7     41.2     63.9     7-0
7.  Oregon               P12   8    20.7     42.0     62.7     7-1
8.  Wisconsin            B10   8    21.5     38.0     59.5     6-2
9.  Michigan             B10   8    17.4     40.6     57.9     7-1
10. Texas A&M            B12   8     9.9     46.3     56.2     5-3
11. Arizona St           P12   8    13.1     42.7     55.8     6-2
12. Notre Dame           IND   8     9.1     45.6     54.8     5-3
13. Nebraska             B10   8    12.3     42.4     54.8     7-1
14. Houston              CUS   8    22.9     30.5     53.4     8-0
15. Missouri             B12   8     5.8     47.6     53.3     4-4
16. South Carolina       SEC   8    11.4     41.5     52.9     7-1
17. Michigan St          B10   8     9.1     43.7     52.9     6-2
18. Texas                B12   7     8.7     44.1     52.8     5-2
19. Clemson              ACC   9    12.1     40.2     52.3     8-1
20. Georgia              SEC   8     9.1     43.1     52.3     6-2
21. Southern Cal         P12   8     5.5     46.7     52.2     6-2
22. TCU                  MWC   8    16.2     34.7     50.9     6-2
23. Ohio State           B10   8     5.8     44.7     50.5     5-3
24. Arkansas             SEC   8    12.3     37.8     50.0     7-1
25. Toledo               MAC   8     9.8     40.1     49.8     5-3
26. Florida St           ACC   8    14.1     35.5     49.6     5-3
27. Virginia Tech        ACC   9    11.9     37.5     49.4     8-1
28. Penn State           B10   9     8.4     40.8     49.3     8-1
29. Kansas St            B12   8     6.4     42.6     49.0     7-1
30. Cincinnati           BgE   7    17.4     31.5     48.9     6-1
31. Baylor               B12   7     3.5     45.1     48.6     4-3
32. Southern Miss        CUS   8    15.8     32.4     48.1     7-1
33. West Virginia        BgE   8     9.9     38.0     48.0     6-2
34. Georgia Tech         ACC   9    12.7     34.7     47.4     7-2
35. Florida              SEC   8     3.9     43.4     47.3     4-4
36. Tennessee            SEC   8    -2.4     49.3     47.0     3-5
37. Miami FL             ACC   8     5.0     41.6     46.6     4-4
38. Temple               MAC   8    14.9     31.3     46.2     5-3
39. Illinois             B10   9     6.2     39.7     45.9     6-3
40. Washington           P12   8     3.0     42.8     45.8     6-2
41. North Carolina       ACC   9     5.3     39.8     45.1     6-3
42. Tulsa                CUS   8     4.0     40.9     44.9     5-3
43. Auburn               SEC   9    -0.3     45.3     44.9     6-3
44. Mississippi St       SEC   8     4.8     40.0     44.7     4-4
45. South Florida        BgE   7     8.0     36.5     44.5     4-3
46. Utah                 P12   8     1.1     42.9     43.9     4-4
47. Texas Tech           B12   8     4.9     38.9     43.9     5-3
48. Arizona              P12   8    -5.1     48.6     43.6     2-6
49. Vanderbilt           SEC   8     1.1     42.3     43.4     4-4
50. Iowa                 B10   8     9.2     34.0     43.2     5-3
51. San Diego St         MWC   7     3.2     39.6     42.8     4-3
52. Rutgers              BgE   8     7.3     35.5     42.8     5-3
53. Nevada               WAC   8     4.9     37.3     42.2     5-3
54. SMU                  CUS   8     3.0     38.7     41.7     5-3
55. Pittsburgh           BgE   8     1.1     40.1     41.2     4-4
56. California           P12   8     0.8     40.1     41.0     4-4
57. Arkansas St          Sun   8     9.6     31.0     40.5     6-2
58. Air Force            MWC   8     2.3     38.0     40.3     4-4
59. Brigham Young        IND   9     3.2     37.1     40.3     6-3
60. Iowa St              B12   8    -6.8     46.4     39.6     4-4
61. Purdue               B10   8     2.3     37.2     39.4     4-4
62. Northern Illinois    MAC   8     6.4     32.5     38.9     5-3
63. Utah St              WAC   7     4.4     34.4     38.8     2-5
64. UCLA                 P12   8    -6.8     45.4     38.6     4-4
65. Louisiana Tech       WAC   8     0.6     38.0     38.6     4-4
66. Syracuse             BgE   8     0.6     37.9     38.5     5-3
67. Northwestern         B10   8     0.5     37.4     37.9     3-5
68. Virginia             ACC   8     1.5     36.3     37.8     5-3
69. Washington St        P12   8    -0.4     38.1     37.7     3-5
70. Louisville           BgE   8     0.3     37.3     37.5     4-4
71. Ohio U.              MAC   8    10.9     26.6     37.5     5-3
72. Western Michigan     MAC   9     4.1     33.4     37.5     5-4
73. Wake Forest          ACC   8     1.4     36.0     37.4     5-3
74. Hawai`i              WAC   8     7.2     29.9     37.0     5-3
75. Navy                 IND   8    -3.3     40.0     36.7     2-6
76. Central Florida      CUS   8     8.9     26.5     35.4     4-4
77. North Carolina St    ACC   8    -0.9     36.3     35.4     4-4
78. Oregon St            P12   8    -7.1     42.5     35.4     2-6
79. Maryland             ACC   8    -8.3     43.4     35.2     2-6
80. Mississippi          SEC   8    -9.4     44.5     35.1     2-6
81. Fresno St            WAC   8    -5.1     40.1     35.0     3-5
82. Connecticut          BgE   8    -0.3     35.1     34.8     3-5
83. UTEP                 CUS   8    -0.6     34.8     34.3     4-4
84. Wyoming              MWC   7     1.1     33.0     34.1     5-2
85. Marshall             CUS   9    -5.7     39.5     33.9     4-5
86. Florida Int'l        Sun   8     2.8     31.1     33.9     5-3
87. Louisiana-Lafayette  Sun   9     5.2     28.3     33.6     7-2
88. Bowling Green        MAC   9    -1.1     34.3     33.2     4-5
89. San José St          WAC   8    -6.4     39.4     33.0     3-5
90. Miami OH             MAC   8    -4.6     37.3     32.8     3-5
91. Boston College       ACC   8    -5.7     38.0     32.3     2-6
92. Duke                 ACC   8    -5.4     37.7     32.2     3-5
93. East Carolina        CUS   8    -3.9     35.6     31.7     4-4
94. Kentucky             SEC   8    -9.4     40.7     31.3     3-5
95. Ball St              MAC   9    -6.7     37.7     31.0     5-4
96. Colorado             P12   9   -16.6     46.8     30.2     1-8
97. Army                 IND   8    -0.4     30.4     30.0     3-5
98. Minnesota            B10   8   -13.8     43.6     29.8     2-6
99. Eastern Michigan     MAC   8    -2.6     32.1     29.4     5-3
100.Rice                 CUS   8   -13.0     41.9     28.9     2-6
101.Kansas               B12   8   -19.4     47.8     28.4     2-6
102.Western Kentucky     Sun   8    -2.4     30.4     28.0     4-4
103.New Mexico St        WAC   8    -3.8     31.6     27.8     3-5
104.Louisiana-Monroe     Sun   8    -5.9     33.2     27.3     2-6
105.Kent St              MAC   8   -12.1     39.0     26.9     2-6
106.North Texas          Sun   9   -11.7     38.4     26.8     3-6
107.Indiana              B10   9   -12.4     38.9     26.6     1-8
108.Buffalo              MAC   9   -10.4     36.9     26.5     2-7
109.Idaho                WAC   8    -9.4     35.6     26.1     1-7
110.Middle Tennessee St  Sun   7    -4.0     29.3     25.3     2-5
111.Central Michigan     MAC   9    -7.6     32.6     25.1     3-6
112.Troy                 Sun   7    -9.5     34.3     24.8     2-5
113.Colorado St          MWC   8    -5.4     29.5     24.1     3-5
114.UNLV                 MWC   7   -18.0     37.7     19.7     2-5
115.Akron                MAC   8   -15.5     33.5     18.0     1-7
116.Florida Atlantic     Sun   7   -19.2     37.0     17.7     0-7
117.Alabama-Birmingham   CUS   8   -17.3     34.8     17.6     1-7
118.Tulane               CUS   9   -11.9     27.8     15.9     2-7
119.Memphis              CUS   9   -16.0     29.9     13.9     2-7
120.New Mexico           MWC   8   -26.9     39.4     12.5     0-8

Here are the same standings but listed by conference:

Rk  Team                 Con   G    MOV      SOS      SRS      W-L
19. Clemson              ACC   9    12.1     40.2     52.3     8-1
26. Florida St           ACC   8    14.1     35.5     49.6     5-3
27. Virginia Tech        ACC   9    11.9     37.5     49.4     8-1
34. Georgia Tech         ACC   9    12.7     34.7     47.4     7-2
37. Miami FL             ACC   8     5.0     41.6     46.6     4-4
41. North Carolina       ACC   9     5.3     39.8     45.1     6-3
68. Virginia             ACC   8     1.5     36.3     37.8     5-3
73. Wake Forest          ACC   8     1.4     36.0     37.4     5-3
77. North Carolina St    ACC   8    -0.9     36.3     35.4     4-4
79. Maryland             ACC   8    -8.3     43.4     35.2     2-6
91. Boston College       ACC   8    -5.7     38.0     32.3     2-6
92. Duke                 ACC   8    -5.4     37.7     32.2     3-5
8.  Wisconsin            B10   8    21.5     38.0     59.5     6-2
9.  Michigan             B10   8    17.4     40.6     57.9     7-1
13. Nebraska             B10   8    12.3     42.4     54.8     7-1
17. Michigan St          B10   8     9.1     43.7     52.9     6-2
23. Ohio State           B10   8     5.8     44.7     50.5     5-3
28. Penn State           B10   9     8.4     40.8     49.3     8-1
39. Illinois             B10   9     6.2     39.7     45.9     6-3
50. Iowa                 B10   8     9.2     34.0     43.2     5-3
61. Purdue               B10   8     2.3     37.2     39.4     4-4
67. Northwestern         B10   8     0.5     37.4     37.9     3-5
98. Minnesota            B10   8   -13.8     43.6     29.8     2-6
107.Indiana              B10   9   -12.4     38.9     26.6     1-8
3.  Oklahoma St          B12   8    22.0     45.2     67.2     8-0
5.  Oklahoma             B12   8    21.5     44.1     65.6     7-1
10. Texas A&M            B12   8     9.9     46.3     56.2     5-3
15. Missouri             B12   8     5.8     47.6     53.3     4-4
18. Texas                B12   7     8.7     44.1     52.8     5-2
29. Kansas St            B12   8     6.4     42.6     49.0     7-1
31. Baylor               B12   7     3.5     45.1     48.6     4-3
47. Texas Tech           B12   8     4.9     38.9     43.9     5-3
60. Iowa St              B12   8    -6.8     46.4     39.6     4-4
101.Kansas               B12   8   -19.4     47.8     28.4     2-6
30. Cincinnati           BgE   7    17.4     31.5     48.9     6-1
33. West Virginia        BgE   8     9.9     38.0     48.0     6-2
45. South Florida        BgE   7     8.0     36.5     44.5     4-3
52. Rutgers              BgE   8     7.3     35.5     42.8     5-3
55. Pittsburgh           BgE   8     1.1     40.1     41.2     4-4
66. Syracuse             BgE   8     0.6     37.9     38.5     5-3
70. Louisville           BgE   8     0.3     37.3     37.5     4-4
82. Connecticut          BgE   8    -0.3     35.1     34.8     3-5
Rk  Team                 Con   G    MOV      SOS      SRS      W-L
14. Houston              CUS   8    22.9     30.5     53.4     8-0
32. Southern Miss        CUS   8    15.8     32.4     48.1     7-1
42. Tulsa                CUS   8     4.0     40.9     44.9     5-3
54. SMU                  CUS   8     3.0     38.7     41.7     5-3
76. Central Florida      CUS   8     8.9     26.5     35.4     4-4
83. UTEP                 CUS   8    -0.6     34.8     34.3     4-4
85. Marshall             CUS   9    -5.7     39.5     33.9     4-5
93. East Carolina        CUS   8    -3.9     35.6     31.7     4-4
100.Rice                 CUS   8   -13.0     41.9     28.9     2-6
117.Alabama-Birmingham   CUS   8   -17.3     34.8     17.6     1-7
118.Tulane               CUS   9   -11.9     27.8     15.9     2-7
119.Memphis              CUS   9   -16.0     29.9     13.9     2-7
12. Notre Dame           IND   8     9.1     45.6     54.8     5-3
59. Brigham Young        IND   9     3.2     37.1     40.3     6-3
75. Navy                 IND   8    -3.3     40.0     36.7     2-6
97. Army                 IND   8    -0.4     30.4     30.0     3-5
25. Toledo               MAC   8     9.8     40.1     49.8     5-3
38. Temple               MAC   8    14.9     31.3     46.2     5-3
62. Northern Illinois    MAC   8     6.4     32.5     38.9     5-3
71. Ohio U.              MAC   8    10.9     26.6     37.5     5-3
72. Western Michigan     MAC   9     4.1     33.4     37.5     5-4
88. Bowling Green        MAC   9    -1.1     34.3     33.2     4-5
90. Miami OH             MAC   8    -4.6     37.3     32.8     3-5
95. Ball St              MAC   9    -6.7     37.7     31.0     5-4
99. Eastern Michigan     MAC   8    -2.6     32.1     29.4     5-3
105.Kent St              MAC   8   -12.1     39.0     26.9     2-6
108.Buffalo              MAC   9   -10.4     36.9     26.5     2-7
111.Central Michigan     MAC   9    -7.6     32.6     25.1     3-6
115.Akron                MAC   8   -15.5     33.5     18.0     1-7
6.  Boise St             MWC   7    22.7     41.2     63.9     7-0
22. TCU                  MWC   8    16.2     34.7     50.9     6-2
51. San Diego St         MWC   7     3.2     39.6     42.8     4-3
58. Air Force            MWC   8     2.3     38.0     40.3     4-4
84. Wyoming              MWC   7     1.1     33.0     34.1     5-2
113.Colorado St          MWC   8    -5.4     29.5     24.1     3-5
114.UNLV                 MWC   7   -18.0     37.7     19.7     2-5
120.New Mexico           MWC   8   -26.9     39.4     12.5     0-8
4.  Stanford             P12   8    27.4     39.2     66.5     8-0
7.  Oregon               P12   8    20.7     42.0     62.7     7-1
11. Arizona St           P12   8    13.1     42.7     55.8     6-2
21. Southern Cal         P12   8     5.5     46.7     52.2     6-2
40. Washington           P12   8     3.0     42.8     45.8     6-2
46. Utah                 P12   8     1.1     42.9     43.9     4-4
48. Arizona              P12   8    -5.1     48.6     43.6     2-6
56. California           P12   8     0.8     40.1     41.0     4-4
64. UCLA                 P12   8    -6.8     45.4     38.6     4-4
69. Washington St        P12   8    -0.4     38.1     37.7     3-5
78. Oregon St            P12   8    -7.1     42.5     35.4     2-6
96. Colorado             P12   9   -16.6     46.8     30.2     1-8
1.  LSU                  SEC   8    24.5     43.9     68.4     8-0
2.  Alabama              SEC   8    27.4     40.7     68.1     8-0
16. South Carolina       SEC   8    11.4     41.5     52.9     7-1
20. Georgia              SEC   8     9.1     43.1     52.3     6-2
24. Arkansas             SEC   8    12.3     37.8     50.0     7-1
35. Florida              SEC   8     3.9     43.4     47.3     4-4
36. Tennessee            SEC   8    -2.4     49.3     47.0     3-5
43. Auburn               SEC   9    -0.3     45.3     44.9     6-3
44. Mississippi St       SEC   8     4.8     40.0     44.7     4-4
49. Vanderbilt           SEC   8     1.1     42.3     43.4     4-4
80. Mississippi          SEC   8    -9.4     44.5     35.1     2-6
94. Kentucky             SEC   8    -9.4     40.7     31.3     3-5
57. Arkansas St          Sun   8     9.6     31.0     40.5     6-2
86. Florida Int'l        Sun   8     2.8     31.1     33.9     5-3
87. Louisiana-Lafayette  Sun   9     5.2     28.3     33.6     7-2
102.Western Kentucky     Sun   8    -2.4     30.4     28.0     4-4
104.Louisiana-Monroe     Sun   8    -5.9     33.2     27.3     2-6
106.North Texas          Sun   9   -11.7     38.4     26.8     3-6
110.Middle Tennessee St  Sun   7    -4.0     29.3     25.3     2-5
112.Troy                 Sun   7    -9.5     34.3     24.8     2-5
116.Florida Atlantic     Sun   7   -19.2     37.0     17.7     0-7
53. Nevada               WAC   8     4.9     37.3     42.2     5-3
63. Utah St              WAC   7     4.4     34.4     38.8     2-5
65. Louisiana Tech       WAC   8     0.6     38.0     38.6     4-4
74. Hawai`i              WAC   8     7.2     29.9     37.0     5-3
81. Fresno St            WAC   8    -5.1     40.1     35.0     3-5
89. San José St          WAC   8    -6.4     39.4     33.0     3-5
103.New Mexico St        WAC   8    -3.8     31.6     27.8     3-5
109.Idaho                WAC   8    -9.4     35.6     26.1     1-7

Again, these rankings are far from perfect. But at least in theory, they’re pretty simple and easy to understand. They don’t get at everything we want to know, but you can quickly scan any team’s line and get a good sense of how their season has been. Each team’s rating is simply the sum of their (adjusted) margin of victory and their sum of their opponent’s average (adjusted) MOV. The SRS puts equal weight on all games, something the brain is not wont to do.

By way of example, let’s look at the craziness Texas Tech fans have had to deal with the past three weeks. Against Kansas State (SRS rating of 49.0), the Red Raiders lost at home by 7, so that grade counts as a 39 for purposes of the SRS. Then playing in Norman against the Sooners (SRS of 65.6), Tech somehow managed to upend Oklahoma by three points (+7 adjusted MOV, for an SRS grade of 72.6). This past weekend, the second worst team in the conference, Iowa State (SRS of 39.6), blew the doors off of Texas Tech, 41-7. That goes down as an adjusted MOV of -30.5, for an SRS grade of 9.1. There’s no way to make sense of a team with SRS grades of 39.0, 72.6 and 9.1 in three consecutive weeks.

One thing the SRS and almost everyone else agrees on: it’s pretty hard to separate LSU from Alabama.

  • Anonymous

    Nice to have a new contributor and more posts. Welcome, and I’ll now be following you on twitter as well.

  • Chase Stuart

    Thanks, PatrickH.  Looking forward to joining the community here.

  • http://twitter.com/tjfaust Timothy Faust

    Hi Chase, thanks for this! I’m trying to learn more about football stats–and be a smarter watcher instead of just a beer-chugging Packers fan–but I’m not sure how you derive the SOS stat.

    Also, are MOV and SOS averaged or summated?

  • Cromulent

    Chase, do you really need “hundreds of thousands of iterations” to get stable ratings? I find six iterations does well. What criteria do you use to stop?

  • Chase Stuart

    Deriving the stat takes a bit of work, done in Excel.  The short answer to your question is MOV and SOS are averaged.  After the fact, it’s easy to come up with the ratings.  After the iterations are performed (more on this in my next comment), it’s easy to calculate the SOS.  For example, why is Ohio State at 44.7 and Boise State at 41.2?  From toughest to easiest, BSU played Georgia (52.3), Toledo (49.8), Tulsa (44.9), Nevada (42.2), Air Force (40.3), Fresno State (35.0) and Colorado State (24.1); that averages to 41.2.  Ohio State played Wisconsin (59.5), Nebraska (54.8), Michigan State (52.9), Toledo (49.8), Miami (46.6), Illinois (45.9), Colorado (30.2) and Arkon (18.0).  That averages to 44.7.

    The home/road adjustment goes into the MOV component.  Again, there are a lot of flaws with the SRS that makes it imperfect, but I like that you can quickly come up with ratings that mean something and make sense.

    And how about this? Ohio State is 23.9 points better in the SRS than Indiana, and is a 27.5 point favorite at home against Indiana.  Boise State is 44.2 points better than UNLV according to the SRS, and is a 41-point road favorite this weekend.  It doesn’t always work like this — injuries, trends, teams whose reputations are inflated, etc. — but the SRS serves as a really good check (for me, at least) on what to expect from any given game.

  • Chase Stuart

    Cromulent,

    No, you don’t need to, but in Excel performing thousands of iterations is not much different than performing six.  I used to do these “by hand” in the sense that I would make all of the iterations manually in Excel, but then Sports-Reference extraordinaire Neil Paine created a video for me on how to do these in Excel.

    Be warned: This video may make you much geekier than ever before.  But Neil made this for me (I’m not trained as a programmer) to create NFL SRS ratings, and the same technique applies for NCAA ones.

    Let me know if you have any questions.

  • Cromulent

    I do much the same thing with my ratings, though being a programmer myself I do them in Delphi.

    Have you ever tested whether your adjusted margin predicts as well as an unadjusted margin?

  • Racer1

    This seems very similar to the sagarin predictive ratings. Is the main difference that you have broken it into two components or are there other distiguishing characteristics?

  • Jeremy De Shetler

    by any chance is this video accessible to the public?  I like the concept of SRS and a while ago was trying to figure out how to do the same in Excel.

  • Chase Stuart
  • Chase Stuart

    Racer1 – They should be similar to Sagarin’s ratings, but I’ve never quite understood why they diverge when they do.  As far as I know, Sagarin has never explained the formulas behind the Predictor ratings, so we’re left to guess.  But anyone can replicate the SRS ratings, and they work well across most sports.

    If you want, you can break the SRS ratings into offensive and defensive SRS ratings (although it’s just Points Scored and Points Allowed, so the usual issues with using those two stats to measure offense and defense appy), which is what Neil Paine has done with the PFR standings: http://www.pro-football-reference.com/years/2011/

  • Brad

    Chase where do you get you underlying data from.  I have been wanting to toy with this stuff but getting a clean data set is always gotten in the way

  • Bill M.

    Hi Chase…. followed you here from PFR’s blog.  My condolences in advance for the Jet’s upcoming big loss to the Bills.

  • http://twitter.com/AuburnElvis Auburn Elvis

    I see absolutely no problems with these predictive rankings. For example, it’s completely appropriate to have Auburn 27 points below South Carolina and 8 points behind Florida, since it’s obvious they’d lose to both those teams.

  • LonghornScott

    Chase,

    You may be interested in http://www.adjustedstats.com which has similar analysis but he’s actually analyzed each portion of a team’s adjusted stats and weighted them according to their correlation to winning.  If you checkout the matchup analyzer then you can actually see categorical breakdowns of how teams matchup with each other and predicted outcomes.

  • Vivek

    Vegas has Alabama as 6 pt favorites over LSU…we have them as underdogs here

  • Anonymous

    Chase, I was wondering why these numbers differ from those at College Football Reference. Do you and the CFR folks use different methodologies?

  • Chase Stuart

    Glad to see you, Bill, but please: no more drinking.

  • Chase Stuart

    Yes, going to be a fantastic game. It’s interesting: in my mind, I like Alabama to win, even with the six points.  But I wonder how much of that is per-conceived notions from before the year, as I’m struggling to think of any reason to think that on the field, ‘Bama has been more impressive than LSU.  The Tigers have faced a much harder schedule and came out looking just as good.   I think people still feel a lot of the LSU scoring is lucky or gimmicky or too-heavily dependent on the defensive and special teams, and that’s why the line is what it is.  

    My personal, worthless, subjective guess is Bama gets out to a lead early and LSU struggles to respond. But can’t wait for the game.

  • Chase Stuart

    Dr. Peter Wolfe, who publishes rankings for the BCS, also posts the game scores on his website: http://prwolfe.bol.ucla.edu/cfootball/scores.htm

  • Chase Stuart

    Frug,

    Sports-Reference’s College Football site (http://www.sports-reference.com/cfb/years/2011-standings.html) does the same SRS, but I think the two key differences are:

    1) They don’t put in caps or floors (this is why the high scoring teams will be a little higher over there)

    2) They exclude FCS games (reasonable, but since we have access to the games, I include them.)

    But obviously it’s the same philosophy, as that’s the sister site to PFR.

  • Chase Stuart

    Interest, LonghornScott.  Had not seen that site before.

  • Anonymous

    For what it’s worth, I expect LSU to win with special teams playing a huge factor. But never count out Saban.

  • Bill M.

    LOL :)

  • evan

    Hi, Chase. Enjoyed your post. 

    What do these look like if you ignore the “running up the score” correction? The reason I ask is that these system seems to have a built-in advantage for someone like LSU or Alabama over Boise State or Wisconsin, who have weaker schedules but who demonstrate their quality by outscoring weaker opponents by a huge margin. 

  • Anonymous

    Thanks for the response.

    One more related question.  While it’s entirely possible (and frankly quiet likely) that I am misinterpreting the data, as I understand it, CFR’s version of SRS uses league average as a baseline, and SRS score reflets the expected margin of victory/defeat if the team played an average team on a neutral field (i.e. Alabama would beat an average team on a neutral field by an average score of 25.17 points).

    I was wondering if you had a baseline for your system.  Since their aren’t any negative values I’m guessing it is replacement, but I wondering what that would mean. Is it just “generic FCS team”?

  • Cromulent

    Sagarin’s have a factor that weights recent games a bit more. The best public power ratings I know of are the Massey numbers – mratings.com. In fact for a few years a betting syndicate paid him for exclusive use.

  • Cromulent

    What you mean is “never count out Saban when he gets to sign several extra players per year and cut the dead weight”.

  • Xon Hostetter

    Sigh.

  • Chase Stuart

    Astute observation. There actually is a baseline, because I use all NCAA and NAIA scores, since that’s what Wolfe publishes.  So there are negative scores, but I don’t publish the scores for the FCS teams.

  • Chase Stuart

    Evan,

    Right now, Stanford and Alabama have easier SOSs but also come ahead of BSU in here because of higher MOVs.

    Boise State is actually not a team that runs up the score all that often, so this correction might help them relative to a team that tends to run up the score (and there are a lot of those).

    But the nice part about the SRS is that for the most part, you can remove whatever games you want from the schedule and just recalculate the SRS by excluding that game.

  • Chase Stuart

    I spoke with Neil, and Sports-Reference does in fact use the caps/floors. So I think the main difference is the exclusion of FCS games (and a slightly different home/road calculation).   The numbers, for the most part, should be very close.

  • Anonymous

    Sounds good.  Look forward to reading more of you at this site now that the PFR blog has been retired.