How do you predict whether someone will be a good college coach?

We looked at the head coaches, offensive coordinators and defensive coordinators for the 66 major-conference schools, plus Notre Dame, and found that with a few high-profile exceptions, NFL experience isn’t a great recipe for success on Saturdays. Most notably, Pittsburgh’s Dave Wannstedt, the former Bears and Dolphins head coach, resigned under pressure in December. Meanwhile, California, Virginia and Oregon State all finished below .500 despite the gaudy NFL résumés of their coaches. The staff that logged the most NFL years was Stanford’s. New 49ers coach Jim Harbaugh and his coordinators, David Shaw and Vic Fangio, combined to coach in the NFL for 35 years, and the 12-1 Cardinal were better for it.

But Monday’s BCS championship game was more proof that coaches can do just fine without NFL grooming. Of the game’s two coaches and four coordinators, only Oregon defensive coordinator Nick Aliotti made a pit stop in the NFL, while Auburn’s troika was one of 21 that’s never worked on Sundays.

That’s from the Wall Street Journal. I’d like to see what factors do predict winning, preferably by running a regression analysis of BCS conference coaches, with, Y, the dependent variable, being winning percentage (with, say, a minimum of three years coaching). I’m curious what Xs, or independent variables, would be statistically significant. A non-exhaustive list of candidates:

  1. Years of NFL coaching experience.
  2. Years of previous head coaching experience (any level).
  3. Years of coordinator-level experience (college or higher).
  4. Rank of offenses/defenses in scoring, total yards, and yards per play.
  5. Rank of offenses/defenses in rushing or passing, individually, in adjusted yards per attempt.
  6. Years of total college experience (proxy for recruiting experience?).
  7. Winning percentage at prior coaching stops.
  8. Rank of punting and kicking units in net punt averages and kickoff/kickoff return averages.
  9. Red zone touchdown percentage of offenses and defenses at prior coaching stops (use both regardless whether offensive or defensive coach).

I’m sure there are other plausible ones; please add on in the comments. Also, please tell me why the test wouldn’t work if set up this way, and how it could be improved. I’d actually be surprised if any of these factors turned out to be statistically significant, but I’m also not aware of anyone working something like this out.

  • Peter

    I would suspect recruiting is a key indicator. perhaps yearly recruiting class rank?

  • http://www.twitter.com/badger_dave_ BadgerDave

    Should leave off the first three or four years to allow for an adjustment phase for recruiting to catch up.

  • J. Jorg

    Is raw ‘winning percentage’ is a fair dependent variable?

    I wonder if lesser ‘resumed’ coaches would be injured by assuming control over ‘rebuilding’ project schools. Would winning percentages after 2 or 3+ years be more representative of actual success?

    Additionally, maybe the answer is not ‘winning percentage’ of the coach but comparative winning percentage of the coach relative to his peers (similar programs: era/prestige/market scope/etc) and/or to his predecessors and, if you data set contains historical coaches, successors. Reasoning, 8 wins for a team that has never consistently won 4 would demonstrate a potential for “great” coaching where 8 wins in a school that consistently wins 10 could demonstrate a potential for “poor” coaching.

    It would be an interesting analysis regardless.

  • http://brophyfootball.blogspot.com brophy

    1. The staff he surrounds himself with
    2. The ability of that staff to recruit talent rich areas of the nation
    3. The strength of schedule for the season
    4. The amount of buzz and effect on culture by the Head Coach during the first 18 months

  • James

    For offensive coaches(Meyer, Rich Rod, etc)…how successful their defense is.

    For defensive coaches(Rex Ryan)… how successful their offense is.

    “Should leave off the first three or four years to allow for an adjustment phase for recruiting to catch up.”

    Disagree, some coaches have a knack for doing well from the get-go.

  • Kevin

    Did you hear Dave Brandon at his press conference say that he had someone put something like this together? He didn’t mention the specific statistical tests (or how exactly they computed correlations), but he said that familiarity with the location (for purposes of recruiting) correlated highly with success.

  • Scottwood

    As mentioned above, Michigan’s AD Dave Brandon said that before they hired Hoke they did an analysis of successful college coaches over the last 25 years and found several correlations. On the surface, Hoke’s resume doesn’t look all that impressive. But, I would be interested in reading that report. Similar to traditional stats, I imagine looking at a resume simply from a surface level will leave out a plethora of information.

  • KungFuPanda9

    I think J.Jorg has hit on an important variable, i.e., relative strength of program. Using Chizik as an example, he took Auburn to an undefeated season as a defensivie coordinator. He then took Texas to an undefeated season as a defensive coordinator. But then as head coach at Iowa State he struggled. (But then who hasn’t?) He then takes the HC job at Auburn and has a winning season and then goes undefeated.

    At least one significant variable seems to be the relative strength of Iowa State relative to Auburn and Texas.

  • Displaced Cane

    You should probably factor in the success of the program up to the point where the coach in question took over. Taking over a 8-5 team and leading them to a 10-3 record means something different than doing the same with a team that finished 4-8 the previous year.

    I like the idea of controlling for recruiting, but I don’t know how to meaningfully do that. And I don’t know about waiting 3 years for the coach to recruit his own players since some coaches don’t last that long and you’re throwing out 3 years of meaningful data (records count, even if you’re coaching with someone else’s players).

  • Ryan

    We look at NFL coaching experience. Given Gus Malzahn’s success how about looking at the other extreme: High School Coaching experience.

  • dale

    It appears to me that many head coaches and ADs are hiring coaches with NFL ties in order to sell recruits that they will be coached in NFL style offensive and defensive systems by coaches with NFL backgrounds with the idea that being in that type of program better prepares them for the NFL.

    Worked for Carroll at USC and Harbaugh at Stanford.

  • quigley

    Could work.
    Data sets may be different in different eras also so looking at 1970’s predictors may not be valid today.

    Need pick variables carefully because data points should be clustered by hiring period and I’d guess there are only ~15 hires at the D-I level annually. Like the idea of comparing performance to historical average.
    Could set up hires made in 2000-2004 as primary data set. Then can use hires from 2005-2009 as validation cohort. Huckleberry at Barking Carnival or Bill C at Rock M Nation may have resources to do it. I’d be shocked if none of the headhunting firms has done this.

  • blusage

    I believe Michigan’s AD Dave Brandon spoke about a study he found which analyzed why coach hirings FAILED. The main reason was not having any ties or familiarity with the region they were plopped down into. So even a genius coach has a high probability of failing at a school and region he knows nothing about.

    The inverse, which is that local ties leads to success, is not necessarily true. You need the familiarity with the region for sure, but then the genius part comes into play.

  • Brad

    As others have mentioned I think two control variables should be added to any analysis.

    Winning percentage over the past 3-5 years at the school and winning percentage over the last 20-30 years at the school.

    The first captures the situation the coach inherited and the second measures the schools relative place in the football pecking order.

  • Andrew

    First time poster, but one thing that occurs to me is something like “differential in win percentage vs. previous head coach.” Say, for example, you take Coach X’s win-loss record for his first two years (two years to help reduce the effects of random chance) and compare it to that of his predecessor during the predecessor’s last two years. It’s not perfect by any means, but I submit that this would help you get close to quantifying the idea of “is this a coach who comes in and makes programs better, or worse,” which I would think would be a strong predictor of future success.

    E.g.: if Coach #1 went 6-7 and then 3-10 and lost his job, and then Coach #2 came in and posted records of 5-8 and then 8-5, Coach #2 would have taken the program from 9-17 to 13-13, a “win differential” of +4. Of course, it is somewhat simplistic, and there are tons of other variables at play here (as with any independent variable you try to pin down), but I think this might be statistically significant in trying to predict whether Coach #2 will make another program better.

    Thoughts?

  • Charles

    NFL HC/C experience is not a necessity. Look at Charlie Weis at Notre Dame. An offensive genis with 3 super bowl rings. But he could not run a college program. However, he can be better with Muschamp in charge due to his experience.

  • dazz

    I would think there is a large standard deviation.
    If not, then maybe it’s a non linear equation. Call it the Chizik factor or BCS Chaos theory by Ian Malcolm.

  • Hermus

    I’ve played around with coaching ratings, and one of the primary considerations is the state of the program when the coach takes over. Is the coach being handed the keys to a Ferrari, or is it a rebuilding job? If the former, winning percentage is fine for the Y variable. If the latter, the rate of increase in winning percentage over his first few years with the program is a better measure (slope of the regression on annual winning percentage). The challenge is how to determine which factor is more applicable to which coach.

  • Young buck

    There are so many variables. You listed great ones and i agree with Hermus also. Look at gene chizik ( i don’t like him, im from iowa). At iowa state he did absolutely nothing with the program and i highly doubt he cared much. He gets hired at Auburn where there is a unbeleiveable amount of talent, and he has a great offensive coordinator. Gene chizik had very little to do with the auburn success and still got the coach of the year. I think you need to look for wins, and what type of programs the coach has taken over, who hes worked with. Rich rod was a good coach at WV and won games with the spread, but it didnt transfer at michigan. I think the coach needs to adjust the system to the players, not the players to the system.

  • Rob

    Another thing I’ve noticed is #unique positions coached (ie, positional/coordinator roles) might also be a factor. This factor can possibly be viewed from the frame of “promotion rate” which would hopefully be indicative of coaching aptitude. Just like any data set tho you will find some outliers (I think both Grobe @ WF and Ferentz only held one position coaching job before becoming HC’s…probably a boatload MORE too!)

    I have been very interested in this exact same topic for quite sometime. However, I still need to strengthen my statistical chops before I could even imagine building a LR model like that… Especially when identifying the appropriate factors.

  • Big Chief

    I think a logistic regression would be better here. The Y variable would be the probability of success.