The futility of measurement, NFL combine edition

From Jonah Lehrer in the WSJ:

. . . We live in a society obsessed with maximum performance. Think of exams like the SAT and the GRE. Though these tests take only a few hours, they’re supposed to give schools and companies a snapshot of an individual’s abiding talents.

Or consider the NFL Scouting Combine, in which players entering the draft perform short physical and mental tasks, such as the 40-yard dash. The Combine is meant to measure physical ability; that’s why teams take the results so seriously.

It’s easy to understand the allure of such maximal measures. They don’t take very long, so we can quantify many people. Also, they make assessment seem relatively straightforward, reducing the uncertainty of selecting a college applicant or football player.

But as Mr. Sackett demonstrated with those supermarket cashiers, such high-stakes tests are often spectacularly bad at predicting performance in the real world. . . .

Even the NFL Combine is a big waste of time. According to a recent study by economists at the University of Louisville, there’s no “consistent statistical relationship” between the results of players at the Combine and subsequent NFL performance.

Is the NCAA a coercive cartel?

Gary Becker:

[T]he NCAA sharply limits the number of athletic scholarships, and even more importantly, limits the size of the scholarships that schools can offer the best players. NCAA rules also severely restricts the gifts and housing players are allowed to receive from alumni and others, do not allow college players to receive pay for playing for professional teams during summers or even before they attended college, and limits what they can be paid for non-playing summer work. The rules are extremely complicated, and they constitute hundreds of pages that lay out what is permitted in recruiting prospective students, when students have to make binding commitments to attend schools, the need to renew athletic scholarships, the assistance that can be provided to players’ parents, and of course the size of scholarships.

It is impossible for an outsider to look at these rules without concluding that their main aim is to make the NCAA an effective cartel that severely constrains competition among schools for players. The NCAA defends these rules by claiming that their main purpose is to prevent exploitation of student-athletes, to provide a more equitable system of recruitment that enables many colleges to maintain football and basketball programs and actively search for athletes, and to insure that the athletes become students as well as athletes.

Unfortunately for the NCAA, the facts are blatantly inconsistent with these defenses.


Things that are self-recommending – football statistics edition

Bill Connelly — the college football expert for Football Outsiders — has a new SBN Blog, Football Study Hall; such a fact is high on the list of self-recommending things. The idea for the site is to provide a one-stop shop for advanced stats for college football, with a bit more of a fan-flavor than some of the other “stat heavy” sites out there. And Bill’s already got some good stuff up:

Success Rate: A common Football Outsiders tool used to measure efficiency by determining whether every play of a given game was successful or not. The terms of success in college football: 50 percent of necessary yardage on first down, 70 percent on second down, and 100 percent on third and fourth down.

The idea behind Success Rate is simple: every play is deemed successful or unsuccessful based on down, distance and yardage gained. Plays on first, second and third downs (and fourth, for that matter) all have as close to the same success rate as possible (between 40% and 45%).

To see what Success Rate tells us, exactly, let’s have a look at it in action. Two notes before moving forward:

1. Any reference to Success Rates as it pertains to rankings eliminates garbage time plays. Rankings are derived from plays that took place while the game was “close”: within 28 points in the first quarter, 24 in the second, 21 in the third, or 16 in the fourth.

2. As a frame of reference, the average success rate for FBS teams from 2005-10 was 41.6%. . . .

Ten Best Single-Season Success Rates, 2005-10
1. Hawaii (2006): 60.8%
2. Texas Tech (2008): 56.1%
3. Wisconsin (2010): 55.3%
4. Oklahoma (2008): 55.2%
5. Florida (2007): 55.0%
6. BYU (2008): 54.8%
7. Missouri (2008): 54.7%
8. USC (2005): 54.1%
9. Boise State (2010): 54.0%
10. Texas (2008): 54.0%

One of my favorite things about college football is how there are so many different ways to move the chains. Seeing a team like Wisconsin or Navy on the list above would be no surprise — they’re the prototypical grind-it-out, three-yards-and-a-cloud-of-dust teams. But while Wisconsin locked down the three-spot, a run-and-shoot offense took the top ranking, while spread teams filled out most of the Top 10. Invention in college football derives from trying to find different ways to gain five yards, and in college football, there are many, many different ways.

(Speaking of invention … it really is incredible to see just how far ahead of the curve the Big 12 was when it came to the spread. Of the top ten teams above, four were from the 2008 Big 12 alone. That was truly the perfect confluence of innovation and skill position experience.)

Call it the Mike Leach effect, but he’s right that: other than a few other elite offenses and talent laden Florida and USC squads, the Big 12 had the brightest offenses of the decade. In any event, head over to Bill’s new spot.

Smart Links – grit, brain injuries, football playoffs – 3/15/2011

Blind spots of economists. What are some blind spots for football coaches (and fans)?

What traits predict success? Punchline:

The second takeaway involves the growing recognition of “non-cognitive” skills like grit and self-control. While such traits have little or nothing to do with intelligence (as measured by IQ scores), they often explain a larger share of individual variation when it comes to life success. It doesn’t matter if one is looking at retention rates at West Point or teacher performance within the Teach for America program or success in the spelling bee: Factors like grit are often the most predictive variables of real world performance. Thomas Edison was right: even genius is mostly just perspiration.

Taken together, these studies suggest that our most important talent is having a talent for working hard, for practicing even when practice isn’t fun. It’s about putting in the hours when we’d rather be watching TV, or drilling ourselves with notecards filled with obscure words instead of getting quizzed by a friend. Success is never easy. That’s why talent requires grit.

The 2011 Sloan MIT Conference Papers are up. There are no football papers because football is too complicated. Seriously.

Hockey is beginning to come around on head injuries as well.

“It turns out” you read this.

Dr Saturday’s college football playoff proposal. Brian Cook offers some thoughts; my old playoff vs. BCS vs. who knows rumblings can be found here and here.

The day the movies died. Do people buy this? (1) I don’t know if I agree that movies will continue to get worse (the internet provides more avenues for niche audiences, etc) and (2) I’m wearing a bit thin on the after-this-movie-the-movie-industry-changed (Jaws, Star Wars, Top Gun, Raiders of the Lost Ark, Transformers 2, etc.). The movie industry changed a lot when the old contract system died, but other than that I’d guess it’s societal forces and changes in tastes. We’ve had little pockets of artistic flurry before.

And, after the jump, something depressing:


Tom Brady, et al. v. National Football League, et al.

Read the complaint here. There is more to say about the NFL CBA dispute (here’s a summary of some, but not all, of the legal issues here), but that’s one of the issues here: No one knows what to think or who to listen to. As interesting as the legal issues are (and they are pretty interesting, though not as interesting as, say, the old free agency fight the players won), the fascinating thing to me is the complete breakdown in the quality of media coverage. This is not surprising or even fair to the genuinely diligent reports who have gotten to where they are and spend their days talking to football people about football things — who is hurt this week, will Donovan McNabb be traded, is Bill Belichick videotaping someone, what blitz package will Pittsburgh use — suddenly are forced to understand rather complex anti-trust and labor issues. Indeed, the labor law and anti-trust law are individually extremely complex, and their interaction — with a splash of straight up negotiating bravado — makes this all difficult to wrap your head around, particularly in football, which is built on easy narratives.

As a result, suddenly reliable sources (“Fred Taylor won’t start this week because of his ankle problem, though the team isn’t talking”) is suddenly completely uninformative, even if honest (“I’m not a lawyer but Bob Kraft told me the deal gets done tomorrow”). Nothing that has happened is at all surprising, and yet the media acts like it has been and has consistently failed to bring any coherence. That’s why the people to listen to — and I’ve yet to completely find them, and it’s not necessarily me who has not dived in to the details enough — is someone who knows complex labor negotiations and the state of antitrust law on reasonable restraints of trade, not the guy who accurately broke Spygate.

But again, I can’t fault the Chris Mortensens or Adam Schefters — they are in the same positions that fans, with an extra monetary penalty if there is no season. ESPN isn’t in the tank for either side (at least I don’t think), but instead is in the tank for getting a deal done: if there’s no football ESPN loses money, and if it loses money there might not be a need for Mortensen or Schefter, or certainly not the many people operating on the fringes who want to reach that place. They, like you and me and everyone else, don’t really care what the split of $9 billion in revenue is: in the immortal words of David Cross, we don’t care what else is going on in the world, why won’t somebody pick up the ball and play some damn football.

Update: James Surowiecki sides with the players:


Michel de Montaigne on bloggers versus journalists versus aggregators versus SEO

It is more of a job to interpret the interpretations than to interpret the things, and there are more books about books than about any other subject: we do nothing but comment upon one another. The world is swarming with commentaries; of authors there is great scarcity.

From On Experience.

Smart Notes – Bill Simmons, football in 1938, Mark Cuban – 2/19/2011

Football has changed. The New Yorker (yes) writes about one of the handful of most famous football players of the day, in 1938, and this is what it had to say:

Bill Platt is just about what you’d expect the captain of a Yale football team to be — tall, heavy, blond, and offhand. We stopped by to see him in his rooms at Berkeley College, one of the newest of the Yale dormitories, a few nights before the Harvard game and thoughtlessly kept him up an hour past bedtime. Platt played his last game of football Saturday; next year he’ll have to start thinking about a career, which may be either politics, a tradition in his family, or law. He’s not in any hurry to decide. As an undergraduate, he spent his summers fishing in Maine; he said it seemed like no time at all each year before he was reporting to Ducky Pond again for fall practice. …

Once, Platt said as we got up to go, he’d decided idealistically that he needed something besides football as a character builder, and the spring he left Andover he shipped as deckhand on an American freighter to Shanghai and back. It was, he thought, the most interesting experience of his life. “Did it help your character?” we asked. “No,” he said.

How should you split your rent? First read the study, then check out the calculator.

Bill Simmons creating the Bill Simmons sports and literary site, and it actually sounds kind of great. Be sure to check out Quickish/Shanoff on this. (Very happy Klosterman is on board.) Even though this is bankrolled by ESPN, I look at this as similar to Freakonomics going off the NY Times website. For all the hoopla about HuffPo/AOL, etc, I think this is really the model: a multi-platform channel that focuses on web content but offers podcasts, books, ebooks, and other media “consumption,” with an actual editorial voice. Again, read Dan’s take on this, who probably thinks about this stuff more than anyone on planet earth (not a surprise given his Harvard MBA and years in the online media world.

Why is college so expensive? Here’s a dialogue from the Times with David Leonhardt; Matt Yglesias with an old post and a critique of the “Olive Garden Theory of Higher Education.”

Bob Sanders, released. Sad story, but (a) injuries are brutal and (b) in the NFL you must be ruthless.

Don’t worry, Bryant is on the case. Supposedly HBO’s real sports is investigating the Cam Newton drama.

I still don’t buy that Mark Cuban’s playoff idea will go anywhere, even if he did set up some kind of entity.

Sentences to ponder, NFL labor lockout/dispute edition

The owners overspent on unnecessary stadiums, and now they want the players to work more for less pay to help pay down the debt. That’s your entire labor dispute in one sentence.

That’s from the Mises blog, and the post is interesting throughout. The following is also important for understanding the NFL as government sanction and sponsored business entity:

The NFL is really two distinct products. There’s the in-stadium product represented by Jerry World and its taxpayer-financed brethren. And then there’s the way most people consume football, the television product produced by the major broadcast networks and ESPN. NFL-TV is a great product whose popularity remains high. NFL-Stadium is struggling to pay the mortgage.

Regardless of how this labor dispute plays out, the future of football has to be in NFL as TV (and internet video, i.e. ESPN3), rather than the stadium experience. I don’t find going to pro games all that enjoyable. I much prefer going to college games (or even baseball games, and I don’t really even like baseball); such games are typically a blast, largely because they are a much more personal experience. Indeed, the only thing I really like about going to a pro game in person is that I get a better angle with which to watch the game, and TV and internet video options are beginning to finally provide more options in this respect. At the end of the day, for the NFL, the TV experience, whether at home or in a bar, is really superior.

The NFL business model is bizarre and discussions with most people about the labor dispute quickly degenerate because of the complexity as they take up the slogans of one side or the other. The analysis is neither laissez-faire capitalism nor typical labor economics nor even a the economics of a regulated public utility; it’s some weird unknowable mixture of a cartel and consumer business, and that makes all of these disputes both fascinating and maddening.

It’s not really a shock that there is a strange labor dispute going on, unilaterally initiated not by the union but by the owners in a way that few can analyze satisfactorily. Indeed, count up the factors: (i) The NFL is heavily subsidized by the government, (ii) with respect to the players selling their services, it’s a monopsony, and (iii) for the rest of us (and with government and to a lesser extent court approval), it’s an oligopoly. Put together, that is unlikely to lead to an efficient market. In the end, most of us just want to see someone pick up the damn football and play.

Worldly wisdom

For it is in the essence of his behaviour that he should be eccentric, unconventional and rash in the eyes of average opinion. If he is successful, that will only confirm the general belief in his rashness; and if in the short run he is unsuccessful, which is very likely, he will not receive much mercy. Worldly wisdom teaches that it is better for reputation to fail conventionally than to succeed unconventionally.

That is Keynes, not talking about coaching football, though it certainly applies.

NY Times vs the HuffPo

Tom McGeveran asks an important question, in his analysis of the AOL-HuffPo deal:

What is it about the environment of traditional journalism that makes it so that readers are more likely to interact with the Huffington Post reblog of a New York Times article than they are with the article itself?

The answer to this question, I think, is also a key part of the reason why the NYT paywall is a bad idea.

It’s worth using a specific example here, so let’s take Dave Pell’s suggestion and look at the NYT’s Olbermann scoop last night, and HuffPo’s reblog of it. When Pell first tweeted the comparison, the NYT blog had no comments, while the HuffPo blog had “hundreds of comments/likes.” Now, the NYT post is up to 93 comments, but the HuffPo post is still miles ahead: 2,088 comments, 1,392 likes on Facebook, 340 Facebook shares, 89 tweets, and 52 emails. All of which figures are easily visible in a colorful box at the top of the story. . . .

Still, the difference between the two pages is much starker than it needs to be: the NYT page is like walking into a library, while the HuffPo page is like walking through Times Square. The HuffPo page is full of links to interesting stories elsewhere on the site — about Egypt, or the kid in the Superbowl Darth Vader ad, or the stories my Facebook friends are reading. And there are lot of links to media stories, too; each one has a photo attached. (more…)