Thoughts About Drafting Quarterbacks

So let’s say you’re [Chicago general manager] Pace, and you’ve determined that you really want Trubisky. You call the 49ers and trying to work out fair compensation if the Browns do not pick him at one. You think Trubisky’s going to be the long-term Bears quarterback, starting in 2018 or later…. The market for quarterbacks is always weird…. For quarterbacks, NFL history says you pay Four Seasons prices. That’s why I can’t fault Pace for what he did. He wasn’t willing to risk losing the guy he loved.” — Peter King, The MMQB

I like my odds

The biggest story from the 2017 NFL draft was the surprise move by the Chicago Bears to pay a hefty price to the San Francisco 49ers — the 3rd overall pick plus the 67th and 111th picks, plus a 2018 3rd round pick — in order to move up a single spot in the draft, where they selected North Carolina quarterback Mitchell Trubisky. It was a stunning trade, in no small part because Chicago, having just spent significant money to acquire free agent QB Mike Glennon, was not an obvious candidate to draft a quarterback let alone pay a steep price to do so. But the Trubisky trade wasn’t the only trade involving a first round QB, as both the Chiefs and Texans also paid premiums to get their (hopeful) quarterbacks of the future, namely Patrick Mahomes and Deshaun Watson, respectively. Per both the traditional Jimmy Johnson trade chart as well as Chase Stuart’s updated version, both trades were expensive for the teams trading up: the Chiefs paid 120 cents on the dollar under the Jimmy Johnson chart and 170 cents on the dollar under Chase’s version, while the Texans paid approximately 125 cents on the dollar under the Jimmy Johnson chart and 154 cents on the dollar under Chase’s chart.

Was this rational? Well, a common response — and it’s the response baked into Peter King’s excellent MMQB piece from today — is that if the QB is a success and becomes a star and leads the team to Super Bowls, then the price tags for these teams will be cheap. The counter is essentially an extrapolation from the trade value charts: while that may be true ex post, on an ex ante basis the price tags are hefty — and later success won’t change that — particularly given that we all know many highly drafted QBs nonetheless fail.

I am not particularly interested in proving one perspective right or wrong (and I have not watched enough of Trubisky to really evaluate him), but I am quite interested in how to think about this question, as it’s one of the most vexing questions in all of sports and it’s a discussion that seems to frequently get off track. Further, it’s clear NFL teams approach QB draft picks entirely different than they do every other person; as Bill Barnwell wrote about and then separately said to me, “Teams are working with entirely different trade parameters for QBs than they are for players at most other positions.” To begin the process of answering the question, I wanted to lay out a handful of statements that I think (believe?) to each be true, although some point in different directions.

  • The most important thing is to understand the concept of “base rates,” or Bayesian probability. The basic idea — which can quickly get complicated — is that if you think there is a 95% chance of something happening (say, of Mitch Trubisky being the next Peyton Manning), the probability of that is not 95%, it is instead a blended probability that includes your estimated probability and the base rate, or historical success rate, for similar selections. Around 50% of 1st round QBs fail, and that number goes up for every round. (UDFAs actually produce more starters than 6th and 7th rounders, but that’s in part because there are so many more QBs that enter the league as UDFA than late rounders) As Dan Kahneman explained in Thinking, Fast and Slow (which every draftnik should read):

    [Bayesian reasoning] is named after an English minister of the eighteenth century, the Reverand Thomas Bayes, who is credited with the firsy major contribution to a large problem: the logic of how people should change their mind in light of evidence…. There are two ideas to keep in mind about Bayesian reasoning and how we tend to mess it up. The first is that base rates matter, even in the presence of evidence about the case at hand. This is often not intuitively obvious. The second is that intuitive impressions of the diagnosticity of evidence [i.e., the accuracy of your evaluation of how likely a QB prospect is to succeed] are often exaggerated…. The essential keys to disciplined Bayesian reasoning can be simply summarized:

    (1) Anchor your judgment of the probability of an outcome on a plausible base rate.

    (2) Question the diagnosticity of your evidence.

  • When I looked at the evidence most recently, the best data I could come up with was that QBs selected in the first round “succeeded” (fairly loosely defined; not limited to All-Pros and Super Bowl champs) around 50% of the time. The odds were slightly higher for top 10 picks and then slid down as the first round continued.
  • If you haven’t guessed, the failure to account for base rates is the flaw in King’s argument about Chicago (at least as presented; whether it was in fact a bad deal is something we’ll get to): Even if Pace is “sure” Trubisky was his guy, that doesn’t imply that no price is too steep as the base rates will anchor that probability down to something closer to 60-65%, even assuming a very high degree of confidence by the Bears. Ignoring base rates is one of the largest draft fallacies, and while it afflicts decisionmakers on both sides of the ledger it is most commonly an issue for those who are over optimistic about specific prospects. So trading up for a QB can be very risky.
  • However, having a good starting QB is extremely valuable. It is essentially impossible to win in the NFL with a bad QB. It is difficult, though not impossible, to win without a serviceable or replacement-level starter QB. It is much, much easier to win with a top tier QB. The position is essentially the most valuable one in all of sports.
  • Furthermore, a good QB on his rookie deal is maybe the most valuable asset in all of sports. Hitting on a rookie QB presents enormous organizational-level rewards: not only do you find your franchise savior, the CBA and rookie wage scale — plus the fifth year option for first rounders — results in massive savings for the franchise, money that can be spent on other assets. Barnwell: “No player in the NFL is creating more surplus value for his team than Dak Prescott, who appears to be an above-average quarterback in line to make less than $2 million combined over the next three years. Not far behind him is Carson Wentz, who should still be a massive bargain despite being guaranteed $21.8 million over the next three seasons (with a fifth-year option to come). When you consider that the free market guaranteed Brock Osweiler two years at an average of $14 million per year and Mike Glennon $18.5 million for one season, even the $7.3 million Wentz is going to earn is a relative pittance.”
  • Finding a serviceable QB is a real challenge, one exacerbated by the concept of a draft itself. When combined with a scarcity of good, starter-level QBs in the NFL more generally, the draft hands over to chance whether or not a team will ever hit on a rookie QB: A team needs a top pick in a year when there are such QBs available, and those sorts of QBs only come around every few years. In other years the team either must weigh drafting a QB too high or taking other needs (and teams drafting near the top typically have many). Skill and organizational stability also play roles, but there is a lot that simply comes down to chance — not to mention a negative cycle where downtrodden teams reach for first round QBs and thus propagate the cycle of failure->fire coaching staff/GM->new regime reaches to pick new “savior” QB of the future->failure->repeat, while a few teams go from one top-flight QB to the next, like the Packers from Favre to Rodgers or the Colts from Manning to Luck. Exacerbating this cycle is that elite QBs almost never come onto the free agent market.
  • Nonetheless, it is at least theoretically possible to compare the odds and cost of trading up and paying a premium for a first round QB — and thus taking a 50-70% chance on him becoming an effective starter — versus recreating that 50-70% probability by accumulating QBs in later rounds, i.e., the 2nd through 4th round. (Tom Brady aside, rounds 4-7 rarely produce quality NFL starters, and while there are a fair number of productive UDFAs, that is a much larger pool.) The analogy here is that most NFL draft picks are 2- or 3-star high school recruits, but that’s a quirk of the numbers because there are many more 2- and 3-star recruits by volume than there are 4- and 5-star recruits; a given 4- or 5-star recruit is significantly more likely to be drafted than a 3-star recruit.

Based on the above, below are some tentative conclusions:

  1. Good QBs, and rookie QBs in particular, are extremely valuable, and while the market price to acquire them is high it’s at least arguable that the potential payoffs justify even very steep price tags, even if the binary probability of success versus failure remains near 50% (or worse for later rounds). In other words, it still might be a coin flip whether Trubisky, Mahomes or Watson succeed, but if they do they would produce an incredible amount of surplus value to the entire organization in terms of on-field production as well as relative cost savings versus free agents and stability (i.e., job security).
  2. Nonetheless it is still possible to “overpay” (typically in terms of trade value) for rookie QBs, an issue that is most likely to arise if a team ignores the impact of base rates, i.e., the cost of a trade looks much more attractive if you think the likelihood of success of a potential QB is 85-95%, whereas even if that is your organization’s internal judgment — and I’d question you if it was — the true probability is somewhere closer to 60-70% given the impact of base rates. The same analysis applies in later rounds as well, applied to lower base rates of success (often dramatically lower), albeit compared to lower costs.
  3. It may be possible to more cost-effectively replicate the odds of hitting on a first rounder by taking multiple shots at QBs in the 2nd through 4th round range, until the organization hits on a QB. Seattle effectively used a version of this in hitting on Russell Wilson.
  4. Nonetheless, the most high percentage pool for identifying future star NFL QBs remains the first round, and that’s even after factoring in “first round” QBs who had no business being picked in the first round (EJ Manuel, Jake Locker, etc.).
  5. Finally, none of the above matters if your organization and coaching staff lacks the structure, knowledge and expertise to develop and coach a young quarterback. Without that you are doomed.

Sentences of the Day: Bill Walsh Edition

TIGHT END — Ideal size: 6-41/2, 245

Requirements for a tight end depend heavily on the system being deployed. It’s almost a necessity to find the athlete who best fits your system of football….

[…]

Now there is one more type of tight end — the great, all around type who is a Hall of Fame type. He is so gifted that he can do all of the things you would usually require two types of tight ends to do. That type of player makes this a unique position in the NFL. One man who can do all these things, the great, all around tight end becomes the essence of the National Football League. And there have been very few — John Mackey, Mike Ditka, Jackie Smith, are the only two who have made the Hall of Fame.

Interestingly, I believe Tony Gonzalez of Cal this year has the potential to become that type of all around great tight end.

Pretty good prediction. You can read the entire thing here, and read the full set of Walsh’s analysis of various positions here, which are interesting throughout.

The Real Deal: Longtime NFL O-Line Coach Larry Zierlein Breaks Down NFL Draft Prospects

Great stuff from the Sideline View, run by my friend Lance Zierlein, where his Dad, Larry Zierlein — longtime NFL and college O-Line coach, most recently of the Pittsburgh Steelers, breaks down the top NFL Draft prospects at center, guard and tackle.

Bonus: A fantastic presentation from Coach Zierlein on pass protection fundamentals, techniques, and concepts.

Down with the draft

Like most everyone else, I’ll tune into the NFL draft. It’s a big media spectacle, and certainly draws a lot of interest, but does it make any sense? And for who? NFL Commissioner Roger Goodell warns us:

In the union lawyers’ world, every player would enter the league as an unrestricted free agent, an independent contractor free to sell his services to any team. Every player would again become an unrestricted free agent each time his contract expired.

Trust us, the draft works great

Rather than some corrupt dystopia, this sounds a lot like the free market and — I don’t know — real life to me, even setting aside the fact that most of Goodell’s arguments are hypocritical (“some players might get paid less” yet the players support the proposal). The draft is extremely anachronistic (and autocratic), and, with free agency now otherwise widespread, probably is not nearly as significant as it once was. Chase Stuart has explained that NFL teams derive something like 50-60% of their “production” from players acquired in the draft versus free agency (with 40% on the low end and 80% on the high end), and my guess is this is skewed given the very short time expectancy for NFL players. So the draft only makes up a piece of how good a team is.

So what if there was no draft? The current draft system is bizarre, and the mystery leads to bizarre turns of events where teams draft players they never met with because they didn’t want to tip off other teams — hiring guys based on his resumes but without the interview) or you uncomfortable watch someone like Texas A&M runningback Leeland McElroy sit through hours of waiting on live television on his way to becoming the lowest drafted player to actually attend. Instead of this, once the NFL imposed waiting period was lifted (call it “Signing Season,” and could run from, say, March through May) teams would begin working out players and giving them offers. Players won’t have to wonder, “Will I be a first round guy? Or will I slip to the third round?” Instead they will know that, say, the Panthers have offered them three years at $15 million, while the Saints have offered four years at $25, or in the case of some other player maybe two years for a total of $1.2 million. Rather than the smoke screens and forced marriages the draft creates, you’d have actual, objective value out there, and players would be more likely to go to teams that fit them.

But what of the teams, and, most importantly, what of that bogeyman, “competitive balance”? As I mentioned above, close to half of all teams (in terms of production) is determined by free agency now anyway, and the NFL still has a reputation as a “balanced” league. More importantly, however, the draft adds very little to competitive balance. As Richard Thaler has noted, having a high draft choice is more of a “Loser’s Curse” than anything else, with the heavy salaries and high risk of a top pick doing more to destroy competitive balance than to help it. And the NFL already has the two best mechanisms for keeping and establishing competitive balance: the salary cap and revenue sharing. A recent study discussed the idea of sharing venue revenue as a way to remedy any remaining competitive balance issues; the draft does little to help. And I’m not persuaded that small market teams will suffer as much as Goodell claims; they said the same thing before free agency, and they seem to be doing fine with that (in the last eleven years, small or mid market teams in St. Louis, Baltimore, Tampa Bay, Indianapolis, Pittsburgh, New Orleans, and Green Bay have all won Super Bowls).

Ultimately, I don’t think the draft is going away. It’s too entrenched and people have convinced themselves they want it. But just think about how much more orderly — and fair — it would be if teams could bid on players rather than this messy, strange draft system. With minimum salaries for players based on experience (as we have now), revenue sharing, and a salary cap, I don’t know why this system wouldn’t be infinitely better than the one we have now. But it’s likely a pipe dream.

The most tepid defense of scouting you’ll ever read

The above is the title of my new piece, available over at the NY Times Fifth Down. Thanks to everyone over there for putting it up. Feel free to post comments here, where it is easier for me to respond.