Thoughts About Drafting Quarterbacks

So let’s say you’re [Chicago general manager] Pace, and you’ve determined that you really want Trubisky. You call the 49ers and trying to work out fair compensation if the Browns do not pick him at one. You think Trubisky’s going to be the long-term Bears quarterback, starting in 2018 or later…. The market for quarterbacks is always weird…. For quarterbacks, NFL history says you pay Four Seasons prices. That’s why I can’t fault Pace for what he did. He wasn’t willing to risk losing the guy he loved.” — Peter King, The MMQB

I like my odds

The biggest story from the 2017 NFL draft was the surprise move by the Chicago Bears to pay a hefty price to the San Francisco 49ers — the 3rd overall pick plus the 67th and 111th picks, plus a 2018 3rd round pick — in order to move up a single spot in the draft, where they selected North Carolina quarterback Mitchell Trubisky. It was a stunning trade, in no small part because Chicago, having just spent significant money to acquire free agent QB Mike Glennon, was not an obvious candidate to draft a quarterback let alone pay a steep price to do so. But the Trubisky trade wasn’t the only trade involving a first round QB, as both the Chiefs and Texans also paid premiums to get their (hopeful) quarterbacks of the future, namely Patrick Mahomes and Deshaun Watson, respectively. Per both the traditional Jimmy Johnson trade chart as well as Chase Stuart’s updated version, both trades were expensive for the teams trading up: the Chiefs paid 120 cents on the dollar under the Jimmy Johnson chart and 170 cents on the dollar under Chase’s version, while the Texans paid approximately 125 cents on the dollar under the Jimmy Johnson chart and 154 cents on the dollar under Chase’s chart.

Was this rational? Well, a common response — and it’s the response baked into Peter King’s excellent MMQB piece from today — is that if the QB is a success and becomes a star and leads the team to Super Bowls, then the price tags for these teams will be cheap. The counter is essentially an extrapolation from the trade value charts: while that may be true ex post, on an ex ante basis the price tags are hefty — and later success won’t change that — particularly given that we all know many highly drafted QBs nonetheless fail.

I am not particularly interested in proving one perspective right or wrong (and I have not watched enough of Trubisky to really evaluate him), but I am quite interested in how to think about this question, as it’s one of the most vexing questions in all of sports and it’s a discussion that seems to frequently get off track. Further, it’s clear NFL teams approach QB draft picks entirely different than they do every other person; as Bill Barnwell wrote about and then separately said to me, “Teams are working with entirely different trade parameters for QBs than they are for players at most other positions.” To begin the process of answering the question, I wanted to lay out a handful of statements that I think (believe?) to each be true, although some point in different directions.

  • The most important thing is to understand the concept of “base rates,” or Bayesian probability. The basic idea — which can quickly get complicated — is that if you think there is a 95% chance of something happening (say, of Mitch Trubisky being the next Peyton Manning), the probability of that is not 95%, it is instead a blended probability that includes your estimated probability and the base rate, or historical success rate, for similar selections. Around 50% of 1st round QBs fail, and that number goes up for every round. (UDFAs actually produce more starters than 6th and 7th rounders, but that’s in part because there are so many more QBs that enter the league as UDFA than late rounders) As Dan Kahneman explained in Thinking, Fast and Slow (which every draftnik should read):

    [Bayesian reasoning] is named after an English minister of the eighteenth century, the Reverand Thomas Bayes, who is credited with the firsy major contribution to a large problem: the logic of how people should change their mind in light of evidence…. There are two ideas to keep in mind about Bayesian reasoning and how we tend to mess it up. The first is that base rates matter, even in the presence of evidence about the case at hand. This is often not intuitively obvious. The second is that intuitive impressions of the diagnosticity of evidence [i.e., the accuracy of your evaluation of how likely a QB prospect is to succeed] are often exaggerated…. The essential keys to disciplined Bayesian reasoning can be simply summarized:

    (1) Anchor your judgment of the probability of an outcome on a plausible base rate.

    (2) Question the diagnosticity of your evidence.

  • When I looked at the evidence most recently, the best data I could come up with was that QBs selected in the first round “succeeded” (fairly loosely defined; not limited to All-Pros and Super Bowl champs) around 50% of the time. The odds were slightly higher for top 10 picks and then slid down as the first round continued.
  • If you haven’t guessed, the failure to account for base rates is the flaw in King’s argument about Chicago (at least as presented; whether it was in fact a bad deal is something we’ll get to): Even if Pace is “sure” Trubisky was his guy, that doesn’t imply that no price is too steep as the base rates will anchor that probability down to something closer to 60-65%, even assuming a very high degree of confidence by the Bears. Ignoring base rates is one of the largest draft fallacies, and while it afflicts decisionmakers on both sides of the ledger it is most commonly an issue for those who are over optimistic about specific prospects. So trading up for a QB can be very risky.
  • However, having a good starting QB is extremely valuable. It is essentially impossible to win in the NFL with a bad QB. It is difficult, though not impossible, to win without a serviceable or replacement-level starter QB. It is much, much easier to win with a top tier QB. The position is essentially the most valuable one in all of sports.
  • Furthermore, a good QB on his rookie deal is maybe the most valuable asset in all of sports. Hitting on a rookie QB presents enormous organizational-level rewards: not only do you find your franchise savior, the CBA and rookie wage scale — plus the fifth year option for first rounders — results in massive savings for the franchise, money that can be spent on other assets. Barnwell: “No player in the NFL is creating more surplus value for his team than Dak Prescott, who appears to be an above-average quarterback in line to make less than $2 million combined over the next three years. Not far behind him is Carson Wentz, who should still be a massive bargain despite being guaranteed $21.8 million over the next three seasons (with a fifth-year option to come). When you consider that the free market guaranteed Brock Osweiler two years at an average of $14 million per year and Mike Glennon $18.5 million for one season, even the $7.3 million Wentz is going to earn is a relative pittance.”
  • Finding a serviceable QB is a real challenge, one exacerbated by the concept of a draft itself. When combined with a scarcity of good, starter-level QBs in the NFL more generally, the draft hands over to chance whether or not a team will ever hit on a rookie QB: A team needs a top pick in a year when there are such QBs available, and those sorts of QBs only come around every few years. In other years the team either must weigh drafting a QB too high or taking other needs (and teams drafting near the top typically have many). Skill and organizational stability also play roles, but there is a lot that simply comes down to chance — not to mention a negative cycle where downtrodden teams reach for first round QBs and thus propagate the cycle of failure->fire coaching staff/GM->new regime reaches to pick new “savior” QB of the future->failure->repeat, while a few teams go from one top-flight QB to the next, like the Packers from Favre to Rodgers or the Colts from Manning to Luck. Exacerbating this cycle is that elite QBs almost never come onto the free agent market.
  • Nonetheless, it is at least theoretically possible to compare the odds and cost of trading up and paying a premium for a first round QB — and thus taking a 50-70% chance on him becoming an effective starter — versus recreating that 50-70% probability by accumulating QBs in later rounds, i.e., the 2nd through 4th round. (Tom Brady aside, rounds 4-7 rarely produce quality NFL starters, and while there are a fair number of productive UDFAs, that is a much larger pool.) The analogy here is that most NFL draft picks are 2- or 3-star high school recruits, but that’s a quirk of the numbers because there are many more 2- and 3-star recruits by volume than there are 4- and 5-star recruits; a given 4- or 5-star recruit is significantly more likely to be drafted than a 3-star recruit.

Based on the above, below are some tentative conclusions:

  1. Good QBs, and rookie QBs in particular, are extremely valuable, and while the market price to acquire them is high it’s at least arguable that the potential payoffs justify even very steep price tags, even if the binary probability of success versus failure remains near 50% (or worse for later rounds). In other words, it still might be a coin flip whether Trubisky, Mahomes or Watson succeed, but if they do they would produce an incredible amount of surplus value to the entire organization in terms of on-field production as well as relative cost savings versus free agents and stability (i.e., job security).
  2. Nonetheless it is still possible to “overpay” (typically in terms of trade value) for rookie QBs, an issue that is most likely to arise if a team ignores the impact of base rates, i.e., the cost of a trade looks much more attractive if you think the likelihood of success of a potential QB is 85-95%, whereas even if that is your organization’s internal judgment — and I’d question you if it was — the true probability is somewhere closer to 60-70% given the impact of base rates. The same analysis applies in later rounds as well, applied to lower base rates of success (often dramatically lower), albeit compared to lower costs.
  3. It may be possible to more cost-effectively replicate the odds of hitting on a first rounder by taking multiple shots at QBs in the 2nd through 4th round range, until the organization hits on a QB. Seattle effectively used a version of this in hitting on Russell Wilson.
  4. Nonetheless, the most high percentage pool for identifying future star NFL QBs remains the first round, and that’s even after factoring in “first round” QBs who had no business being picked in the first round (EJ Manuel, Jake Locker, etc.).
  5. Finally, none of the above matters if your organization and coaching staff lacks the structure, knowledge and expertise to develop and coach a young quarterback. Without that you are doomed.