Right before the NFL season began, we put out a tweet with a link to our “World’s No. 1 QB Rankings.”

Patrick Mahomes was No. 1. Josh Allen was No. 2.

And Derek Carr was No. 3.

This looks smarter now, thanks to Carr’s unusually great start.  We’re not here to take a victory lap though.

At the time we sent the tweet, the Twitterverse didn’t like it, and understandably so. We heard from a bunch of members of the “Delete Your Account” club.

One of the issues with the tweet is that it didn’t offer anything in the way of context or explanation as to what the “World’s No. 1 QB Rankings” was other than to say it was “Based on 3-year statistical evaluation of all aspects of QB performance.”

Another is that it didn’t offer any explanation as to why Carr ranked so high and how he could rate above Lamar Jackson and Dak Prescott, among others.

So let’s address both those things here.

What is the World’s No. 1 QB Rankings?

 A bunch of years ago, Bill James created something called The World’s No. 1 Starting Pitcher Rankings. Bill wanted a way to rank starting pitchers and thought it would be the kind of thing that would catch on with the statistical populace and the general fan population.

He calculated rankings with a multi-year system that utilized Game Score, a stat he devised to evaluate individual pitching performances, as its chief stat. The rankings ran on his website and The Bill James Handbook each year.

A few years ago, our content lead, Mark Simon, asked the R&D team if it could build something similar for World’s No. 1 QB. Our now director of football analytics, Alex Vigderman, obliged.

The World’s No. 1 Quarterback Ranking uses our player value stat Total Points to make a cumulative evaluation of game performance over a three-year period.

The key to our calculation is Points Above Average (PAA), the statistical underpinning of Total Points, which is centered at zero so that positive numbers are above average and negative numbers are below average.

Success over time is key to maintaining a good ranking on the World’s No. 1 QB list. Prolonged inactivity due to injury or benching results in a ranking drop. The most recent performances carry the most weight.

What’s up with Derek Carr ranking No. 3 overall?

Imagine making a Top 10 quarterbacks list in which the following three factors were the most important:

  • Performance (most recent = most important)
  • Track record (same)
  • Playing time

Most lists probably would have included the following eight names:

Patrick Mahomes, Josh Allen, Lamar Jackson, Dak Prescott, Jalen Hurts, Justin Herbert, Matthew Stafford, and Jared Goff

Track record rules out C.J. Stroud and Brock Purdy. Injuries take out Aaron Rodgers, Joe Burrow, and Kirk Cousins. You could make a case for Tua Tagovailoa and Trevor Lawrence, but we’ll leave them off for now.

Carr belongs in that Top 10 list too.

He ranked 9th in Total Points in 2021, 11th in 2022 (when he missed two games), and 7th in 2023. He shows up to work every week and puts in solid, albeit unspectacular efforts.

If we’d had Carr 10th, you wouldn’t have batted an eye. It wouldn’t be an outlandish statement.

If we’d had him 8th, you would have said eh, ok, and moved on.

If we’d had him 6th, you would have given us the raised eyebrow.

But 3rd … geez.

So let’s recollect about a prominent chunk of the 2023 season.

Through Week 6, Carr ranked 9th in the World’s No. 1 Quarterback Rankings. Carr was … just there. Every week. Respectable, not superstarish. The Saints were a non-factor.

We mentioned how Points Above Average was the key stat within the calculation.

Here are the Points Above Average Rankings and Totals from Week 7 on

  1. Josh Allen 66
  2. Patrick Mahomes 60
  3. Jordan Love 50
  4. Derek Carr 41
  5. Lamar Jackson 38
  6. Matthew Stafford 25
  7. Justin Herbert 22
  8. Jalen Hurts 22
  9. Justin Fields 21
  10. Joe Burrow 21
  11. Dak Prescott 21
  12. Jared Goff 18

Carr wasn’t just better than most quarterbacks in the final 12 weeks of the season. He was a lot better. A lot better than Stafford, Herbert, Hurts, Prescott, and Goff.

What exactly does Carr do to merit this? One thing that’s important – his on-target percentage was a lot higher than his expected on-target percentage (based on factors such as length of pass and pressure).

That’s wordy, so let’s simplify. Did you see the two on-the-money long throws he made on Sunday? He put those throws in just the right spot.

He’s good at that.

The other aspect of this is that the timing of his ‘bad’ was the best possible time per the stats. Half of his interceptions and sacks were on 3rd and 4th down with at least 6 yards to go. So the penalty for these wasn’t steep. On early downs, Carr did his job (heck, he ranked 2nd in passing EPA on early downs for the season).

This was all there for everyone to see. But some things you just don’t notice. That’s what these stats are for.

The thing you might notice on that PAA list is that Carr and Jackson were nearly equal. And entering Week 7, Jackson was 6th, three spots ahead of Carr in the World’s No.1 QB Rankings.

Why was a 3-PAA edge enough for Carr to jump Jackson?

This goes to how the World’s No. 1 QB system functions. The most recent body of work is most important.

Obviously Jackson went on to win the MVP last year, so he had some good games on his ledger. In the five games leading up to Week 7, he had three PAA marks that were higher than any of Carr’s from that same time period,

In the two playoff games at the end of his year, though, Jackson totaled 0.8 PAA and -0.7 PAA.

That didn’t sit well with World’s No. 1 QB. For better or worse, the system does not consider strength of opponent nor that you are playing playoff games. So Jackson dropped. Is that fair? Maybe. Maybe not.

That gets to a larger point. It’s very difficult to build a QB ranking system, especially one that is going to consider everything and please everyone. In fact, if there was such a system, it wouldn’t be worth making, because it wouldn’t tell you anything you didn’t already know.

No one at SIS agrees entirely and blindly with what our metrics say. You can rate No. 3 in a statistical system  and not be the No. 3 quarterback by one’s opinion.

But we feel strongly that our process is thorough and well-thought-out, and that there are important phenomena being captured that we wouldn’t otherwise note.

It’s clear to us that Derek Carr is more consistently productive than most anyone gives him credit for, and not just because an offensive scheme is propping him up. And with a hot start to 2024, a scheme improvement might have unlocked a new level of appreciation for his ability.