The Detroit Lions, the NFL, and Luck

>> 11.30.2010

Two weeks ago, Michael David Smith of the Wall Street Journal’s online edition wrote that the Detroit Lions may be the unluckiest team in NFL history.  Despite, at the time, outscoring their opponents, the Lions had won only 2 of 9 games.  Certainly, Lions fans expected better—and hoped for much better.  Infuriatingly, the Lions seem much improved, but there’s been no change in the bottom line.  However, it’s hard not to consider Bill Parcells’ famous line, “You are what your record says you are.”  Many fans, bloggers, and media pros subscribe to this idea: no matter how much more competitive the Lions look, they are not actually better until they have more Ws next to their name.

So, what do we make of this?  Do we ignore what our eyes tell us?  Do we disregard increased production on both sides of the ball as window treatments on the Titanic?  Or, do we foolishly embrace false “progress” because we’re so desperate to believe?  How much of the Lions’ 2-9 record can be blamed on happenstance, and how much of it is just the Lions’ lack of ability?  Fortunately, Brian Burke of Advanced NFL Stats recently wrote an article exploring exactly how random win-loss records are in the NFL.

Imagine flipping a perfectly fair coin 10 times. It would actually be uncommon for the coin to come out 5 heads and 5 tails. (In fact, it would only happen 24% of the time). But if you flipped the coin an infinite number of times, the rate of heads would be certain to approach 50%. The difference between what we actually observe over the short-run and what we would observe over an infinite number of trials is known as sample error. No matter how many times you actually flip the coin, it’s only a sample of the infinitely possible times the coin could be flipped.

As a prime example, the NFL's short 16-game regular season schedule produces a great deal of sample error. To figure out how much randomness is involved in any one season, we can calculate the variance in team winning percentage that we would expect from a random binomial process, like coin flips. Then we can calculate the variance from the team records we actually observe. The difference is the variance due to true team ability.

I strongly, strongly encourage you to read “The Randomness of Win-Loss Records” at Advanced NFL Stats in its entirety.  Go ahead, I’ll wait.

Okay, back?  Great.  Lost?  Don’t worry: I’ve got you covered with some bullet points:

  • 42% of an NFL team’s regular season record can be accounted for by randomness, otherwise known as sample error.
  • The correlation coefficient (r) between observed team records and a team’s true ability the square root of 0.58, which is 0.75.
  • After a full season of 16 games, your best guess of a team's true team strength should regress its actual record one quarter of the way back to the league-wide mean of .500.
  • The theoretical maximum accuracy of any predictive model is about .75. (from the comments, and Burke’s earlier work about luck & NFL outcomes).

If 42% of the Lions’ 2-9 record can be accounted for by randomness, that’s 4.62 games’ worth out of the eleven.  Assuming that the Lions have had nothing but bad luck to this point—they’re at the very nadir of randomness—then we flip it to nothing but good luck, we can see the theoretical maximum given this talent.  So, if Lions had gotten all the bounces: no Stafford injury, no Megatron Referee Fail, no Wendling/McCann freak TD return, no Alphonso Smith Disasters, Drew Stanton competes that pass, Shaun Hill doesn’t airmail that two-pointer (neither of which would happen anyway because Stafford would’ve been healthy, remember?), a few fewer specious penalties for the Lions, a few more for the opponents, recover a few more of the forced fumbles, catch a couple of dropped INTs . . . the Lions could be as good as 6-5 right now.

Before you freak out: that assumes both a 16-game season, and that the Lions are currently having the rottenest luck possible.  An 11-game sample isn’t the same as a 16-game sample; there may yet be some regression to the mean—that is, if the Lions really aren’t what their record says they are, their luck will turn before we get to the end of the season.  Well, either that, or next season will be a 16-game dip in the strawberry river:

Let’s assume for a second that there’s no sudden switch in the Lions’ fortunes, and they don’t sweep the NFC North at home during these next five games.  Let’s also assume they maintain their current pace: a winning percentage of .182.  Applied to 16 games, that’s 2.912 wins.  What’s the “best guess at their true strength,” if we regress them one-quarter of the way back to the mean?  If I understand this correctly, the difference between .500 and .182 is .318—and a quarter of that is .0795.  So, the Lions’ “true strength” should be a winning percentage of .262: just over four wins.

Again: this assumes the Lions only win one more game.  If the Lions finish 3-13, we’ll have no business saying “well this was really a 7-win team that got screwed.”  Sure, if everything had broken the Lions’ way, and they’d been the beneficiary of some truly rare luck, then maybe they’d have won six or seven games—but as they are, busted-up Stafford and all, if the Lions only win one more game, they really are only a 3-to-4 win team.

So, again, perspective: this is applying Brian Burke’s analysis of win/loss randomness in the NFL to the Detroit Lions’ current record.  All it can do is tell us, at the end of the season, what role “the Football Gods” have played in making the Lions’ record what it is—it is a redictive system, giving us a way of understanding what's already happened.  It can’t tell us which games were the result of randomness, if “the randomness” has already happened, or if the Lions are “due” for a hot streak.  It can’t tell us what we really want to know: how many games the Lions will win going forward. 

Let’s attack this from the other direction: with a predictive model, one that can actually assess teams' relative strengths and project a winner.  I’m choosing the Simple Ranking System, as published by Doug at Pro Football Reference.

Yes, this is required reading too.  Yes, I’ll wait.

Fortunately, it is as simple as the name implies, so it only requires one bullet:

  • Every team's rating is their average point margin, adjusted up or down depending on the strength of their opponents.

Okay, so average point differential, adjusted by strength of schedule, which adjusts the rankings, which adjusts the strengh of schedule, which adjusts the rankings, which adjusts the strength of schedule, over and over and over until the numbers stop changing.  Very simple indeed, yes—but as Doug says, “As it turns out, this is a pretty good predictive system.”

Chart?

Chart:

Team W L T W-L% PtDif SoS SRS
Green Bay Packers 7 4 0 0.636 103 1.2 10.6
New England Patriots 9 2 0 0.818 68 2.2 8.4
Pittsburgh Steelers 8 3 0 0.727 73 1.5 8.1
New York Jets 9 2 0 0.818 77 1 8
Atlanta Falcons 9 2 0 0.818 67 -0.1 6
Philadelphia Eagles 7 4 0 0.636 53 1.1 5.9
Baltimore Ravens 8 3 0 0.727 62 0.3 5.9
San Diego Chargers 6 5 0 0.545 85 -2.1 5.7
Tennessee Titans 5 6 0 0.455 39 0.5 4
Chicago Bears 8 3 0 0.727 50 -1.1 3.5
Indianapolis Colts 6 5 0 0.545 30 0.5 3.3
New York Giants 7 4 0 0.636 37 -1.4 1.9
Miami Dolphins 6 5 0 0.545 -20 3.5 1.7
Kansas City Chiefs 7 4 0 0.636 54 -3.3 1.6
New Orleans Saints 8 3 0 0.727 68 -4.5 1.6
Cleveland Browns 4 7 0 0.364 -13 1.4 0.2
Detroit Lions 2 9 0 0.182 -24 2.1 -0.1
Houston Texans 5 6 0 0.455 -23 1.5 -0.6
Oakland Raiders 5 6 0 0.455 -1 -1.9 -2
Minnesota Vikings 4 7 0 0.364 -50 2.5 -2.1
Buffalo Bills 2 9 0 0.182 -66 3.7 -2.3
Washington Redskins 5 6 0 0.455 -47 2 -2.3
Dallas Cowboys 3 8 0 0.273 -45 1.3 -2.8
Tampa Bay Buccaneers 7 4 0 0.636 -4 -3.1 -3.5
Cincinnati Bengals 2 9 0 0.182 -63 2 -3.7
Jacksonville Jaguars 6 5 0 0.545 -54 0.9 -4
St. Louis Rams 5 6 0 0.455 -18 -4.1 -5.8
Denver Broncos 3 8 0 0.273 -73 -0.1 -6.7
San Francisco 49ers 3 7 0 0.3 -59 -2.5 -8.4
Seattle Seahawks 5 6 0 0.455 -66 -3 -9
Arizona Cardinals 3 7 0 0.3 -104 -1.6 -12
Carolina Panthers 1 10 0 0.091 -136 -0.9 -13.3

Guess how this chart is sorted?  By SRS rank.  You can see the Packers, Patriots, Steelers, and Jets up there at the top, and Seahawks, Cardinals, and Panthers scraping the bottom of the barrel.  But wait, that team in bold, the one that’s darn near in the center?  That’s the Lions, ranked 18th overall.  When we take into account who they’ve played—per SRS, the Lions have played the 6th-hardest schedule in the NFL to this point—and how their offense and defense has performed, the Lions are the 18th-strongest team in the NFL.

This isn’t “with Stafford,” “with that Megatron touchdown,” “with that Drew Stanton pass,” or with anything imaginary added or subtracted.  Quite literally, it’s the scoreboard of every Lions game so far this year; it’s simply been adjusted by the scoreboards of everyone they’ve played.

Ah, but how accurate is this method?  It’s a predictive model, but how predictive is it?  Clearly, if it says the 2-9 Lions are near the middle of the pack in relative strength, it can’t be good at predicting who’ll win and who’ll lose, right?  Well, I regressed the SRS rankings against win percentage, and this is what I got:


image

Check out the correlation factor there: .7449205, or if you round up .001, .745.  What was the theoretical maximum for a predictive model again?  Well, if Brian Burke is right, it’s approximately .75.  That means that given the inherent randomness in NFL outcomes, the Simple Ranking System is as good as it gets when it comes to assessing relative strength of NFL teams, and thereby predicting future NFL outcomes.  Again, according to this system, the Lions are the 18th-best team in the land.  Further, if I’m not mistaken, they’re the biggest outlier on the chart: they’re the lowest, rightest dot (-0.1 SRS, .186 W-L).  Nobody’s getting screwed harder, or helped out more, by Lady Luck than the Lions.  Just trace the Y axis up to the line of best fit (the diagonal one), and you’ll know what the Lions’ win percentage ought to be: .500.  That’s right, SRS expects the Lions to have 5 wins by now.

So what does this all mean?  It means that if the Lions keep playing like they’ve been playing, they’re either going to pick up multiple wins in these last five games—or next season, they’ll be tubing down the strawberry river of regression to the mean.



10 comments:

Jim November 30, 2010 at 12:56 PM  

Ty,

I agree that the Lions are better than their record in a figurative sense. But I think the bulk of the backlash here is that the Lions have been better than their record for a long, long time.

When was the last time the Lions overachieved? The Erik Kramer/Mike Utley year comes to mind? I think the Leos are on the right track, but coming into this year their depth was an issue and instead of having an average injury year, they've had a terrible one.

A Lion in ViQueen Territory,  November 30, 2010 at 1:09 PM  

I love you when you go all Mr Statistician on us.

Old Man Winter,  November 30, 2010 at 2:09 PM  

Let me be the first to applaud you on earning your MWTNY (Master of Wait 'Til Next Year)!

Ty November 30, 2010 at 2:33 PM  

Jim--

Great question. I decided to go right to the source: PFR. In the last 40 years, the Lions have outperformed their SRS expectations by more than one whole game only four times: 2007, with the 6-2 start we all couldn't believe, 1993, the year Barry got hurt, 1991--the year you mentioned--and 1977.

In those same 40 years, the Lions have underperfomed SRS win expectations by more than one whole win twelve times: 1971, 1973, 1976, 1979, 1981, 1986, 1992, 1996, 2001, 2002, 2006, 2008.

Shoot me.

Peace
Ty

For reference:

http://www.pro-football-reference.com/teams/det/2007.htm

http://www.pro-football-reference.com/teams/det/1993.htm

http://www.pro-football-reference.com/teams/det/1991.htm

http://www.pro-football-reference.com/teams/det/1977.htm

Ty November 30, 2010 at 2:34 PM  

ALiVQT--

Thanks! It's definitely foreign territory for me, but as I say I just write what I want to read. Someone had to actually figure this out . . . might as well be me!

OMW--

You kill me, man. Never change.

Peace
Ty

Matt,  December 1, 2010 at 12:22 AM  

Jim, gotta' disagree with you on two points.

First, "the Lions have been better than their record for a long, long time." That was an 0-16 in 2008, in the rankings and on the field. I personally don't think, before this season, that the Lions have been better than their record since, well, probably since Barry. I think you'll be hard pressed to find evidence supporting the contrary. This franchise has been absolutely terrible for a solid decade and the record absolutely reflects it.

Second, "instead of having an average injury year, they've had a terrible one." How have they had a terrible injury year? Sure, Stafford went down, but he did last season, too (along with Megatron for a few games, Pettigrew, KSmith, etc.). The year before that, Kitna went down (along with Kevin Jones - shocker - and Roy Williams). That looks like some pretty bad luck with injuries overall, but my point is that this season is no worse than any other. The Stafford injury certainly skews the whole perspective and, if the Shaun Hill rumors are true, that skews it even further (I would have to concede that losing your top two QBs is pretty terrible). However, Oakland has been cycling through crappy QBs all season and have more than twice the wins that the Lions do. Remember, also, that the 2008 Patriots lost Tom Brady, arguably the greatest QB of all-time, in Week 1 and still went 11-5 because they were a good team outside of Brady. The Lions are not a particularly good team outside of Stafford and their sans-Stafford record reflects that. I'm not claiming here that Stafford's injury sunk the team, I'm claiming that the 2010 Lions just aren't that good of a team even with Stafford and that injuries haven't effected the bottomline all that much. For instance, as per the article linked below, the Lions are actually having an average injury year, not a terrible one. They've got 5 players on IR, the NFL average (I know, not really a fair metric, but it's what I've got). Who leads the NFL with 10 players on IR? The almost-surely-play-off-bound Green Bay Packers. They are truly having a "terrible" injury season (#1 RB & #1 TE on IR; #2 CB cut because he couldn't come back from PUP), but they're 7-4. Really, outside of Stafford and Schwartz stupidly playing Best through his turf toeS (cap on purpose), have any of the Lions injuries really affected the W-L column? Having, say, Levy or Avril on the field is definitely better than having them in civies, but they aren't "game-changers." With no injuries (which is, of course, impossible), the Lions might get a little closer to the record Ty suggests with the statistics, but they still aren't a play-off contender. As it stands, they've had an average injury season and their record reflects what they are from the top of the roster to the bottom. . .just not that good. . .yet.

Matt,  December 1, 2010 at 12:26 AM  

Oops, forgot the link:

http://www.associatedcontent.com/article/6004876/2010_nfl_injured_reserve_report.html?cat=14

Rob, Manchester,  December 1, 2010 at 8:33 AM  

Really hope we are due a swim in the strawberry river, the problem is, though theorectically after an infinite number of games this will all even out, we will not be alive to witness that entire time period...

So we are stuck with the brutal statistic that, even if you flip a coin 1000 times and heads comes up every time, when you flip the coin the 1001 time there is still only a 50-50 chance of getting tails...

Thus, despite this brutal season, there is no reason to believe that our luck will turn (in the non-infinite future).

On the plus side, i'd bite someones hand off right now for just an even rub of the green.

Jim December 1, 2010 at 11:44 AM  

Matt,

I'll say this, Ty did a lot of my dirty work for me. My thinking while poorly put together could more accurately be stated as: The problem is primarily that the Lions rarely exceed their expectations and more often fall below them. Which is supported by Ty's SRS stats.

We all knew the 2007 season beater was fake, it was like JLS MSU football, 4-0 in September, 1-7 the rest of the year. So now we're back to 1993 as the last good year which is a time we can all agree the Lions were "good". According to those stats anyway.

I don't have time to do the homework on the SRS stats for teams like the Vikes, Steelers, Eagles. Teams I think we could mostly agree are traditionally "good" teams. How often do they exceed expectations vs. meet or fall below?

At any rate, my point is that people I think anyway, get upset because the Lions are almost never better than they're supposed to be and according to Ty's stats anyway are three times likely to be worse.

As far as point two goes, it's mostly the absence of Stafford that has skewed my opinion of this. While the Lions may have had an average number of injuries, we cannot pretend that having a Matt Stafford who in his eleven quarters this year has thrown 6 TD's and a pick. Projected out to a "game year" that's 35 TD's and 6 picks. While those are unrealistic numbers to be sure, it is the pace he's been on.

My point in short, is that QB is more important than the other positions on offense. You know all too well how badly those Detroit QBs need to be healthy. As far as the Raiders go, when you have Jason Campbell as your starter on purpose, you're not planning on the QB being that important anyway.

As far as the other injuries incidental players, I'll concede they're about average. Although with apologies to Levy, I think they play better when he's in the game. But he's not to be feared.

Matt,  December 2, 2010 at 6:05 PM  

Jim:

Ok, yeah, I think I mistook your first point a little bit. I agree that the Lions have consistently underperformed their "talent." The problem is that their talent has only been worth, say, a 5-11 record and so when they go 2-14 instead, well, it's kinda' much ado about nothing.

As for injuries, yeah, like we've both said, the fact that it's been Stafford has skewed things. I am now going to make a logically incongruent statement: I think Matt Stafford is WAY better than Shaun Hill, but I don't think Shaun Hill is that much worse than Matt Stafford. Hill has filled in very admirably, as almost any analyst would agree. But Stafford has that intangible "it" factor. Something, maybe everything, is different when he is the QB. Anyone would have to agree that, had Stafford been healthy all 60 minutes of all 11 games so far, the Lions would have stolen at least a couple of those games they came so close in (CHI, PHI, GB, NYG, NYJ, BUF). What actually happened, though, was that Shaun Hill could never quite get it done. But then I have to go back to that Patriots season. Again, they lost a QB who is, at this point, two or three levels above Stafford and put in a guy who, at this point, is, at best, one level above Hill. And those Patriots went 11-5, again, because the rest of the team was good. The rest of the Lions' team is still pretty bad, so you can't blame too much on the Stafford injury. If the team, as a whole, were actually any good, they could have weathered the Stafford injury with Hill at QB. They're not, so they didn't. They are who we thought they were. End of story.

And that leads me to the Raiders. Certainly I take your point on Campbell. On the other hand, with the limited body of evidence we have on Stafford, can you really say he's any better than Campbell at this point? And is Shaun Hill worse than Gradkowski and/or Campbell? It leads me back to the team concept. If Oakland can win 3 more games than the Lions (3-0 in their division, btw, compared to the Lions' 0-3) with those "bums" at QB, then it's tough to argue that they aren't a better team overall (despite not having a Suh or a Megatron or whoever). Although I guess Ty's article might explain it a little bit.

The bottomline is that, injuries aside, this team still needs a lot of work. As I've been harping on since this blog started, the OL is about 10 years overdue for an overhaul aka we need to overhaul the overhaul that started 10 years ago. Maybe if this OL could open holes for Best like Oakland's does for McFadden, he would be more productive and not so "turf toe-y." Maybe if they protected better, Stafford WOULD still be the QB - although, the pass protection has been pretty good and my conclusion, at this point, is that Stafford is dangerously close to being "injury-prone." The DL seems to be on track, but LB is a mess and DB is a problem (although I'm not sure exactly WHO the problem is). The "skill positions" are all A-1, in my opinion, with Stafford/Hill, Best/Smith/Morris, Johnson/Burleson/anyone except BJ/etc, Pettigrew/Scheffler, but if you can't open holes and keep the QB upright, none of that matters. Until the OL is fixed and the defense takes another step forwad, the Lions will continue to underperform whatever statistical expectations one can come up with.

Post a Comment


  © Blogger template Simple n' Sweet by Ourblogtemplates.com 2009

Find us on Google+

Back to TOP