The Defensive Line and the Secondary, Part II

>> 6.14.2010

When I first delved into the connections between pass rush and pass defense, I came away conclusions fueled partly by data, and partly by ignorance.  They were:

  • We have found that there is no inverse correlation between that pass rush effectiveness and passing effectiveness; on plays where a sack does not occur, per-play pass yardage is unaffected.
  • We have found that there is no inverse correlation between interception rate and passing effectiveness; on plays where an interception does not occur, per-play pass yardage is unaffected.
  • These may have been exaggerations.  Commenters, emailers, and strangers on the street have since filled me in.  The correlation coefficients looked dismissible to my amateur eyes, but weren’t—not entirely, anyway.  There was a –.068 correlation coefficient between “dropbacks per sack” and “raw yards per attempt”.  Essentially zero, I thought, but it’s closer to –.1 than to zero—and that’s more than nothing.  Not much more, but more.

    The correlation coefficient between "attempts per interception" and "yards per attempt" was –.091.  That’s still only approaching .1; a weak correlation at best.  Still, my two statements above aren’t entirely accurate.  Over the past 20 years of NFL data, there has been a very mild inverse correlation between sacks, and pass effectiveness on non-sack downs (and the same goes for INTs).

    This all implies that the pass rush might indeed affect offense even when it doesn’t hit home—but I couldn’t prove it with the data I had.  I’d been working with the 50,000-foot-view: year-over-year leaguewide NFL trends; he low resolution of this data may have blurred the very effects I’m hoping to measure.

    As I said in my “parallel efforts” post last week, the Mathlete, of MGoBlog fame, did some of that further research.  As the Mathlete compared sacks, picks, and overall performance, he found:

    Not entirely surprisingly, the better a defense is at producing sacks and interceptions, the better it is on downs where neither occur. For every point per game that a defense generates due to sacks, the overall pass rush generates 1.2 ppg of additional value. Interceptions are also powerful, but not as much so. Each ppg of value a defense generates through interceptions is worth 0.9 ppg of additional value.

    This relies on “expected points,” here explained by Brian Burke of Advanced NFL Stats.  EP is a way to recognize the reality that four yards gained on 3rd-and-3 are not nearly as valuable as four yards gained on 3rd-and-8.  By finding the historical likelihood of scoring on a given down and distance from a given field position, Burke has created baseline EP values.  By finding the difference between the EP values from before a play to that play, you can gauge what that play is worth (in EP Added, or EPA).

    Using EP allowed the Mathlete to quantify exactly what sacks—and the associated pass rush on non-sack downs—are worth.

    From 2006-2009 in games between two D1 teams in competitive game situations (the “universe” for this and most of my analysis) the average defensive unit produced 2.3 ppg worth of sacks and 2.0 ppg worth of interceptions.

    So, an average defense produced 2.3 ppg worth of sacks, and gained 2.76 ppg worth of “footsteps effect” from the pass rush.  Likewise,  the average defense snared 2.0 ppg worth of interceptions, and 1.8 ppg worth of “dink-and-dunk effect”.  Mathlete concluded that all he found was the dog wagging the tail: sacks and picks are merely indicators of good defensive performance, not the other way around.

    However, this is all still limited to sacks, and attempting to derive the “cringe effect” on non-sack plays.  However, I’ve pulled down data from ProFootballFocus.com. They use film review to grade individual performances, as well as capture unofficial or untracked stats—like QB hits, and QB pressures.  Rather than attempt to measure the size of the tyrannosaur by the size of the ripples in the water cup, I’ll look at the actual data.

    The data I’m using is PFF’s player data for the 2009 season, aggregated into team numbers.  A caveat, these numbers do vary from the official NFL totals, especially with subjective stats like sacks, half-sacks, and tackles.  From what I’ve seen, the differences are small, but it’s worth getting out there.  Anyway, the first order of business: test my assumption that pressures correlate with sacks.

    image

    Result?  “Yes,” but not as strongly as I’d assumed.  The correlation coefficient was .584, quite strong.  But as an anonymous commenter noted:

    One of the best indicators of nature of the relationship is the effect size calculation. You can obtain an effect size by squaring your correlation coefficient. This will tell you the amount of variability the variables contribute to each other. There's a lot of noise in NFL data so an effect size will tell you how much unique data is explained by the correlation alone.

    Rule of thumb for effect sizes: small .2; medium .5; large .8

    The effect size here is displayed in that graph as R-squared: 0.3413.  It’s far from one-to-one, but it’s safe to say that teams that get lots of pressures tend to get lots of sacks, and vice-versa.  For what it’s worth, the best “finishers” are the Redskins, getting 43 sacks to 109 pressures.  For academics’ sake, the Falcons were the other way around: 160 pressures to only 31 sacks.  The Lions were . . . well, terrible: 23 sacks, and 101 pressures.

    Next up, let's take that “hit” and “pressure” data out for a spin:

    image

    This is the total yards per attempt allowed, regressed against the percentage of dropbacks where “pass rush” was generated.  Dropbacks are “attempts + sacks”, and I’m dividing that number by cumulative sacks, hits, and pressures.  The resulting correlation is weak, –0.152, and when squared to get the effect size, it’s a negligible .0232

    So, once again, we’re seeing no real connection between pressure and good pass defense.  You see the most interesting data point?  Yeah, that one—the one where they’re allowing three-quarters of a yard less per pass attempt than the 2nd-best team.  That’s the New York Jets; they allowed a miniscule 4.91 yards per pass attempt, while finishing 19th in PFF-scored sacks, and 9th in per-dropback “pass rush” rate.  So, the Jets had a good pass rush, but nothing that correlates to the incredible per-play passing effectiveness seen here.

    Secondly, my eyes were drawn to the third-rightmost data point, and how we’d have to scrape it off the Yards-per-attempt ceiling.  That would be the Cleveland Browns, whose 47 sacks, 42 QB hits, and 139 pressures generated pressure on 42.93% of dropbacks.  That pass rush rate is third-best in the NFL, behind only Dallas (46.91%) and Minnesota (44.77%).  Yet, Cleveland’s pass D was flat terrible: it allowed 7.44 yards per attempt, ahead of only Oakland, St. Louis, Miami, and, yes Detroit (7.80). 

    Things are not looking great for our thesis.  If the Lions improved their 2009 per-dropback pass rush rate to match Cleveland’s, from 27.85% to 42.93%, we’d certainly expect a better improvement in allowed Y/A than 7.80 to 7.44.

    Last time, passes defensed were to coverage what sacks were to pass rush; I presumed that in years where passes defensed were up, “coverage” was better.  In this case, though, I have Pro Football Focus’s “thrown at” and “reception allowed” figures.  Again, let’s dispense with trying to derive the whole picture from one rare subset of outcomes.  FYI: Total TAs don’t equal attempts, and total receptions allowed don’t equal total receptions.  I presume the discrepancies are from uncatchable passes, screen and swing passes that aren’t thrown “into coverage”.

    I’ll divide total per-team receptions allowed by total per-team TAs to get a per-team Percent Caught, as PFF does on a per-player basis.  This should be an actual measurement of the quality of pass coverage.  Let’s regress it against Y/A:

    image

    Bingo.  There’s as strong of a correlation between percent of catches allowed by coverage and yards-per-attempt, as there is between pressures and sacks.  Oh, and who’s that team with the astonishingly low percent-caught, and correlatedly low Y/A?  Yup, it’s the Jets.  The evidence is mounting: pass rush doesn’t equal pass defense, pass defense equals pass defense.

    But, that’s not the whole story, is it? After all, isn’t the magic of a 4-3 defensive line supposed to be that it can generate pressure without a compromising coverage?  Won’t there be great improvement in a 4-3 that generates pass rush from the line, without having to blitz linebackers and DBs?  Stay tuned for the next exciting adventure!


    10 comments:

    Anonymous,  June 14, 2010 at 8:51 PM  

    Hey Ty, I just wanted to ask if you minded if I reference your blog or post links to your blog on some Lions forums. Its really cool what you do here and I think most fans will appreciate your insight.

    Ty June 14, 2010 at 11:02 PM  

    Anon--

    Yeah, sure! In fact, please do. Thanks for the encouragement!

    Peace
    Ty

    Anonymous,  June 15, 2010 at 9:50 AM  

    This is fascinating. Thank you for the analysis.

    Ty June 15, 2010 at 10:22 AM  

    You're welcome! More to come . . . If it wouldn't bore everyone to tears, I could probably give up writing for a month, and emerge with incontrovertible mathematical proof that the Lions are going to be tantalizingly mediocre this year, which we already knew.

    Peace
    Ty

    TimT June 15, 2010 at 12:19 PM  

    Off of memory, the best run-defenses were (for the most part) 4-3, while the best pass-defenses were 3-4's.

    A 3-4 will have more open lanes for the RB. As they move players around to provide a pass-rush, they're providing less run-support.

    Anonymous,  June 15, 2010 at 2:24 PM  

    hmmmmmmm
    Next step might be the difference between accurate passes and bad passes or throw aways.

    If the ball makes it to the receiver the rush has no effect with what happens from there. In this case the effect of the rush was felt before the study.

    Anonymous,  June 15, 2010 at 4:03 PM  

    I think this is a better analysis than part 1, but you've still got to realize that the starting front 4 has at least two new members and our DBs are potentially worse than last year. There's no guarantee that our pass rush is going to be any better or that a drop in production from the secondry couldn't cancel out any improvement from the line.

    You can correlate from the last few years that the Lions defense will be bad though.

    Anonymous,  June 22, 2010 at 9:06 PM  

    One MAJOR FLAW is that you need to look at pressure in even more detail.

    #number of pass rushers vs
    #number of blockers

    Amount of time before pressure was applied.

    The more players you can drop back and still get pressure the better your pass defense is going to be.

    Obviously if two teams get the same % of pressures but it takes one team 6 guys on average to generate that pressure and another team 4 the. The team with 4 should seemingly have better pass defense if the CB's are equal.

    Ty June 24, 2010 at 4:06 PM  

    Anon 1--

    Don't forget, I'm limited by the data. I can't really tie these back to individual plays (don't have access to the database itself). I asked the PFF guys about a couple of things (like calculating NY/A on plays where there was a QB Hit or QB Pressure but no sack), and they couldn't help me.

    Peace
    Ty

    Ty June 24, 2010 at 5:00 PM  

    Anon 2--

    I don't disagree. If I thought an improved pass rush is going to make horrible corner play irrelevant, I don't think I'd be spending a gazillion hours doing this.

    Anon 3--

    The data I have access to is NOT that atomic . . . in fact, I don't think "seconds from snap to pressure" is a stat that anybody has, outside of a few NFL team HQs, and then only for their own stuff.

    I CAN isolate DL pass rush, though, so that's part III. With that will come some data from when a blitz came, and the blitz pickup freed a D-lineman, but that should be a small percentage of the total.

    Peace
    Ty

    Post a Comment


      © Blogger template Simple n' Sweet by Ourblogtemplates.com 2009

    Find us on Google+

    Back to TOP