Okay so let's talk about this Watchtower thing

>> 9.15.2011

Recently, Commenter Matt—longtime friend of this blog—challenged several of the principles this Watchtower thing is built on. Many of you weren't around for the Watchtower's genesis, and I've been doing these things for three years now. Now's a good time to air this stuff out, explain my methodology, and make sure I haven't made a wrong turn somewhere in my execution.

The original mission of the Watchtower was to find out what the new-look Lions were going to look like. To, like the forest ranger in the old Lassie episodes, use a combination of familiar landmarks and mathematics to pinpoint a direction and distance to the truth.

With so much of the new-look Lions roster being turned over from the year before, we didn't know what to expect when they took the field that first week. Since Linehan and Cunningham have long track records with distinct schematic styles--and, because I've always thought the schematic interplay between offense and defense is overlooked--I  decided to analyze historical matchups between the Lions' coordinators and their first opponents'.

I found their season averages of points scored and points allowed, saw how each team performed relative to their averages when facing the other coordinator, and concluded the following:

As all three metrics of output--per-play passing, per-play rushing, and points scored—are way above their averages for the season, I’m going to say that given superior talent and execution, and/or excellent pass protection, Scott Linehan’s balanced offense disproportionately gives Gregg Williams’ attacking 3-4 defense fits.

Given equal talent, Cunningham’s hyperagressive 4-3 is extremely effective against Payton’s pass-heavy offense, but only if that aggression leads to mistakes and turnovers—otherwise, the holes in the defense will be exploited. Effective quarterback play may neutralize the defensive advantage.

Therefore, the most probable outcome of this game is a shootout that the Lions lose.

I didn't project a score, and didn't project passing or rushing effectiveness. I just tried to retell the story of the upcoming game, as clearly as I could read it in the data. Over time, I've added more data, made more projections, and reviewed the articles and methodology for their accuracy and completeness. I emphasized method's qualities as predictive tool because it showed great promise as such--and, I came to discover, that's what people want: a prediction.

However, it is still a study of offense vs. defense, and analysis based purely on per-play effectiveness and scoring output has limitations. Sacks, turnovers, penalties, injuries, special teams play, weather conditions, crowd noise, and clock management all dramatically affect the relationship between "how good is the team doing" and "how many points the team has scored or allowed". Look no further than the Lions’ 2009 trip to Chicago for proof of that: the Lions outgained the Bears 440-292, but lost 24-48.

When reviewing historical games, I always check for these factors to see if they're a more likely cause for unexpected outcomes than "Ol' Gun's just got Coach So-and-so's number," and I not only note that but build it into the prediction. Frequently, Cunningham's defenses have held offenses to lower totals than you'd expect based on their relative talent levels, frequently because Gun's Ds are designed to get sacks and force mistakes.

Matt said:

I totally disagree that fumbles and fumble recoveries are "random." Unpredictable, sure, but not random.

Here's some reading material on that:

  • Football Outsiders Basics: "Fumble recovery is a major reason why the general public overestimates or underestimates certain teams. Fumbles are huge, turning-point plays that dramatically impact wins and losses in the past, while fumble recovery percentage says absolutely nothing about a team's chances of winning games in the future. With this in mind, Football Outsiders stats treat all fumbles as equal, penalizing them based on the likelihood of each type of fumble (run, pass, sack, etc.) being recovered by the defense. Other plays that qualify as 'non-predictive events' include blocked kicks and touchdowns during turnover returns. These plays are not 'lucky,' per se, but they have no value whatsoever for predicting future performance."
  • Advanced NFL Stats: “The ratio is not significant for wins, points scored, or points allowed while overall fumbles and fumble rates are significant. This result indicates recoveries are indeed random. It also suggests that fumbles, and not fumbles lost is a better stat for estimating a team's future likelihood of fumbing.”
  • CougarBoard.com: “Look at LSU. They were 70th and 102nd in 2005 and 2006, then suddenly finished 1st in back-to-back years in 2007 and 2008. Oklahoma went from 118th to 1st in the space of two years. BYU went from 112th to 2nd in two years and two years later were back at 95th. There is no pattern here. If you take a histogram of average fumble lost rankings by team, you get a tight, centralized cluster which is the hallmarks of a random distribution.”
  • As the New York Times explained, Jim Schwartz once told Bill Belichick the same thing: "“Fumbles are a random occurrence,” Schwartz said he told Belichick. “Being able to get interceptions or not throw interceptions has a high correlation with good teams. But over the course of a year, good teams don’t fumble any more or less than bad teams. Bill didn’t agree. He said, ‘No, good teams don’t fumble the ball.’ But actually, they fumble just as often as bad teams.”'

    Long story short, Matt is right that fumbles themselves aren't completely random. Daunte Culpepper fumbled the ball 80 times in 73 games as a Minnesota Viking; that's definitely his fault and not pure misfortune. But in general, using a team's current, or recent, fumble rate and projecting it forward to the next game simply won't be accurate.

    When it comes to recoveries, there's no doubt--once that ball hits the turf, it's statistically random. Just look at the ridiculous tip drills that occurred last Sunday. There were several near-heroic individual efforts on the part of the Buccaneers to force that fumble, keep it in play, and come up with it; that it ended up where Rob Sims could make a play on it was pure chance. The Bucs could have put their hands in their pockets and gotten the same result!

    The same thing goes for interception and fumble returns; whether they can be returned for scores depends more more on the spot on the field they occur than on team defensive ability. In my opinion, Matt is right that the Will Heller tip-pick was caused by a combination of Stafford throwing it too high and too outside, Heller batting it up instead of down, and Talib having excellent awareness and ball skills.

    However, Talib picking that pass off with nothing but green turf and white stripes between him and the end zone was purely a matter of location. Chris Houston's interception over Arrelious Benn's head was no less outstanding a play, but he was in no position to return it. If the same players had made each other's plays, the result would have been the same.

    The upshot of this: even though fumbles, fumble recoveries, special teams play, return touchdowns, and the like are either completely random or completely unpredictable, Matt’s right that without accurately accounting for them I’ll never be able to accurately predict the scores of NFL games. In fact, Advanced NFL Stats concluded that 42% of all NFL wins-and-loss records are determined by random variance, not relative performance. There’s a very hard cap on how close my projections can get . . . that’s why I remove, or adjust for, as much of these non-predictive factors as possible and then see how close I got.

    Maybe I’ve taken my eye off the ball a bit. Maybe I’ve been too focused on the score projections, and the accuracy thereof. I really only do it because, as Brian Cook from MGoBlog says, “the strictures and conventions of sportswriting compel me.” What do you folks think? Is The Watchtower a worthwhile read even if I don’t take a stab at the score? Should I be reviewing every Watchtower I write?

  • 25 comments:

    A Lion in ViQueen Territory,  September 15, 2011 at 3:51 PM  

    Don't stop the Watchtower. Ever. I love the O-Coord vs the D-Coord analysis for each team. It's fascinating. Your watchtower isn't a good predictor, per se, but it does show how we or they should play.

    Keep it up. Also, you owe us a watchtower this week.

    Ty September 15, 2011 at 3:53 PM  

    ALiVQT--

    Yup, you got it. Chiefs OC Bill Muir threw me a huge curveball: he was the OC in Tampa from 2002-2008 but never called the plays. He also had no prior West Coast background and doesn't run it now, so all of those seasons are basically garbage.

    Peace
    Ty

    randomguy313 September 15, 2011 at 4:20 PM  

    Ty, keep doing what you are doing. Your watchtower and old mother hubbard pieces offer a unique angle on the Lions.

    Anonymous,  September 15, 2011 at 4:29 PM  

    I agree. I like the watchtowers. I wouldn't put money down based upon it but it gives a good look at what we could be watching when the game rolls around. Keep it up.

    Anthony September 15, 2011 at 6:06 PM  

    Ty-

    Accurately predicting a final score will always be futile, BUT using statistics to analyze what is likely is awesome. Anybody that has an issue with the accuracy of the predictions should try making their own and publishing them to be critiqued. That's why the NFL is so engrossing, what should happen rarely does. The Watchtower gives guidelines and helps set expectations, but the reader should never have the expectation that a game can be accurately predicted. If there was a truly accurate method, whoever developed it sure wouldn't be posting it for free, they'd keep it to themselves and take Vegas by storm.

    -Lionsgab

    Diraghio September 15, 2011 at 6:42 PM  

    At the end of the day, no matter how well-educated, you're always going to be guessing. Still makes for a fun and informative read.

    As for reviewing the Watchtowers, maybe incorporate it into the 3 cups deep posts.

    I'll keep reading as long as you keep posting. Keep up the good work.

    Anonymous,  September 15, 2011 at 8:23 PM  

    I think you are concerned about pleasing everybody. You get praises article after article, week after week. What you do on here is absolutely amazing. I stop by here almost every single day hoping that a new article is up. Don't worry about making everybody happy because it is impossible. There will be another criticism next week.

    StreetWorm,  September 15, 2011 at 8:48 PM  

    I agree with what was mostly said above...it's a great idea that gives a very differing perspective from, well, really any other gameday analysis that comes to mind. It drills down past the players into systemic advantages that just don't get looked at in traditional analyses'.

    I can certainly see a drawback though. One, the majority of your datapoints just don't have enough data to be effective. And two, there is so much turnover year to year (and even during the year), that it's hard to determine what's systemic advantage, what's having better players, and what's just dumb luck.

    Michael Bodalski September 15, 2011 at 8:54 PM  

    I look forward to the Watchtowers every week. Short of adding each coaches astrological leanings into equation, I expect to keep doing so.

    Anonymous,  September 15, 2011 at 9:56 PM  

    please keep posting the watchtowers! i suspect there are a lot of stat-junkies like myself roaming al gore's internet...and the stats you come up with and the way you present them make for a must-read. i'm not gambling on your score predictions, either...but i would prefer that you continue to post those as well. just keep in mind that if you miss the score too many times, some of us just may ask for our money back. ;)

    Anonymous,  September 15, 2011 at 10:03 PM  

    Ive' got a motto about The Watchtower: I'm gonna make that whore my wife!

    alvin2112,  September 15, 2011 at 10:31 PM  

    I like the watchtower alot, hope you keep it and predict the score as well. Last week was a shot in the dark prediction and it was almost right on,lol.

    rames September 16, 2011 at 3:06 AM  

    You know. It makes sense to not include a score prediction. But there's just something so inviting about it. I get suckered into posting my own predictions often and even while I feel it's futile, I just can't resist.

    I love the history you proved and glimpse into schematic advantages - that's something not offered anywhere else, thus it's uniquely valuable.

    Clusterfox,  September 16, 2011 at 9:58 AM  

    Your Watchtowers are great. you can't please everyone. In the end it is still a game that needs to be played to see the outcome. More often than not, you give the common game watcher some more advanced details to watch for. I like it, and I believe its appreciated even if it not a perfect indicator of game results. Keep up the good work.

    Anonymous,  September 16, 2011 at 10:53 AM  

    Your analytical approach to the game of football (Watchtower, PFF grades, etc) is outstanding and is the one thing that truly sets your Lions blog apart from others. Keep up the great work!

    JP September 16, 2011 at 1:39 PM  

    Ty,
    I wouldn't change a thing! Your current format is great. The Watchtowers' analytical approach to that weeks matchup is the perfect compliment to Neil's, umm, schizophrenic(and yet all too true, and I mean that in the best way) take on the Lions.

    Their powers combined!

    P.S. I don't know where you get the time to juggle TLiW, B/R, and a family. However you do it, keep it up!

    Matt,  September 16, 2011 at 2:25 PM  

    ABSOLUTELY keep going with the Watchtowers! They are extremely interesting, a unique perspective, and a great read. I was really critiquing (not criticizing) the one short-coming. My suggestion was simply to stay focused on the OC vs. DC stuff, especially since that's the intended purpose, and worry about the score predictions less.

    That being said, the score projections are kind of necessary. As you and others point out, it's obligatory to "bottomline" your analysis. Perhaps a slight change in format: do the usual coordinator stuff (which is awesome), give us a score based purely on Watchtower analysis, then give us your personal prediction based on more abstract information. And, yes, Watchtower review is also obligatory. There's no point in doing pre-game analysis if you're not going to go back and see how you did.

    As for the put-up-or-shut-up crowd, I could throw some predictions out there, but they wouldn't be any better than what Ty puts up. I'm not the guy putting the blog or the "system" out there, but I don't think that precludes me from critiquing a system. I've also never played NFL QB, but feel comfortable calling Stafford's Pick-6 a bad throw (not "terrible," but "bad"). Never run for office, either, but don't get me started on politics. :-) If it gets me a pass, though, then I think the Lions will beat the Chiefs 34-10 because I think the Lions are a much better football team with much better players than the Chiefs this season.

    As for the statistical stuff, that's about what I expected. First, yeah, fumble recoveries are pretty random (I think we'd all agree to that). But, as Advanced NFL Stats points out, fumbles are not. It logically follows then that a team that fumbles more often would also be likely to have more fumbles lost and returned for TDs. The key word there is "likely." What the Football Outsiders stuff says is fumble recovery percentages are not predictive of future fumble recoveries or overall team performance. They can tell why a team won or lost a game in hindsight, but don't accurately predict whether or team will win or lose a future match-up. Agreed. But they also suggest that recoveries aren't random, "Football Outsiders stats treat all fumbles as equal, penalizing them based on the likelihood of each type of fumble (run, pass, sack, etc.) being recovered by the defense." The whole thing seems a lot like hockey goals. The Red Wings typically outscore their opponents because they typically have more shots-on-goal and scoring chances than their opponents, not necessarily because they make a higher percentage of their shots. Being a "good team" is about creating more opportunities, then cashing in on as many as possible.

    Let me try an example of extremes from last season: Denver was the worst defense in terms of points allowed per game (29.4) and Pittsburgh was the best (14.5). Pittsburgh forced 28 fumbles, recovering 14, returning 0. Denver forced 14, recovered 8, and returned 1. So, the "better" defense (and team) forced and recovered more fumbles, but at a lower recovery and return rate. Note: Pittsburgh also had 21 INTs to Denver's 10 and Pittsburgh's offense had fewer INTs and fewer lost fumbles than Denver's, so PIT's over give/take margin was much better than DEN's. I think we can all agree that the Steelers were a more successful team than the Broncos last season and that fumbles (on both sides of the ball) were relevant to that success.

    Matt,  September 16, 2011 at 2:25 PM  

    As for Schwartz v. Belichik, I find that very intriguing. I think the argument needs to be parsed out a little more. Schwartz says INTs, unlike fumbles, correlate highly with good/bad teams. I'd argue that what he's really talking about isn't INTs, but good or bad QB play (of which INTs are indicative). When Belichik says "good teams don't fumble," I hear "good teams don't turn the ball over" which would include lost fumbles, INTs, and 3-and-outs/stalled drives. All due respect to the Grandmaster, I'm going to go with the Hoodie on this one and say fumbles matter.

    I guess my overall point is that fumbles, recoveries, and returns aren't purely random but are fairly unpredictable on a game-by-game basis (I'd still say predicting PIT will force and recover more fumbles than DEN this season is a pretty good bet) and that there are certainly stats out there that are more strongly/directly correlated to overall team success (i.e. scoring). However, I don't think fumbles are so random as to be irrelevant. Trying to crunch solid numbers on them to predict a specific game would be futile, but that doesn't mean you can completely leave them (along with other things mentioned like special teams play, etc.) out of the "equation."

    Oh, and for the record, I don't use Ty's or anyone else's information to bet on sports (unless you consider having a fantasy football team betting on sports). This is all purely entertainment.

    Anthony September 16, 2011 at 3:27 PM  

    Matt-

    Just want to clarify my comments because they weren't meant to be an attack on you, more of a generalization of critics.

    Your critique was exactly that a critique. Not a critical assault or challenge, and I think it's good that there are readers that challenge writers with well thought out questions.

    I also assumed that Ty was questioning the usefulness of the Watchtowers based on more than just your questions. I was assuming he's gotten some of the mindless criticism that plagues many comment boards.

    My comments were directed at the more generalized critics, not your critique. There's a very big difference.

    As far as the gambling thing, it was more of a general statement that more often than not predicting something as unpredictable as football will never be perfect. And if somebody did have a perfect system, they sure wouldn't share it for free.

    Sorry if the tone came across the wrong way, it definitely wasn't meant to.

    Tiger Hebert September 16, 2011 at 4:19 PM  

    Don't stop writing it bro. The insight is always worth the read, regardless of a predicted score. When you back up your "argument" with supporting information it is worth reading. Unlike a lot of the substance-less windbags at places like BR.

    Anonymous,  September 16, 2011 at 5:34 PM  

    Honestly though, what's your record for predicting games based on these? I've been following them for about a year now and you DO seem to predict the winner more often than not, even if the score isn't always right...

    Ty September 16, 2011 at 8:12 PM  

    Thank you all for your awesome comments. Ironically, I haven't replied in-depth because I'm working on the Watchtower. Up tomorrow if it kills me.

    Peace
    Ty

    Matt,  September 16, 2011 at 9:37 PM  

    Anthony, thanks for the clarification. To clarify back, I wasn't trying to defend myself against you specifically, but I think the sentiments you pointed out are probably "out there." As you said, you were directing your comments at "critics in general." I had the same intention. Same goes for the gambling stuff. I didn't take it as an attack and wasn't trying to counter-attack. I just thought I'd put it out there. Don't want my opinions discounted 'cause someone (not you) thinks I'm just a disgruntled gambler who thinks Ty cost him a bunch of money. By the way, so as not to offend anyone else, I have absolutely no problem with people who DO engage in sports-betting or any other kind of gambling, assuming they "keep it under control."

    Ty, hopefully the comments on this post have made it clear that the Watchtowers are very welcome and worthwhile contributions to the InterWebs, no matter what anyone else has to say about them. Keep up the good work.

    bigwalt2990,  September 16, 2011 at 10:03 PM  

    "If the same players had made each other's plays, the result would have been the same."....um, no?

    Anonymous,  September 18, 2011 at 8:28 PM  

    Love the watchtower. And guessing the final score is always fun, even though difficult. Keep it up it all makes for a fun read.

    Post a Comment


      © Blogger template Simple n' Sweet by Ourblogtemplates.com 2009

    Find us on Google+

    Back to TOP