The guys at Football Outsiders are geniuses. They crunch football numbers to provide a fabulous layer of depth and brilliance to a sport that is ripe for analytical progress. I am their faithful Charlie, and they are my map to candy mountain. At least, that was my sappy romantic view of things until I got my hands on the Football Outsiders Almanac and found that they had picked the Broncos to finish 11-2 and be the 39th best team in the country.
What gives? I went to FO writer Brian Fremeau to get some answers. Here are those answers.
(Note: The following interview is based on the Boise State content from Football Outsiders Almanac, available for purchase at the FO Store. I highly recommend that you but it. The statistics used by FO are explained in more detail at the site's glossary page.)
Kevan Lee: First off, I really admire the work that you and Football Outsiders have done to bring advanced stats to college football. You guys are heroes. Don't let anyone tell you otherwise.
Brian Fremeau: Thanks very much for the compliments. We believe we are on the cutting edge, but we know we have only scratched the surface with college football. It's great to have support, encouragement and perhaps in particular, challenging questions that shape and reshape our approach.
Kevan Lee: As a diehard, biased Bronco fan, I see no way how the Broncos will finish any less than undefeated this season. You see them finishing 11-2. Why? Or more specifically, with the Mean Wins measurement factoring "Projected Win Estimate (PWE) numbers based on the likelihood of victory for the team itself in individual games," am I correct in understanding that the likelihood of victory for Boise State in games against Oregon/Tulsa and Nevada/Fresno is not terribly high?
Brian Fremeau: Boise State is projected in our system to be better than all but one team on its schedule -- Oregon (No. 27). And the Broncos get the Ducks on the blue turf. But even if a team were significantly better than all of its opponents and were projected to be 90% likely to win each of its individual games, that team should only be expected to win 90% of them total, not be undefeated. In our calculations, Boise State will have something in the neighborhood of a 50-50 matchup with Oregon at home, and maybe 55-45 with Tulsa and Fresno State on the road. Most of their other opponents - including Nevada - are not projected to be particularly strong, but even a 90+% per game PWE over 9 games may result in a loss to one of those teams along the way. Also, Boise State's 10.7 Mean Wins are more than any other team except Florida, and their 7.1 Mean Conference Wins are more than any other team except USC (which plays 9 conference games) and Florida.
Kevan Lee: If you had to choose the teams that would account for Boise State's losses on the season (one conference, one nonconference), which ones would they be?
Brian Fremeau: Teams almost never beat everyone they are supposed to and lose to only the teams they are supposed to lose to. That said, my best guess for two losses would be Oregon and Fresno State. Nevada would be a more significant upset, but not out of the realm of possibility. And I don't expect the Broncos to, but they better not fall asleep against Bowling Green and Tulsa.
Kevan Lee: Every preseason poll I've seen so far has the Broncos ranked somewhere in the teens. The Fremeau Efficiency Index (FEI) has them slotted at No. 39. To what would you attribute the noticeable difference between public perception of Boise State (ranked 10-20 in most preseason polls) and the FO projection? Is it media and experts not understanding advanced stats? Is it a love affair with picking a BCS buster? Something else?
Brian Fremeau: I think a 12-1 or 11-2 Boise State team would certainly be ranked in the 10-20 range at season's end in the national polls, so in a sense, I don't disagree with the preseason poll at all.
The polls have a different agenda than FEI, however.
The goal of the FEI system is to identify as precisely as possible the relative strength of all FBS teams through a robust analysis of their drive-by-drive efficiency, filtering out the noise of garbage possessions, adjusting for field position and strength of opponents, etc. The polls assess team strength as well, of course, but there are also considerations of where teams "deserve" to be ranked. There is validity to that approach, and I don't think polls over-rank non-BCS teams habitually (there is ample evidence of the opposite).
But during the season, I am often dismayed when in the national polls, the relative merits of a team's loss column take precedence over a broader evaluation of the team's power, or the recency of a particular loss carries too much weight, or the idea that teams are anchored to poll positions until a loss knocks a team down, or evaluations of teams are not consistently applied, etc.
Kevan Lee: Can you pinpoint the single biggest determining factor in the Broncos' No. 39 FEI?
Brian Fremeau: Boise State has the No. 26 Program FEI rating, the 5-year baseline for our projections, and their relatively low number of returning starters and relatively weak WAC brethren (6 WAC teams are ranked between No. 105 and No. 120 in our projections) pull their national projection down a few pegs.
One of two things must happen this year for Boise State's FEI rating to be much higher than projected: 1. the WAC (or more precisely, the WAC plus BSU's non-conference opponents) are better as a whole than projected, or 2. Boise State doesn't just lose once or go undefeated, it hammers everyone it beats, decisively. If both happen, FEI will love the Broncos this year. As a side note, you might find this link to last year's final FEI column interesting, as it provides a table of the best non-BCS teams since 2003 according to end-of-year FEI ratings. 2006 Boise State (which was predicted by FEI to defeat Oklahoma in the Fiesta Bowl that year, by the way) rated stronger than 2008 Utah; though you may also note that 2008 Utah had a higher year ranking (No. 12) than 2006 Boise State (No. 16), due to the strength of other teams in those respective years. Note also the No. 4 year ranking of 2004 Utah, a non-BCS conference team that obliterated everyone it faced. If any non-BCS team proved they deserved a shot at the team hoisting a national championship trophy at season's end, it was that one.
Kevan Lee: I appreciate how the statistics point out that the Bronco offense wasn't its usual stellar self last year, in large part thanks to a mediocre running game. What was your reaction when you saw this? Surprise? Horror? Ambivalence?
Also, the preview mentions Ian Johnson's loss as being a big hole to fill on offense, but looking at the numbers, the running game wasn't that hot with him last year. How much would you attribute the running game's struggles to an unproductive Ian Johnson or how much would you blame a faulty line and weak overall rushing attack? With the projected Offensive FEI being 73 for 2009, do I already know the answer?
Brian Fremeau: I'd say Ian Johnson's total contribution -- rushing, receiving, special teams play -- will be a big hole to fill. As for pinpointing Johnson vs. the line for the decline, its probably a bit of both. In general, and Bill and his play-by-play success rate stats may or may not disagree with me on this, I lean toward line play as the culprit for under-performing running games; and when things are really clicking, the lines never get enough credit. The thing that stands out for me that is too often overlooked in common stats is field position. Boise State started 55 of its 130 offensive non-garbage possessions last season at or inside its own 25-yard line (42.3%). In 2007, BSU started only 43 of its 140 offensive non-garbage possessions at or inside its own 25-yard line (30.7%). That's a pretty significant difference, and is the type of thing that can exacerbate other performance measures.
Kevan Lee: Would you have Johnson's success rates handy?
Brian Fremeau: Here are Johnson's success rates and other rushing stats as provided by Bill Connelly.
Boise State POE rankings (POE explained here)
- 33. Ian Johnson (+5.1 POE, 110.4 PPP+)
- 95. Jeremy Avery (+3.2 POE, 108.6 PPP+)
- 99. DJ Harper (+2.8 POE, 115.6 PPP+)
So they all pretty much averaged about the same PPP+.
Boise State Success Rates
- DJ Harper 43.6%
- Jeremy Avery 40.5%
- Ian Johnson 40.0%
About the same.
Boise State PPP
- Ian Johnson .417
- DJ Harper .381
- Jeremy Avery .363
Johnson was the most explosive, though .417 isn't an amazing total.
Boise State S&P
- Ian Johnson .817
- DJ Harper .817
- Jeremy Avery .768
Kevan Lee: The projections for Offensive and Defensive FEI (73rd nationally and 29th nationally, respectively) are, how do I put this nicely, bumming me out. The formula mentions returning starters and small conference adjustments "transition factors" as being part of the FEI projection. How much of either had a negative effect on the 2009 FEI projections for the Broncos?
Brian Fremeau: Both play a part, but the returning starters probably has a larger direct effect on the differences between 2008 Off/Def FEI and 2009 Projected Off/Def FEI. Does it bum you out less if I mention that Boise State has the 3rd-best projected Off FEI in the conference (behind Fresno State and Nevada) and the best projected Def FEI in the conference by far?
Kevan Lee: No, but thanks for trying.
The Offensive and Defensive FEI for the Broncos last year (ranked 42 and 14, respectively) and the Offensive and Defensive S&P+ (15 and 5, respectively) were significantly different. Also, I understand that FEI is drive-by-drive and S&P+ is play-by-play. Could you shed some light on what drive vs. play data says about a team and what it might mean going forward?
Brian Fremeau: This is a great question. We are really looking forward to exploring this in much more detail this season at Football Outsiders. One difference obviously has to do with micro- versus macro- observations and the benefits and pitfalls of each. Breaking data down further has its advantages, but it may also make things more difficult for drawing appropriate baselines for opponent adjustments. Measuring the success of a play affords more individual observations in a game, but it might make "success" more difficult to define and isolate consistently.
There are also fundamental differences in the way S&P+ and FEI discard "garbage" plays and possessions, which may have an impact on the rankings. And finally, there are certainly differences in the ways we adjust for opponents, which might make all the difference in the world. This will be a rich topic for research over the upcoming season.
Kevan Lee: Are you at liberty to provide the projected win total for the University of Idaho? I love a good laugh.
Brian Fremeau: Idaho is projected to be No. 120 out of 120 FBS teams. Their projected win total is a generous 2.07 (playing seven games against bottom-20 teams pays off, statistically, for even the meekest of the meek). Here is a link where you can find a table of projections for all 120 FBS teams.
***
That concludes the interview. Now go buy Football Outsiders Almanac.