Interview with Football Outsiders' Brian Fremeau on Boise State's 2009 season projections
The guys at Football Outsiders are geniuses. They crunch football numbers to provide a fabulous layer of depth and brilliance to a sport that is ripe for analytical progress. I am their faithful Charlie, and they are my map to candy mountain. At least, that was my sappy romantic view of things until I got my hands on the Football Outsiders Almanac and found that they had picked the Broncos to finish 11-2 and be the 39th best team in the country.
What gives? I went to FO writer Brian Fremeau to get some answers. Here are those answers.
(Note: The following interview is based on the Boise State content from Football Outsiders Almanac, available for purchase at the FO Store. I highly recommend that you but it. The statistics used by FO are explained in more detail at the site's glossary page.)
Kevan Lee: First off, I really admire the work that you and Football Outsiders have done to bring advanced stats to college football. You guys are heroes. Don't let anyone tell you otherwise.
Brian Fremeau: Thanks very much for the compliments. We believe we are on the cutting edge, but we know we have only scratched the surface with college football. It's great to have support, encouragement and perhaps in particular, challenging questions that shape and reshape our approach.
Kevan Lee: As a diehard, biased Bronco fan, I see no way how the Broncos will finish any less than undefeated this season. You see them finishing 11-2. Why? Or more specifically, with the Mean Wins measurement factoring "Projected Win Estimate (PWE) numbers based on the likelihood of victory for the team itself in individual games," am I correct in understanding that the likelihood of victory for Boise State in games against Oregon/Tulsa and Nevada/Fresno is not terribly high?
Brian Fremeau: Boise State is projected in our system to be better than all but one team on its schedule -- Oregon (No. 27). And the Broncos get the Ducks on the blue turf. But even if a team were significantly better than all of its opponents and were projected to be 90% likely to win each of its individual games, that team should only be expected to win 90% of them total, not be undefeated. In our calculations, Boise State will have something in the neighborhood of a 50-50 matchup with Oregon at home, and maybe 55-45 with Tulsa and Fresno State on the road. Most of their other opponents - including Nevada - are not projected to be particularly strong, but even a 90+% per game PWE over 9 games may result in a loss to one of those teams along the way. Also, Boise State's 10.7 Mean Wins are more than any other team except Florida, and their 7.1 Mean Conference Wins are more than any other team except USC (which plays 9 conference games) and Florida.
Kevan Lee: If you had to choose the teams that would account for Boise State's losses on the season (one conference, one nonconference), which ones would they be?
Brian Fremeau: Teams almost never beat everyone they are supposed to and lose to only the teams they are supposed to lose to. That said, my best guess for two losses would be Oregon and Fresno State. Nevada would be a more significant upset, but not out of the realm of possibility. And I don't expect the Broncos to, but they better not fall asleep against Bowling Green and Tulsa.
Kevan Lee: Every preseason poll I've seen so far has the Broncos ranked somewhere in the teens. The Fremeau Efficiency Index (FEI) has them slotted at No. 39. To what would you attribute the noticeable difference between public perception of Boise State (ranked 10-20 in most preseason polls) and the FO projection? Is it media and experts not understanding advanced stats? Is it a love affair with picking a BCS buster? Something else?
Brian Fremeau: I think a 12-1 or 11-2 Boise State team would certainly be ranked in the 10-20 range at season's end in the national polls, so in a sense, I don't disagree with the preseason poll at all.
The polls have a different agenda than FEI, however.
The goal of the FEI system is to identify as precisely as possible the relative strength of all FBS teams through a robust analysis of their drive-by-drive efficiency, filtering out the noise of garbage possessions, adjusting for field position and strength of opponents, etc. The polls assess team strength as well, of course, but there are also considerations of where teams "deserve" to be ranked. There is validity to that approach, and I don't think polls over-rank non-BCS teams habitually (there is ample evidence of the opposite).
But during the season, I am often dismayed when in the national polls, the relative merits of a team's loss column take precedence over a broader evaluation of the team's power, or the recency of a particular loss carries too much weight, or the idea that teams are anchored to poll positions until a loss knocks a team down, or evaluations of teams are not consistently applied, etc.
Kevan Lee: Can you pinpoint the single biggest determining factor in the Broncos' No. 39 FEI?
Brian Fremeau: Boise State has the No. 26 Program FEI rating, the 5-year baseline for our projections, and their relatively low number of returning starters and relatively weak WAC brethren (6 WAC teams are ranked between No. 105 and No. 120 in our projections) pull their national projection down a few pegs.
One of two things must happen this year for Boise State's FEI rating to be much higher than projected: 1. the WAC (or more precisely, the WAC plus BSU's non-conference opponents) are better as a whole than projected, or 2. Boise State doesn't just lose once or go undefeated, it hammers everyone it beats, decisively. If both happen, FEI will love the Broncos this year. As a side note, you might find this link to last year's final FEI column interesting, as it provides a table of the best non-BCS teams since 2003 according to end-of-year FEI ratings. 2006 Boise State (which was predicted by FEI to defeat Oklahoma in the Fiesta Bowl that year, by the way) rated stronger than 2008 Utah; though you may also note that 2008 Utah had a higher year ranking (No. 12) than 2006 Boise State (No. 16), due to the strength of other teams in those respective years. Note also the No. 4 year ranking of 2004 Utah, a non-BCS conference team that obliterated everyone it faced. If any non-BCS team proved they deserved a shot at the team hoisting a national championship trophy at season's end, it was that one.
Kevan Lee: I appreciate how the statistics point out that the Bronco offense wasn't its usual stellar self last year, in large part thanks to a mediocre running game. What was your reaction when you saw this? Surprise? Horror? Ambivalence?
Also, the preview mentions Ian Johnson's loss as being a big hole to fill on offense, but looking at the numbers, the running game wasn't that hot with him last year. How much would you attribute the running game's struggles to an unproductive Ian Johnson or how much would you blame a faulty line and weak overall rushing attack? With the projected Offensive FEI being 73 for 2009, do I already know the answer?
Brian Fremeau: I'd say Ian Johnson's total contribution -- rushing, receiving, special teams play -- will be a big hole to fill. As for pinpointing Johnson vs. the line for the decline, its probably a bit of both. In general, and Bill and his play-by-play success rate stats may or may not disagree with me on this, I lean toward line play as the culprit for under-performing running games; and when things are really clicking, the lines never get enough credit. The thing that stands out for me that is too often overlooked in common stats is field position. Boise State started 55 of its 130 offensive non-garbage possessions last season at or inside its own 25-yard line (42.3%). In 2007, BSU started only 43 of its 140 offensive non-garbage possessions at or inside its own 25-yard line (30.7%). That's a pretty significant difference, and is the type of thing that can exacerbate other performance measures.
Kevan Lee: Would you have Johnson's success rates handy?
Brian Fremeau: Here are Johnson's success rates and other rushing stats as provided by Bill Connelly.
Boise State POE rankings (POE explained here)
- 33. Ian Johnson (+5.1 POE, 110.4 PPP+)
- 95. Jeremy Avery (+3.2 POE, 108.6 PPP+)
- 99. DJ Harper (+2.8 POE, 115.6 PPP+)
So they all pretty much averaged about the same PPP+.
Boise State Success Rates
- DJ Harper 43.6%
- Jeremy Avery 40.5%
- Ian Johnson 40.0%
About the same.
Boise State PPP
- Ian Johnson .417
- DJ Harper .381
- Jeremy Avery .363
Johnson was the most explosive, though .417 isn't an amazing total.
Boise State S&P
- Ian Johnson .817
- DJ Harper .817
- Jeremy Avery .768
Kevan Lee: The projections for Offensive and Defensive FEI (73rd nationally and 29th nationally, respectively) are, how do I put this nicely, bumming me out. The formula mentions returning starters and small conference adjustments "transition factors" as being part of the FEI projection. How much of either had a negative effect on the 2009 FEI projections for the Broncos?
Brian Fremeau: Both play a part, but the returning starters probably has a larger direct effect on the differences between 2008 Off/Def FEI and 2009 Projected Off/Def FEI. Does it bum you out less if I mention that Boise State has the 3rd-best projected Off FEI in the conference (behind Fresno State and Nevada) and the best projected Def FEI in the conference by far?
Kevan Lee: No, but thanks for trying.
The Offensive and Defensive FEI for the Broncos last year (ranked 42 and 14, respectively) and the Offensive and Defensive S&P+ (15 and 5, respectively) were significantly different. Also, I understand that FEI is drive-by-drive and S&P+ is play-by-play. Could you shed some light on what drive vs. play data says about a team and what it might mean going forward?
Brian Fremeau: This is a great question. We are really looking forward to exploring this in much more detail this season at Football Outsiders. One difference obviously has to do with micro- versus macro- observations and the benefits and pitfalls of each. Breaking data down further has its advantages, but it may also make things more difficult for drawing appropriate baselines for opponent adjustments. Measuring the success of a play affords more individual observations in a game, but it might make "success" more difficult to define and isolate consistently.
There are also fundamental differences in the way S&P+ and FEI discard "garbage" plays and possessions, which may have an impact on the rankings. And finally, there are certainly differences in the ways we adjust for opponents, which might make all the difference in the world. This will be a rich topic for research over the upcoming season.
Kevan Lee: Are you at liberty to provide the projected win total for the University of Idaho? I love a good laugh.
Brian Fremeau: Idaho is projected to be No. 120 out of 120 FBS teams. Their projected win total is a generous 2.07 (playing seven games against bottom-20 teams pays off, statistically, for even the meekest of the meek). Here is a link where you can find a table of projections for all 120 FBS teams.
***
That concludes the interview. Now go buy Football Outsiders Almanac.
0 recs |
8 comments
|
Comments
Numbers can lie
Don’t sweat the number crunching of them pointy headed geeks. :) Not everything in life can be explained in a mathematical formula nor can the future be predicted with one.
The natural wonder that is the Bronco Football Program is one of those things. It can’t be explained or rationalized by someone who understands numbers more than football.
We will prevail Homer, don’t worry your pretty little head over it. We are One Bronco Nation under God, not under Football Outsider Almanac. :)
"I'd rather be a smartass than a dumbass."
by Belexes on Jul 25, 2009 9:52 AM PDT reply actions 0 recs
Acutally, I agree with FOA
If you don’t think numbers can rule our world, you need to take more brainwashing math classes.
We knew our offense wasn’t clicking the way we wanted it to last year. The defense was great for scoring, but not the best with yards allowed. Those are immediate examples of the weaknesses Boise had.
However, the data from last year can only imply slightly for this year. They don’t collect data from spring/fall practices and can only project what may happen this year. As they said, make FOA very happy be obliterating opponents and the data will be better.
"You know where i'm from, a little suspicion about one's true identity and motives is considered good manners."
-- Nale
by Loque on Jul 25, 2009 10:47 AM PDT reply actions 0 recs
The numbers would have D.J. Harper be the Broncos' starting running back
I found that point fascinating, and I think you’re right, Loque, about their being value in statistics insomuch as determining past performance and discovering trends. Predicting future success? Some statisticians might think so, but it seems like any sort of predicting methodology (preseason polls, guessing) is faulty.
"We're not teaching them about sports. We're teaching them about life!" - Tony Perkis
by Kevan Lee on Jul 27, 2009 6:18 AM PDT up reply actions 0 recs
Que me estan diciendo?
Kevan, was FOA speaking English during that interview? I’m glad you logically minded folks can understand the numbers and stats. As for me, I prefer blatant bias-ness and over-inflated expectations to number crunching and stark relaity any day of the week.
Go BSU, beat everyone by 100 points.
"The brothers, they don't think I know a butt-load of crap about the gospel, but I DO." -Nacho
by BroncoFever on Jul 25, 2009 12:34 PM PDT reply actions 0 recs
Donde esta la biblioteca, Pedro?
"We're not teaching them about sports. We're teaching them about life!" - Tony Perkis
by Kevan Lee on Jul 27, 2009 6:18 AM PDT up reply actions 0 recs
Nice
Besides the great content, I LOVE the two avatars
by Jeremy Mauss on Jul 25, 2009 4:27 PM PDT reply actions 0 recs
“if a team were significantly better than all of its opponents and were projected to be 90% likely to win each of its individual games, that team should only be expected to win 90% of them total, not be undefeated.”
So how does Einstein explain 3 undefeated regular seasons in 5 years or 54-2 vs WAC opponents since ’02? That means BSU wins 96% of the time vs WAC. Maybe that extra 6% is the “Luck Factor Index”?
I guess I tend to dismiss ANYONE who suggests BSU is going to struggle with either FSU (See: 61-10 in ’08 and Pat Hill vs BSU all time) or Tulsa (#83 ranked defense in ’08).
by MooreWalksOnWater on Jul 27, 2009 8:35 AM PDT reply actions 0 recs
I understand why they're "Outsiders".
Just doesn’t mean a whole lot to me.
Boise State is 55-9 in the last 5 seasons. Only one of those losses is at home.
In that time frame Boise State has averaged 40.94 points per game.
While giving up an average of 20.38 points per game.
Average margin of victory of 20.56 points.
In all nine losses, Boise State lost by average margin of 10.8 ppg. 98 points total.
Only once in 9 losses did they reach their average of 40 points (Louisville, 2004)
In fact, they only cracked 30 in one other game (ECU, 2007)
Five of the nine losses were decided by 6 points or less.
The point of the useless numbers is that Boise State has only been stopped either by themselves or really good defense.
2004- Louisville 2004 44-40 (had the ball with a minute left, Zabransky INT)
2005- Georgia 2005 48-13 (Zabransky has 6 TOs…by himself, that and really good defense)
2005- Oregon State 30-27 (GW field goal, with 1:03 left)
2005- Fresno State 27-7 (Dominated by the defense, only loss in which the Broncos were held to single digits.)
2005- Boston College 27-21 (Had the ball with 40 seconds left in the red zone, Zabransky INT in the endzone…AGAIN)
2006- Flawless
2007- Washington 24-10 (No good excuse here, they got pushed around.)
2007- Hawaii 39-27 (Led 27-26 at the 3:30 mark of the 3rd, gave up 13 unanswered points)
2007- ECU 41-38 (GW FG as time expires, Bronco came back from 24 points down to tie in the 4th)
2008- TCU 17-16 (Broncos led until the 9 minute mark of the 4th. Moore throws an INT with 1:53 left, then, Jeremy Childs a reception with 0:06 left, TCU recovers)
If you want to beat the Broncos, keep them out of the endzone. Only now they’re defense is allowing them to put up less and still win.
Boise State - The best in all the land (The "land" being Idaho, and large parts of California, Oregon, and Nevada.)
by Mikrino on Jul 27, 2009 2:37 PM PDT reply actions 0 recs















