Is a team allowed to decline a bowl invitation? And if they did what would the fallout be for the program?

I ask this because I was reading some other blogs and ran across a blog site where the fans of one team are so dissappointed in their home teams record that they feel any bowl invite they get should be declined even though they're bowl eligible. This should be interesting to the replies to this thread?