(... Continued from Part I)
Flaw #3: The BCS Lacks Security
The BCS standings are compiled from the results of two human polls (Harris Interactive and Coaches’ Polls) and six computer ranking systems (Anderson/Hester, Billingsley Report, Colley Matrix, Massey, Sagarin, and Wolfe). How far should we trust the integrity of these rankings? I’ll cover the human polls in more detail below, so I’ll focus on the computer rankings here.
The six computer ranking systems are typically run by individuals, individuals who started as passionate fans of sports and/or mathematics. The systems are time-proven over multiples years and generally well respected. The concern here isn’t with the methods used by the computers. It is with the methods used by the BCS to ensure the safety of the results.
I asked BCS Administrator Bill Hancock what they do to verify the data. He says, “The BCS does not compile the six computer rankings, nor the two ‘human polls.’ Those are compiled by outside entities that have given permission for them to be used by the BCS. We have been assured by each computer rankings provider and by the administrators of the Harris and Coaches Polls that they have thoroughly checked their work.”
So basically the BCS takes their word for it. That was fine when the computer rankings were in the realm of hobbyists and used mostly for entertainment value. Now these rankings are part of the BCS. They help decide who goes to what bowl and who gets on TV. The BCS should not take their word for it, and the concern shouldn’t just be that they “check their work.” There are more serious issues they need to guard against.
While some of the computer rankings do have some form of double-checking internally (with a partner – like Jeff Anderson and Chris Hester of Anderson & Hester College Football Computer Rankings) or with a relatively non-proprietary ranking system (like The Colley Matrix). Others have a closed system. For example, Ken Massey of Massey Ratings says, “Nobody checks the calculations, but the software code has been tested for over 10 years now and no bugs have been reported.” And there is an overall reluctance for most to share their software. As Anderson puts it, “that would not be okay with us, because no one could verify our results without having access to every aspect of our program, and our program is proprietary. “
The BCS should hire an independent accounting firm to verify the work of every computer ranking system. They should check for errors and they should check for tampering. It may seem unlikely, but there are millions of dollars at stake to universities and far more at stake in gambling. It is not outside the realm of possibility that somebody could try to “pressure” the individuals who run the computer rankings into changing the results for malicious purposes. Certainly there are plenty of instances of this in professional sports. Recently, NBA referee Tim Donaghy was sentenced for making dishonest calls based on pressure from organized crime. Who is to say this can’t happen within the BCS, and how would anyone ever know? An independent group (not a specific person) verifying the results is a good investment and helps keep all those involved safe. I can only imagine the damage a gambling scandal would cause the BCS.
The BCS should only allow computer rankings that will work with the independent accounting firm. This is big business; the attitude should be more like Richard Billingsley of The Billingsley Report, who says, “After completion of negotiations that ensured the safety of my formula and provided proper compensation, yes, I would be willing to cooperate with an external audit." That sounds like a professional arrangement that any businessman would expect. This is not a hobby, guys. Deals involving proprietary Intellectual Property (IP) get signed every day in the corporate world. They sign a Non-Disclosure Agreement (NDA) that protects their work, and then they move forward. Any computer ranking that can’t abide by this doesn’t belong in the BCS and should be relegated to hobbyist status where it can be as secretive as it wants.
For the record, I don’t think there has been any past issue with errors or tampering, and I am not insinuating any past problems. This is a concern about the future. The Academy Awards uses PricewaterhouseCoopers (the professional services/auditing firm) to ensure accuracy. If “Oscar” thinks that the winner of “Best Sound Mixing” is worth protecting, the BCS should at least give the same level of protection to the college football national champion.
While the concern about security also applies to the human polls, they suffer from a bigger, final flaw.
Flaw #4: Secret Ballots Breed Distrust
If there is one major advantage that the AP Poll has over the BCS human polls, it is that every AP ballot is made public every week. This allows anybody – fans, universities and the voters themselves to see that the process is open and fair. I run a site called Pollspeak. On a few occasions we’ve found errors in the posted ballots and reported them. We’ve had voters who have also noticed mistakes in their own ballots, which would never have happened if they weren’t made public. Even if fans don’t agree with the voters, they at least know that the rankings really were their opinions and not some mythical cover-up or mistake.
The Coaches’ Poll and Harris Interactive College Football Poll do release their ballots for the final poll of the regular season. This is a very good thing, and it shows that there is no technical or legal issue preventing them from doing it every week. So why don’t they? The BCS’s Hancock says, “The only BCS standings that matter are those compiled on selection Sunday. After all, the standings are in place for one reason: to identify the participants in the National Championship Game and the teams that are eligible for selection by the other BCS bowls.”
This goes back to Flaw #2 – The BCS Doesn’t Appreciate its Own Power. Again, the BCS is taking the stance that the main reason it exists is to match #1 and #2. However, as discussed above, the polls matter all year long. They determine who gets the best TV exposure, whose name appears in ever paper and how often they appear on ESPN’s bottom line ticker. Hancock also said that “transparency” in the last Harris poll of the year is important, and I agree. So if it is important for the last poll, they just need to realize that it is also important for every poll. If transparency really is important, why wouldn’t it be important for any poll released in any week?
There is no good reason for private ballots. This isn’t like voting for president (which, of course, is a highly secured process). Voting in a poll with secret ballots is more like not knowing how your representatives in Congress are voting on the issues. Although we don’t elect poll voters directly, they are serving our interests as fans, players and universities. Unlike the computer rankings, there is no proprietary technology that must be protected, so there is no reason to use an NDA if the ballots are just made public.
The BCS should take a cue from the AP and only include human polls with public ballots in its formula. The BCS currently uses two different human polls and they have unique situations.
The Harris Interactive College Football Poll is actually commissioned by the BCS. Harris Interactive was hired to replace the AP poll when they served noticed to the BCS in 2004. The rules of the HICFP are set by the BCS so if the BCS wants public ballots, all they need to do is ask. Problem solved.
The Coaches’ Poll is a joint effort between USA Today and the American Football Coaches Association (AFCA). The BCS cannot dictate public ballots for the Coaches’ Poll directly. USA Today’s Craig Bennett says the decision was “made by vote of Division I-A head coaches following the 2004 season. I was not privy to the discussions concerning that vote, so you'd have to check with the AFCA.”
Unfortunately the AFCA has not responded after multiple attempts. It seems unlikely that the coaches would make their ballots public without some motivation, and a BCS requirement for public ballots might just be the motivation they need.
But losing the coaches’ poll would not be a huge loss for the image of the BCS. It is a poll whose voters are directly affected by the outcome of their own vote. Coaches are typically rewarded financially based on the performance of their team, and their employers (universities) are typically rewarded based on the performance of their conference (bowl and TV revenues). It would be hard to imagine a more inherently biased poll. The fact that it uses secret ballots only casts more doubt on it, but the Coaches’ Poll has other issues that are beyond the scope of this article.
The good news for the BCS is that it is bigger than the Coaches’ Poll or any of its individual components. Its clout is even bigger than the longest running poll, the AP.
With the brand awareness it currently enjoys, the BCS could remove the Coaches’ Poll completely or replace it with another poll it commissions similar to the Harris Interactive Computer Football Poll, and it could carry on with the same strength it has today or even stronger, having replaced the sketchiest part of its formula. Of course, if the Coaches’ Poll made all of its ballots public, this wouldn’t be necessary.
There you have it. Four flaws in the BCS, which if corrected, would strengthen the already mighty “event.” This article may have read like a tirade against the BCS, but it isn’t. The BCS is a huge production, and I’ve pointed out only four flaws. Concentrating on the negative can make even the best system look bad.
The BCS isn’t bad; I’ve just neglected to talk about the 20-plus greatest things about the BCS – for example, this weekend’s lineup of college football games. The best part about the BCS is that a Pac-10 fan will be watching a game this weekend between two SEC teams and will cheer like it was a home game. With the BCS, every game counts ... not just every game for your alma-matter, but every game played by a Division I-A team. When the BCS fully embraces its significance to college football and fixes its flaws, we can all go back to concentrating on just the games.