Corps, darned Corps, and statistics

I have been following with some interest the Drum Corps Scores and Rankings mailing list (a Yahoo group, so you have to register with Yahoo to see the archives of the list). What's interesting about it is that as well as the scores from contests it also posts an aggregated score and ranking of all the Division I, II and III corps on major tours this summer. The idea is to make an average of a corps scores over the season and compare them with the same averages of al the other corps. I am not sure if it's a running average for a few weeks or it uses scores for the whole summer. Here's a link to an example for August 1, 2004. (It's on Google's archive of R.A.M.D. newsgroup post. The paranoid conspiracy theories about slotting are just a bonus for reading it on R.A.M.D. :P ). There is also a nice graphic depiction of the Division I averaged rankings here made by James Meister.

It is effectively a running set of rankings something like the positions of baseball and hockey teams over the season. It's a calculated version of something that drum corps fans like to do in their head during the season. I personally don't think this ranking is much use for predicting the winners at Denver, CO this week. The analogy with sports doesn't hold for drum corps.

When I marched we used to hear about the scores other corps were getting early in the season before we "met" them in competition. We would hear about a rival getting higher or lower scores than what we had received that year so far and then there would be much debate and anticipation of the rankings of various corps together. The thing I noticed was that these anticipated rankings never held up when we actually met the corps in the same contest.

This happened and still happens because the scores from different contests are generated by a panel of different judges each time and it doesn't mean anything to compare scores from different contests directly even if they are contests that occur on the week. The only way to tell if a corps is better than another one is to have them appear in the same contest.

I acknowledge that the scores from different panels will probably agree within 2 or three points as they try to make things as standarized as possible but in the end it's a judgement about art and it's difficult to get people to agree about such things, even people trained to do so.

So DCI Worlds is still up for grabs and there are probably some surprises in store. The further away from the top scores you get, the more room there is for change in my experience anyway. The Kavaliers aren't going to win DCI but they may move up two or three rankings from that predicted by this ranking thing.

Don't use the average rankings thing as a reason to claim: "We wuz robbed!".