 My table shows twenty autumn internationals played during 2004. It is an arbitrary selection, naturally, as matches like Japan-Romania are ignored. But the important games are all there.
Columns show scores and the difference between those scores expressed as a fraction. So for example Scotland defeated Japan 100-8 which is a difference of 12.5 (100/8). Columns on the right are the same thing only using IRB rankings. On the same row, for example, Scotland's IRB ranking was 75.4 and Japan's was 65.4, a difference of 1.15 (75.4/65.4)
What does this say? First, it does support the IRB claim that their rankings are pretty good at predicting winners. In this sample they were correct 80% of the time (or 16 matches from 20).
This is all the more impressive when you see that the matches where IRB predictions fluffed have mitigating circumstances. In two matches France were involved. Enough said. Les Bleus defeated Australia 27-14 despite IRB predicting a Wallabies win; most people said France were lucky. France were themselves defeated by Argentina 24-14; most people said France were unlucky. This would be further proof – if any more was required – that you never quite know which French team will turn up on the day.
The IRB were also wrong in predicting South Africa would beat Ireland. The Boks lost by 5 points, 17-12, at Lansdowne Road. However, the difference was a controversial try that will be debated for years. The referee, Paul Honnis from New Zealand, asked the Boks skipper to warn his players about slowing the ball down. In order to do this he temporarily suspended the defensive line. Meanwhile the Irish took a hasty kick-off which Honnis allowed and scored an (unconverted) try over an undefended try line. As five points remained the difference at full time there is some evidence that South Africa can feel robbed by the ref.
England-Australia, the fourth occasion when the IRB were wrong, was also curious. England should have won according to the IRB, just. Yet they were down 15-0 at one stage before finding their feet. Penalties were missed routinely but they came back massively to lead 19-15. Momentum was theirs and to some thinking they deserved a final win, however uncertain. Yet ill-discipline gifted Australia penalties in the final moments and the England coach, Andy Robinson, captured the way the game was won and lost: “You get to 19-15 up, as we did, and then gave two penalties away on our 10-metre line? That, to me, is why we lost the game” It was, in short, an odd game where England came close to both losing and winning.
No relationship between scoring gap and IRB ranking gap
But I do not mean to digress too much into these unusual matches. Rugby is about the unpredictable and if the IRB were right all the time it would be rather boring and rather easy money from the bookies. Suffice to say that the IRB are pretty good at predicting matches.
However, the IRB rankings are close to hopeless at predicting the actual score. Over these twenty matches winning teams scored roughly four points to one. An average winning score was 45 points versus a losing score of 13 (or a difference of 3.5). Yet, and this is the central problem with the IRB system, these scores had virtually no relationship with IRB rankings. The average IRB ranking of the winning team was 82.9 and losing teams were 75.8 (or 1.09). This is a slender difference of just a few percent. Scoring differences of 3.5 versus IRB ranking differences of 1.09? As related to each other as the proverbial bishop is related to his niece the actress.
Here's some examples that really brings this home. Wales whipped Romania 66-7 or by a ratio of 9:1. Yet the IRB spaced the team by merely 13%; Wales had an IRB ranking of 76.9 versus Romania’s 68.2. When Scotland thrashed Japan 100-8, or by twelve times as many points, the IRB ranked their gap by a meagre 15% (75.4 divided by 65.4 = 1.15). New Zealand rubbed Italy's nose in it 59-10, or by six points to one, on a tiny IRB gap of 23% (1.23). England beat Canada 70-0 despite a tiny IRB gap of 28% (1.28).
In defence of the IRB this need not be fatal. The objective of the rankings is not to predict actual scores. It aims to divide winners from losers. Changing a ranking system is also difficult. The IRB will say that they have spent time in developing and explaining how the current rankings work. This is a fair point. And the IRB rankings are explained in quite some detail if you don't mind a little discombobulating. The IRB also deserve credit for a formula that ranks teams in the right order more often than not.
It is also true that a lot of rankings have a similarly narrow spread. The NFL in America, for example, ranks several dozen teams using approximately a 30 point spread.
A bigger spread is needed for rugby rankings
But rankings don't have to work this way. Tennis rankings have a much larger spread as one example from several. The current world #1 player, Roger Federer, has a decidedly perky ranking of 1,267 points. The tenth ranked player has 384 points. The fiftieth ranked player has a limp 148 points. In other words Federer is nearly 9 times stronger than the 50th ranked player (1267/148 = 8.6). I'm not a tennis fan but I bet this provides a fair idea of distance in the score between those players if they were to play each other.
Sizing up everything there is a big case for expanding the IRB rankings in some way that can help fans predict scoring margins. Knowing this in advance would only add to the game's enjoyment. |
