League tables and the perfect buffet

Gratuitous Luton Town picture. Loose connection;
the mighty Hatters finished top of their league last year.

The league table season is upon us (yet again), with data of numerous shapes and sizes being churned out in an attempt to capture and drive readership. And, it would appear, to mystify and infuriate University leaders and those with a stake in monitoring performance and reputation. Which includes Market Researchers.

The University of Bedfordshire’s recent emphasis on consolidating our long standing commitment to listening to and enhancing our students’ experience has started making a real impact on our rankings. We certainly saw this in last year's excellent National Student Survey results and are crossing fingers and quietly hoping for similar improvements this year. We’ve also performed well in the 2015 Complete University Guide (CUG) , leaping an impressive seven places over last year’s ranking.

But how are these league tables put together? Why don’t they broadly repeat each other’s findings?

Well, take Harper Adams, a small Shropshire institution specialising in agriculture. Harper Adams doesn’t feature in Guardian league tables, but ranks a creditable 60 and 63 for the CUG and The Times respectively. Meanwhile, WhatUni? boldly report them as the country’s second highest rated institution. Hmm.

The pithiest explanation for such variations is in looking at how tables are compiled. Different - sometimes radically different metrics - are used, from comprehensive sector statistics to relatively small samples from student surveys. Already messy things get even messier, as metrics are then given different weights in calculations. And the fun doesn't end there; individual methodologies, too, change year on year. Our own performance in the CUG table would have been even more impressive, for example, had the weightings in the two measures used for research not changed since last year.

All of which raises a number of questions about how prospective applicants should and how they might view these rankings. Any serious consideration should certainly be underpinned with an appreciation of how individual league tables are compiled. And in a landscape of around 130 institutions of various shapes and sizes, how valid is it to make sweeping, reductive comparisons? Harper Adams, for example is a small, specialist, very rural institution, most of whose students are mature and study part-time. An entirely different beast to city based Russell Group institutions (Birmingham, Manchester), or heavily internationalised/heavily research-orientated London institutions (King’s, UCL) or institutions like Bedfordshire (and Hertfordshire, Northampton, Coventry etc) who have urban campuses with a more ‘traditional’ mixture of home, EU and international undergraduate students.

 

Harper Adams and Manchester - who stole my pint?

All this, before any mention of applicants and students themselves. I am proud and pleased to have studied at the University of North London (now London Met). Despite the institution's lowly reputation, I didn't consider any other uni. North London was where I wanted to be; the English course and the vibrant mixture of life experiences appealed to me far more than dreaming spires, Madchester (as was) or the comforts of my local Poly or FE College. I still believe I made the right choice.

As a Market Researcher, I'm keen on analogies. I also enjoy chasing down a good buffet. Buffets rely on a good array of sandwiches, but then, depending on tastes, diets, budget etc., also typically involve quiches, salads, cocktail sausages, pickles etc. No one element is inherently 'better' than the other. And if the bride's uncle is partial to a party egg, it doesn't follow that everyone else will be. Indeed, there may be guests with egg allergies, for whom the bride's uncle's food choice would be a disaster.

Granted, the HE sector isn't a wedding buffet. But hopefully the point stands; you can trick a vegan into eating a ham sandwich ("there's only.a bit in there"), or fob off a foodie with cheese and pineapple on sticks. Or you can transplant a perfectly happy medical student from Edinburgh to London Met, or a perfectly happy Computing student from Bedfordshire to the Highlands and Islands. Either way, there are round holes, square pegs and a whole truckload of trouble brewing.

So, are league tables worth all the fuss and coverage? Do potential students read, let alone act on them? Our research suggests our students' engagement with league tables is cursory at best, with other wisdom suggesting that as long as institutions don’t appear in the very lowest or very highest reaches of a table, they won’t really be noticed. And there are so many other 'truths' which a 1 to 120 overall ranking won't tell you; individual university departments may have a reputation and ranking which belies the institution's overall placing and there will be factors individuals consider important which just can't be quantified (location and being close to home, anyone??) Some league table providers carefully explain and contextualise - here's a good example from Which? - but league tables - any league tables - notoriously take on a life all of their own and offer quick and easy news copy.

In summary, logic might suggest that if a university applicant can't get into the top rated uni, they might go for the second placed institution. And why not Harper Adams? Well, why not indeed; the evidence points to Harper Adams providing an excellent experience, just as long as you're looking for something "agricultural" and in the countryside. What's good for the goose won't always be good for the gander, no matter how it's dressed up or presented.

Happy league tabling......