Academic league tables in Russia have proved too hot for the state to handle, Nick Holdsworth reports. University league tables in Russia have proven to be as hot a political potato for ministry mandarins as in Britain. The Russian Federation's State Committee for ³ÉÈËVRÊÓƵ - effectively the ministry governing the sector, which still retains its Soviet title - had historically evaluated and measured the relative strengths and merits of those 200 institutions out of the country's 536 universities and higher education institutions which came directly under its control.

But following the 1991 higher educational reforms - which gave much greater academic, political and business freedom to universities, such tables became increasingly sensitive.

Despite two years, 1992/93, when the state committee published league tables for those institutions directly under its authority - using complex methodologies regarded in Russia as among the world's most advanced systems for providing performance statements for the sector - Vladimir Kinelyov, its chairman - in effect minister for higher education - decided last year to delegate the job to the so-called "public organisations", recently formed higher education subject associations, not under official control.

Those in such associations - which include economics, medical, engineering and classical educational areas - wryly observe that Mr Kinelyov's was an astute move. Last year's "politically harsh situation" meant that most people in academia had at best a dim view of the state committee's activities. "It was a smart move because during last year, due to the harsh situation of the government, everybody had a negative attitude towards the committee and its decisions, so it was well timed," one association member said.

³ÉÈËVRÊÓƵ

The league tables - or ranking system, as the Russians prefer to call them, are recognised as useful performance measures by academics, officials and students, according to Alexander Kushel, deputy director general of the Russian Association for Engineering Education, one of the subject associations set up in the last two or three years. Dr Kushel was formerly in charge of licensing and accreditation at the state committee, and was closely involved in the league table policy.

Great care was taken to ensure that the indicators used to measure the relative value and performance of Russia's universities were objective and the complex statistical calculations used to place institutions on the league tables were accurate and objective. The methodology for arriving at tables accurate to around 95 per cent was discussed at length and agreed by the state committee and the Academy of Sciences, which represents senior academics and university vice chancellors in Russia.

³ÉÈËVRÊÓƵ

"The methodology was discussed at meetings of the associations and vice chancellors of Russian universities . . . so when they are published in association publications or the educational press they should be accepted by all involved as they accept the methodology," Dr Kushel said. The methodology used for measuring institutional ratings for potential performance, current activity and overall, or general, value, might differ in specifics according to the type of institutions being evaluated - indicators for engineering universities would vary from those which specialised in the humanities, for example.

But basically the measurements were largely the same, he said. Higher education institutes were measured according to the levels of research conducted; the number and qualifications of senior staff - the number of professors and holders of doctorates and masters degree; the material resources available - the facilities, number and quality of computers and laboratory equipment; the quality of student output, measured by graduate destinations and the numbers later completing professional qualifications or gaining membership of recognised associations.

By measuring and weighing the relative values of these indicators and evidence of work in hand to improve the university's student and research programmes, collators and statisticians working throughout the diverse range of "public organisations" could produce accurate and objective performance indicators, Dr Kushel said. He denied that league tables - which although generally accepted are by no means universally approved of within Russian academia - might be used to deny funding to those at the bottom of the tables. "These tables are accurate for internal, Russian use, and also for interested institutions from overseas - much is already understood about our higher education institutions in Russia anyway.

"We could say that those universities at the top of the list are giving their students the best education, but all are accredited institutions, so even the last institution on the lists award an accepted diploma, which under Russian law is equal to that on the top of the list. It would be a mistake to use these lists as a reason for closing those institutions - the higher society is educated the better it is for everybody."

³ÉÈËVRÊÓƵ

The result is that the relative strengths and shortcomings of Russian universities are, perhaps, more clearly delineated than anywhere in the west. But the attendant political sensitivities inherent in the concept of such tables in Britain are not lost on the Russians.

Alexander Prokopchuk, deputy director of the state committee's directorate of international cooperation, recognises that the political discomfort caused by the committee's experiment in extending and publishing the tables in the past few years - and their subsequent delegation to the non-governmental educational associations - reflects the difficulties inherent in official attempts to grade higher education.

"Trying to develop objective criteria for measuring the strengths of universities is a rather delicate matter . . . we all know that quality of education does not necessarily relate to the numbers or quality of professors. The state committee had supported the idea of developing criteria for measuring the performance of universities and it understood the need to find different ways to evaluate the quality of training and performance in Russian universities, but official publication of these tables was not the best way forward.

Valery Meshalkin, the state committee's current head of the licensing, accreditation and notification directorate, said that the accuracy and objectivity of university ranking tables were currently subject to the same vagaries as the Russian economy. Only when social and economic stability was achieved would accurate and truly reliable statistics be available on which to base performance tables.

³ÉÈËVRÊÓƵ

"It's hard to classify universities because the economic system is so difficult. This work is being done for the future - once things are more stable we will be ready to produce much more accurate rankings for Russian universities."