The document discusses issues around the availability and use of evaluative and comparative information in tertiary education systems. It notes that a variety of actors should be involved in determining what data is collected, including universities, government authorities, students, and industry. Responsibility for collecting accurate data needs to be clearly defined. There is also a need to standardize core data sets and move beyond just reporting inputs to also evaluating outputs and results. Reliable data is important for ensuring consistency and preventing manipulation. Developing strong management information systems and independent agencies can help improve the availability and use of information.

1. Use of Evaluative and Comparative Information
In the Management of Tertiary Education Systems
Bratislava 17 June 2005
Summary report from workshop break-out group discussions, made by Lewis Purser
The groups discussed the broad themes linked to current obstacles regarding the
availability of information, and how to improve this. The discussions developed
around a series of issues pertinent to these questions, and could be summarised in the
following structure.
Actors: the groups discussed the range of actors which should be both involved and
interested in the optimal availability and dissemination of information regarding
higher education. Apart from the obvious involvement of the universities, at both
central and decentralised levels, the list of such actors includes the national
authorities, and also, depending on the size and structure of the country, the regional
and local authorities. The market is obviously also interested in information, and can
be segmented into the student market (students, their families and hopefully also their
high schools), the research market (a wide range of potential users of university
research and development expertise), and the services market, interested in the
economic, social and cultural benefits to be gained from proximity or links to a
university. These actors will all have different perspectives regarding the university
and therefore different interests and needs regarding information. Many of their
interests could be grouped under the wider heading of the needs of society
Responsibilities. The responsibilities regarding information differ at various stages in
the chain, but are essential to identify in order to ensure minimal obstacles and
maximum effectiveness. The choice of which data is useful obviously depends on a
dialogue between different interested actors, each from its own perspective. The main
responsibility to ensure an adequate choice of data responding to the wider
information needs of society should lie with universities and government, but with the
active involvement of the other actors, as users of the information. Agreement on core
data sets or templates would be useful here and could provide a basis for developing
more specialised data sets to respond to specific needs. The issue of responsibility for
the supply of data is linked to the need for the accuracy and impartiality of such data.
While the primary sources for much data will obviously be the higher education
institutions and, for data at a national level, the relevant authorities, there were strong
voices heard regarding the need for an independent responsibility for supplying data
for public use. The presentation and interpretation of such data are similar issues,
although at these stages there are many possibilities for involving wider groups of
actors, including the media, think tanks, etc. In general, the need to evolve from a
reporting culture to an evaluation culture was stressed, and this will change the
responsibilities of the main actors involved.
Types of data. Large amounts of data are currently already available, from a wide
variety of sources: the universities, national statistics offices, various governmental
ministries and other bodies, student and stakeholder groups. There tends to be a focus
on data rather than on readily usable information. Furthermore, many such data are
not readily accessible to a non-specialist, and indeed would take considerable effort to

2. bring together and prepare in a usable format responding to an individual need for
information. The data needs to respond to the perspectives of the various bodies,
which can vary considerably. Some data, eg as used by accreditation agencies, is
essentially part of a control mechanism, while other actors may be interested in data
which focuses on improvement mechanisms. As already noted, there are currently few
agreed templates in the countries represented defining the core data sets necessary. It
was agreed important to move from input-dominated data to output data as part of an
increased focus on the results of higher education for both individuals and society.
The reliability of data is essential to the user and for the wider effective and efficient
functioning of higher education systems. For this a strong emphasis needs to be
placed on mechanisms to ensure consistency of data across institutions and fields. The
publication, wide availability and public scrutiny of the data will help identify
potential errors, and significantly reduce the possibilities for the deliberate
manipulation of data. One key issue for the universities and other major actors is the
use of well designed management information systems. This can imply significant
costs in ICT and in staff training.
Discussions also focused on the role of agencies in supplying data. There were
ideological differences regarding whether such an agency should be publicly or
privately funded, with both models being potentially open to undue influence through
a variety of mechanisms, unless strong governance structures are put in place. A
public-private partnership may be an interesting model to explore. Whatever the
model, the agency must have the authority to produce and publish the data which it
considers necessary, including data which may be uncomfortable for various
important actors. The agency must also benefit from a certain degree of financial
stability and independence in order to ensure its work can proceed without undue
hindrance or interference. The sustainability of the information provision is vital,
therefore the sustainability of the providing agency is also important.