2. Introduction & Overview
• The Institutional Profiles project is an initiative to collect
multidimensional factual data about academic institutions for the
purposes of profiling their activities and performance.
• The objective is to profile 1,000 of the leading academic
institutions around the world. Primarily universities.
• The data is used to inform the
Times Higher Education World University Rankings.
• Thomson Reuters makes the data available
on our InCitesTM platform as the
Institutional Profiles module.
4. Academic Reputation Survey
• An annual survey of academics. Participants are polled to give
feedback on the reputation of institutions in their subject area.
• A clear distinction between Research and Teaching reputation
• Invitation only to prevent manipulation
• Regional balancing and multiple languages to overcome bias
• 16,649 respondents from 144 countries
– 68% Academic staff
– 14% Research staff
– 7% Senior institutional leadership
– 6% Graduate/ post-graduate student
– The remainder were administrative staff and graduate students
• Copy of the survey, white paper of the survey mythology and
demographics of the responses are available at:
http://science.thomsonreuters.com/globalprofilesproject/gpp-reputational/
5. Institutional Data Gathering
• Thomson Reuters collects data directly from the institutions
• We make considerable efforts to collect high quality,
internationally comparable data while minimizing the work
burden.
– Use existing authoritative data sources
– Common set of global data definitions. First in the world.
– Strong support structure
– Comprehensive data validation
• Comparison to previous year
• Logical data checks
• Statistical outliers
• Third party sources
6. Institutional Data Gathering
• Data collected included things such as the official institution
name and affiliations, key contact points and mission
statement
• Institutions also provide detailed information about their
activities across multiple subject areas
• Data include, but are not limited to:
– Numbers of academic & research staff
– Numbers of students (various levels)
– Degrees awarded
– Institutional & Research funding
– International staff & students
7. Institutional Data Gathering
• On average institutions have filled in 83% of the data
requested
• Detailed subject level data across the board –
– 92% of institutions reporting subject data for Academic staff.
– 91% report subject data for students
• Some examples of prestigious institutions that have supplied
comprehensive data include:
– Princeton University – McGill University
– University of Hong Kong – University of Munich
– National University of Singapore – Monash University
– Tsinghua University – University of Oxford
7
8. Bibliometric data
• Utilizes the Thomson Reuters Web of Science data,
considered the gold standard by many research
evaluation organizations.
• Combinations of bibliometric and institutional data can
create new and unique indicators.
– Papers per Academic staff
– Normalized Citation impact
– Papers per research $
• Data must be normalized to
overcome subject bias.
9. Data Analysis and Interpretation
• Analysis and interpretation of the data are essential to
understand the performance.
• Modification of data for unbiased comparability.
• Benchmarking / normalization to overcome bias:
– There are fundamental differences to funding,
publications and PhD rates for difference subjects
– Data for one institution is compared to the average
for all institutions for the same year/subject
to create “relative” performance
– Understanding where an institution
fits within a distribution aids
comparison of diverse data types
10. Institutional Profiles
• Part of the InCitesTM platform.
• Customized institutional comparison and profiling
– 665 institutions from 69 countries
– Profile of each participating institution
– Multiple options for viewing key performance indicators
• Groups of related indicators
• Single indicators across multiple subjects
– Trend analysis
– Scatter comparisons
– Compare peers and peer groups
– Single subject classification for
all data types
11. View Profiles of institutions
View information about an institution
15. Scatter Plot analysis
Identify where a university falls
in a two dimensional array.
Rollover a specific point to see
the details
16. Summary
• An excellent resource to explorer academic institutions
and understand their competencies
• Hundreds of participating institutions globally
• Quality data about all aspects of Higher Education
• Data that is internationally comparable
• Expert interpretation and analysis
• Robust data for evidence based decisions making
17. THANK YOU
To find out more:
http://ip-science.thomsonreuters.com/globalprofilesproject/
Contact us at:
science.profilesproject@thomsonreuters.com
Notas del editor
Welcome to this introduction to the global Institutional Profiles project. This short presentation will explain the basics of how the project is structured and overview of the some of the services that are possible with this rich source of data.
There are three main sources of data that feed into the profiles: Results of the annual academic reputation survey Data provided by the institutions themselves on staff, students, funding etc. Bibliometric data based on the publications and citations of the journal articles of the institution
One of the major components of the profiles project is the annual academic reputation survey. Annually we conduct a survey of academics and researchers and asked them to provide feedback about what they consider to be the best institutions globally within their subject area. The survey makes a clear distinction between reputation for research and for teaching. The survey is by invitation only to prevent institutions from manipulating the results. The invitations are structured to give a fair balance of different geographic regions and subjects. Additionally, the survey was translated into multiple languages to overcome English language bias. However, even with these efforts to overcome regional bias there are still variations in response rate and post survey results were modified to overcome bias.
The second major source of data is factual data from the institutions themselves. We have the support from the highest level at each of the participating institutions. Thomson Reuters has made considerable efforts to collect high quality, comparable data with a minimum work burden for the participating institutions. Use existing data sources when available – for example we used the UK Higher Education Statistics Agency data pre-fill the profiles. So that the institutions have to simply validate the data. Common data definitions for all institutions - this is the first time that this type of data has been collected on a global scale using a single set of definitions. The definitions are based on existing international standards for statistical reporting of education and research from UNESCO and OECD. To develop and fine tune the definitions we worked closely with our external advisory board. Strong support structure, detailed documentation, tutorials and webinars etc. – for example in 2010 we ran a series of 15 webinars with several hundred attendees. We also have a team of dedicated data editors, based in region, to answer questions and help participating institutions. Data validation is very important to us. Validation is done primarily by comparing institutions data submissions to publicly available sources of data such as government reports. We also apply logical validation processes (for example checking to make sure that the number of international students does not exceed the number of students) and outlier identification algorithms.
Data collected included things such as the preferred institution name, key contact points and mission statement. This information makes the profile more informative and helps the users of the profiles to understand the nature of the institution and make to help them contact the right office at the institution should they wish to do so. Potential users of the profiles include: administrators of other institutions who may be looking for partners, industrial organizations looking to help universities commercialise their research, funding agencies and government bodies. It is in the institutions own interests to provide clear yet complete information. Institutions also provide detailed information about their activities across multiple subject areas. This is essential to better understand the make up of an institution and where their particular strengths may lie. Often times a institutional characteristic may be hidden by the subject portfolio, for example an institution that is renowned for its social sciences research and is in fact very well funded for the subjects it is focusing on may appear to be poorly funded when comparing to another institution that focuses on subjects such as the physical or medical sciences that typically have a high level of funding. Data supplied by the institution include, but are not limited to: Numbers of academic staff Numbers of researchers Numbers of students Institutional funding Research funding
The third component of data, bibliometric data, relies on the publications authored by the academics and researchers of the institution. The data is sourced from the Thomson Reuters Web of Science, considered the gold standard by many evaluation bodies globally. Many potential indicators can be generated from this data, such as the total number of papers per subject or the citations per paper. However, combining bibliometric data and data provided by the institutions can create new and unique indicators of performance, for example papers per research dollar spent as a measure of research effectiveness.
Data on its own is difficult to understand. Without understanding of the landscape in which the data sits a single value is a relatively meaningless number. By analyzing and interpreting the data we can turn data and information into meaningful knowledge about institutional performance Some data needs to be modified for comparability. For example funding data is modified, not just to overcome differences in currency exchange rates but also to overcome differences in the costs of living in difference countries. Benchmarking and normalization to used to understand the relative performance. We do this by comparing the data of one institution to the average of all institutions, in this way we can determine if a particular value is average, above average or outstanding. There are also fundamental differences in the characteristics of data for difference disciplines. For example, funding in the medical sciences will typically be higher than for the social sciences reflecting the higher costs of infrastructure that are required. There are also significant differences with regards to publication and citation trends in difference subjects. By benchmarking data within it’s discipline the data becomes much more meaningful, and it is a unique aspect of the Institutional Profiles project that the same subject classification is used across all the different data types. This type of benchmarking and normalization is essential to make comparisons between universities that have a different subject focus. Data is also generated to help understand where a university fits in the distribution of all institutions.
Data is MIT.
One can view the basic profile of any university, see factual information and data a wide variety of data points and indicators.
An excellent resource to explorer academic institutions and understand their competencies Hundreds of participating institutions globally Quality data about all aspects of Higher Education Data that is internationally comparable Expert interpretation and analysis Robust data for evidence based decisions making Helping to make a more robust World University Ranking
