1. MahendraMahey (UKOLN) Stephanie Taylor (UKOLN) Niamh Brennan (TCD) Kevin Kiely (TCD) http://cerify.ukoln.ac.uk

2. Current and past CRIS/CERIF/RIM projects Research Information ManagementProcess Mapping and Gap Analysis TCD CRIS Questionnaires Site Visits Business Process Analysis‘as is’ and ‘to be’ 4 Priority Processes User requirements elicitation Prioritization of RIM Processes / Identification, Mapping, Gap and Stakeholder Analysis Data from at least 2 partners Thomson CRIS Research in View and InCites Increasing level of Engagement with a CRIS / CERIF Evaluation of CERIF and CRIS Import and Export of Data and Facilitation of data exchange CERIFy Approach People Centered Socio Cognitive TCD – CERIF People and Publications Institutions see value of CERIF and CRIS, Tell their regional institutions at regional and national events.CERIF and CRIS Engagement Increases Priority Process Areas – ‘as is’ and ‘to be’ CERIF TCD XML 2008 Mappings and Cross Walks relevant CERIF Schema InCites CERIF TCD XML 2002 CERIF Health Check Other, e.g. PURE? Reducing duplication Research in View Data Surgery Mahendra Mahey Stephanie Taylor Rosemary Russell Michael Day Talat Chaudhri NiamhBrennan Kevin Kiely Jimmy Tang Roisin Croker http://cerify.ukoln.ac.uk

3. InCites Exchange of Data http://cerify.ukoln.ac.uk

4. How Is This Used? RAE requirement Comparison with other universities (e.g. Russell Group) Global comparisons (updated annually) Citation information generated from this tool provides a lot of key information http://cerify.ukoln.ac.uk

5. Collection of Data Two-way process between Thomson Reuters & individual institution Requires a lot of work by institution Institutional data collected in local database of staff research information http://cerify.ukoln.ac.uk

6. User Issues “There is a lot of effort involved in understanding the data - there would be much value in standardising the data.” E.g. - problems with author ID - duplicates & null fields http://cerify.ukoln.ac.uk

7. User Requirements “If the data was in a standardised format it would make life much easier!” Nightly updates Internationalised data sets Relational database with mapping research output onto staff ID number for returned data Better mapping to institutional schools structure http://cerify.ukoln.ac.uk

8. Measures of Esteem http://cerify.ukoln.ac.uk

9. How Is This Used? RAE requirement Personal review for promotion process Assessment & benchmarking - internal & external As key information in the assessment in judging the overall quality of research outputs Drafting REF documentation Part of institution-wide annual planning cycle http://cerify.ukoln.ac.uk

10. Collection of Data From individual researchers on adhoc basis Systematic for RAE Continuous tracking for REF Ask individual schools to provide this on regular basis http://cerify.ukoln.ac.uk

11. User Issues Process is ‘woolly’ - difficult to provide meaningful data Perception varies depending on given point in a career path Citations alone are not accurate enough Context is needed to make sense of raw data Defining this process is very difficult Researchers can undersell themselves and don’t make information available http://cerify.ukoln.ac.uk

12. User Requirements Systematic way of capturing this is required Bringing a bibliometrics perspective to esteem-related information Take into account that one measurement doesn’t fit all Acknowledge differences in different subjects When one size doesn’t fit all, find where to plug the gaps in information Personalised audit tool to be built into RIM system http://cerify.ukoln.ac.uk

13. Thomson Reuters data exchange – what’s InCites?

14. Why CERIFy it? The community expressed interest in this process Current data is unsatisfactory ; the dataset is a specific view Institutional Citation Report is ‘historic’: based on affiliation at the time of authorship. Does not show Schools, Research Centres etc. (WoS API will not provide this either). Author, institution name variants (no automatic DAI in WoS/InCites) Missing papers (aside from general coverage issues) But.. institutions want the metrics + institutional structure in InCites + to embed data in local CRIS/IR: Sample Current exchange process is unsatisfactory: non-standard schema, heavy demands on institution, disappointing results Opportunity to create a CERIF data model for exchange of people, institutional affiliations, publication data & metrics Opportunity to demonstrate a 360⁰ data exchange using CERIF If successful, it should help to show value and help to engage institutions with CERIF Model can be re-used for other similar exchange processes

15. Process 1: InCites‘as is’ –Based on Queen’s University Belfast

16. Data drilldown Creation of CERIF Data Model for ‘InCites’ Exchange Process 1: InCites ‘to be’ –Generic (based on CERIFy Data Surgery) 3 Institutional Data automatically matched with TR records for authors & publications 4. InCites 'Institutional Profile' established and updated automatically 2. TR import and update deltas Thomson Reuters EXTERNAL [6] 5 TR send report to institution on results of matching process (including matched Author ID) 1. Online export of CERIF-based data to TR from RIM/CRIS via web services (nightly updates) MIS/CRIS developers [7] TR data integrated with CRIS data (if required) Note: may use TR API instead or alongside Research Office / Library Business Intelligence Officers/Institutional Analysts 5. Reporting on Research 8.1 Internal Reporting 8.2 External Reporting EXTERNAL DATA EXCHANGE PARTNERS

17. Over to Kevin…

18. Where we are – results CERIF Data model Data exchange next week! Includes: extended ‘publication types’ list Multiple identifiers (Researcher ID; Institutional ID, HESA ID, Publication ID etc Full metrics = standardisation Next steps QUB Thomson Reuters are working on automating the process using Web Services…

20. Process 1: Esteem ‘as is’ –Based on Aberystwyth University 1 Completes ‘Other Activities’ annual form on Research Activity Database (RAD) SharePoint interface Academic 4. Hardcopies printed for Research Monitoring meetings 2. Details of form transferred to RAD 3. Pdf report produced Research Monitoring Team 5. Electronic copy deposited within RAD SharePoint site Departmental document folder HoD/DDR 6. Access electronic copy deposited within RAD SharePoint site Departmental folder Deans 7. Hardcopies printed for Research Monitoring meetings DATA EXCHANGE PARTNERS

21. Process 1: Esteem ‘to be’ –CERIFy Data Surgery 1. CRIS/RIMacademic profile is automatically populated with elements of 'Measures of Esteem' MIS/Library Academic 2. Academic updates CRIS/RIM with other elements of Measures of Esteem 3. CRIS / RIM system data is queried and reported by Research Monitoring Team Research Monitoring Team HoD/DDR 4. Reports generated by HoDs, Deans etc. Deans 5. Reports generated by HoDs, Deans etc. External agencies/Funders 6. Data Exported to external agencies as required

22. Indicators of Esteem: based on CERIFy Data Surgery, current practice In partner institutions + a number of external sources

23. Indicators of Esteem - the extended version: based on original CERIFy IE, With additions to publications & outputs Please see spreadsheet for latest list CERIFy Indicators of Esteem

24. CERIFy Esteem Indicators Model – based on MICE (Impact) Note: current focus for Esteem is more on Indicators rather than on Measures Esteem Indicator language table Link tables cfOrgUnitId etc.cfEsteemIndicatorId cfClassID cfClassSchemeId cfStartDate cfEndDate Esteem Indicator table OrgUnit Project Person Publ. Etc. OrgUnit Project Person Publ. Etc. Link table Esteem Measure table cfEsteemMeasureIdcfEsteemIndicatorId cfStartDate cfEndDate cfEsteemMeasureIdcfEsteemMeasureKind cfCountInteger cfValueFloatingPoint cfValueJudgementalNumeric cfCountIntegerChange cfValueFloatingPointChange cfValueJudgementalChange cfValueJudgementalText Link tables Link tables Link tables cfOrgUnitId etc.cfEsteemIndicatorId cfClassID cfClassSchemeId cfStartDate cfEndDate cfOrgUnitId etc.cfEsteemIndicatorId cfClassID cfClassSchemeId cfStartDate cfEndDate cfOrgUnitId etc.cfEsteemIndicatorId cfClassID cfClassSchemeId cfStartDate cfEndDate NB: this adaptation has yet to be finally validated by Brigitte

25. Esteem Indicators mapped to UK RAE 2008, Australian ERA

26. Esteem Indicators for REF5: Contribution to Discipline/Collaborations

28. ‘Authority-controlled’, taxonomies and definitions lists should be agreed and built into the systemNOTE: the RAE 2008 submissions under ‘Research Environment, Measures of Esteem’ provide an extremely valuable source of information on the type of information provided by institutions in this area. This body of knowledge is not available for other REF-relevant areas such as ‘Impact’. The RAE 2008 ‘Measures of Esteem’ is a particularly rich source of data from the disciplinary viewpoint. The analysis of these ‘texts’ has great potential to assist with developing the type of ‘authority lists’/taxonomies described above. The following slides show the results from analysing the RAE 2008 University of Bath ‘Measures of Esteem’ submission in the field of Pharmacy. This text was analysed using Many Eyes and Open Calais. The results of using Open Calais for semantic analysis and entity extraction even for this small sample are very positive.

30. Brigitte Jorge, EuroCRIS

31. Thomson Reuters InCitesCERIFy team, Philadelphia

32. Bo Alroe, Atira /UK Pure User Group (Esteem Indicators)

33. http://cerify.ukoln.ac.uk