Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Societal Impact

694 visualizaciones

Publicado el

Societal Impact
Nicolas Robinson Garcia, INGENIO (UPV-CSIC), Universitat Politècnica de València, Spain / Daniel Torres-Salinas, Universidad de Navarra and Universidad de Granada (EC3metrics & Medialab UGR), Spain
Recently there is an increasing pressure on the development of indicators and methodologies that can offer evidences of the societal impact of researchers’ activity. This presentation will offer a comprehensive overview on the definition of societal impact, types of impact, and the attribution problem when searching for potential indicators. A special attention will be given to altmetric indicators and their potential role in tracing social engagement and its relation with societal impact. Examples of potential uses and current lines of work will be presented.
***************************
Scientometric procedures are increasingly used to analyse developments and trends in science and technology. Decisions to be taken often have severe implications. Consequently data handling, indicator construction and interpretation require competent expert knowledge, which is currently only available to a limited extent for all stakeholders in Central Europe not the least due to lacking training opportunities. Responding to the lack of a pertinent scientometrics education (especially in German speaking countries) and to the increasing demand (particularly of research quality managers), the University of Vienna (A), the German Centre for Higher Education Research and Science Studies - DZHW (D) and the Katholieke Universiteit Leuven (B) joined cooperatively to found the European Summer School for Scientometrics (esss) in 2010.

Publicado en: Educación
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Societal Impact

  1. 1. European Summer School| September 8, 2016 Societal Impact Nicolas Robinson-Garcia Daniel Torres-Salinas INGENIO (CSIC-UPV) Univ Navarra and Univ Granada (ec3metrics & Medialab ugr) www.ingenio.upv.es
  2. 2. Objectives Understand the motivations that lead for demands of societal impact indicators 1 Know the limitations and current problems existing to meet such demands 2 Learn about potential uses of altmetrics to showcase ‘aspects’ of societal impact 3
  3. 3. Index Brief Intro What is societal impact? How can we measure societal impact? Altmetrics and the ‘broader impacts’ of research Potential uses of altmetrics Food for thought
  4. 4. “The last 30 years have witnessed a slow, but continuous process of reorganization of European universities” Geuna & Muscia, 2009 Brief intro https://static.pexels.com/photos/30824/pexels-photo.jpg
  5. 5. Brief intro Global competition and budget cuts Accountability and transparency Evaluation and funding
  6. 6. Brief intro • Bibliometrics is seen as an answer to provide impact measures within and even outside the scientific sphere citation analysis – patent analysis – univ-industry collab • In the recent years there has been a critical revision on the limitations of these metrics scientific impact ≠ societal impact http://www.logic-canvas.com/wp-content/uploads/2013/11/Transparency-Glass.jpg
  7. 7. Brief intro DORA Declaration Against the use of journal based indicators Leiden Manifesto Best practices on the use of bibliometric indicators The Metric Tide UK Report on the role of bibliometric indicators in research evaluation
  8. 8. What is societal impact? Societal impact is usually referred as the social, environmental, cultural or economic benefits derived from academic activities Teaching Research Third Mission SOCIETAL IMPACT
  9. 9. What is societal impact? Type of impact Output Activities Scientific- scholarly Knowledge Growth Papers, books, reports, cases, magazines Increase and organise the stock of knowledgePublication outlets Research networks Workshops, collaboration, conferences, committees Build and convene networks Supply of human capital Societal Cultural Archives, museums, festivals, exhibition and performance, events Social Policy changes, social benefits Problem solving Socioeconomic Patents, spin-offs, start-ups, contracts, consultancy Support entrepreneurialismTechnological Source: Robinson-Garcia, van Leeuwen & Rafols, in progress
  10. 10. How can we measure societal impact? Socioeconomic & Technological Impact • Largest body of literature • Synergies between industry and university • Tracking of knowledge transfer Social Impact • Translational research • Environmental research Cultural Impact • Field-specific • Largely unexplored • Affects mainly the Social Sciences & Humanities
  11. 11. How can we measure societal impact? The attribution problem Researcher publishes a paper Researcher gives lectures? Media coverage? Patent protection? Advocacy? Local council implements new policy Increase on tourism rates Employment rates increase in industry Creates a spin-off?
  12. 12. How can we measure societal impact? The attribution problem Researcher publishes a paper undertakes a research project? Local council implements new policy Researcher gives lectures? Media coverage? Patent protection? Advocacy? Increase on tourism rates Employment rates increase in industry Creates a spin-off?
  13. 13. How can we measure societal impact? An example. El Cabanyal The district •Old village, now a district •Architectonic value The sting •Access to the beach •Salvem el Cabanyal The link? •Few papers (National journals) •Not all researchers involved are authors The attribution problem
  14. 14. GOALS How can we measure societal impact? UK REF Selective allocation of research funding to Higher Education Institutions Produce evidence of benefits of public investment in research Provide benchmarking information and reputational yardsticks King’s College London & Digital Science, 2015
  15. 15. First attempt to allocate funding based on non- academic impact How can we measure societal impact? UK REF King’s College London & Digital Science, 2015 “Impact is defined as ‘any effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’ (REF, 2011)”
  16. 16. How can we measure societal impact? UK REF PANEL A Biomedical & Health Sciences 6 UoA – 1586 case studies PANEL D Humanities & Arts 10 UoA – 1617 case studies PANEL C Social Sciences 10 UoA – 1965 case studies PANEL B Life & Earth Sciences, Engineering, Physics, Maths 9 UoA – 1469 case studies
  17. 17. Assessing societal impact through impact case studies How can we measure societal impact? UK REF IMPACT CASE STUDY TEMPLATE 1. Summary of the impact 2. Description of the underpinning research 3. References of the research 4. Details of the impact 5. Sources to corroborate the impact
  18. 18. How can we measure societal impact? UK REF Case studies by type of impact
  19. 19. How can we measure societal impact? UK REF Case studies by type of impact
  20. 20. How can we measure societal impact? UK REF Ratings by type of impact
  21. 21. How can we measure societal impact? UK REF Wilsdon et al., 2015 “[…] for the impact component of the REF, it is not currently feasible to use quantitative indicators in place of narrative impact case studies, or the impact template”
  22. 22. Altmetrics and the ‘broader’ impacts of science Wilsdon et al., 2015 “[…] wider use of quantitative indicators, and the emergence of alternative metrics for societal impact, could support the transition to a more open, accountable and outward-facing research system”
  23. 23. Altmetrics and the ‘broader’ impacts of science Classic perspective Author perspective Two perspectives
  24. 24. Case: societal impact indicators at UGR Multidimensional evaluation using biplot. Research at the University of Granada Source: Milanes-Guisado, Y. (2015). Multidimensional evaluation of research. Micro analysis at the university of Granada during the period 2009-2013 [doctoral thesis]. Universidad de Granada. Supervisors: Daniel Torres-Salinas, José Navarrete, Francisco Solis Research questions: • ¿What is the metric profile of research groups taking into account bibliometric and non-bibliometric indicators? Methodology: • Time frame: 2009-2013 • Analysis unit: 161 research groups = 2216 res. • Methods: Correlations biplot analysis
  25. 25. Case: societal impact indicators at UGR We calculated 46 indicators and classified them into 9 different dimensions We reduced and make a biplot analysis Multidimensional evaluation using biplot. Research at the University of Granada
  26. 26. Case: societal impact indicators at UGR Altmetrics Funding and projects Scientific output Knowledge transfer Media and news impact Scientific impact – Citations Teaching dissemination 9 Dimensions of academic activity and impact
  27. 27. Case: societal impact indicators at UGR Altmetrics Funding and projects Scientific output Knowledge transfer Media and news impact Scientific impact – Citations Teaching dissemination Societal impact related dimensions
  28. 28. Case: societal impact indicators at UGR Altmetrics Google + Mentions Facebook Likes Twitter Indicators Knowledge Transfer Patents Contracts Spin offs Media and News Impact Newspapers contributions Public exposure Mentions in Newspapers Teaching Dissemination Teaching Material Innovation Projects Some examples of “societal impact” indicators
  29. 29. Case: societal impact indicators at UGR Some examples of the information sources
  30. 30. Case: societal impact indicators at UGR Method remarks 1) Calculation of 46 indicators for research groups in four areas 2) Normalization and standarization of indicators values using z-scores 3) Representation of dimensions and groups into two dimensions using biplot analysis 4) Reducing the biplot with clockwise representation Now we are going to see an example in Exact Sciences
  31. 31. Potential applications of altmetrics These are the dimensions (blue) These are the research groups (green) BIPLOT: “EXACT SCIENCES 52” Research groups
  32. 32. Potential applications of altmetrics Teaching Dissemination Media and News Impact Knowledge Transfers Altmetrics These are cluster and dimensions used to identify SOCIETAL IMPACT
  33. 33. Potential applications of altmetrics Altmetrics What we see is that most of the research groups are in dimensions related to traditional publications and the impact system (ie: Web of Science) Scientific Impact Scientific output
  34. 34. Potential applications of altmetrics Altmetrics DOES ALTMETRICS MEASURE SOCIETAL IMPACT? We see that altmetrics is more related to scientific impact and scientific output (at the UGR) Scientific Impact Scientific output
  35. 35. Potential applications of altmetrics Teaching Dissemination Media and News Impact Knowledge Transfers So, for Exact Sciences at the University of Granada we have a really clear quadrant related to SOCIETAL IMPACT
  36. 36. Potential applications of altmetrics We have identified four groups with a clear Societal Impact orientation FQM325 FQM171 FQM338 FQM325 FQM324
  37. 37. Teaching dissemination Media and news impact Funding and projects National scientific output Knowledge transfer Scientific impact - Citations International scientific output Altmetrics and webmetrics Finally this is the general and multidimensional overview of the exact sciences at the UGR
  38. 38. Potential applications of altmetrics Social engagement through Twitter 1.Select a researcher 2.Identify its follower/followee network 3.Identify different communities of contacts 4.Identify geographical/cognitive/… proximities
  39. 39. Potential applications of altmetrics Social engagement through Twitter L Waltman’s Twitter network
  40. 40. Potential applications of altmetrics L Waltman’s Twitter network Social engagement through Twitter
  41. 41. • Threats derived from a purely qualitative assessment Peripheries and prestige • Threats derived from purely quantitative assessment Biases by type and scope of impacts Visualizations as a balanced solution Food for thought
  42. 42. European Summer School | September 8, 2016 Questions? Nicolas Robinson-Garcia Daniel Torres-Salinas INGENIO (CSIC-UPV) EC3Metrics SL www.ingenio.upv.es
  43. 43. European Summer School | September 8, 2016 Especial thanks to Yusnelkis Milanés for providing data from her PhD thesis www.ingenio.upv.es Source: Milanes-Guisado, Y. (2015). Multidimensional evaluation of research. Micro analysis at the university of Granada during the period 2009-2013 [doctoral thesis]. Universidad de Granada. Supervisors: Daniel Torres-Salinas, José Navarrete, Francisco Solis

×