Rethinking Metrics of International Growth and Impact of Open Access
•Download as PPTX, PDF•
2 likes•680 views
Panel presentation at the "Symposium on Innovations in Reducing International Knowledge Isolation"
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Rethinking Metrics of International Growth and Impact of Open Access
Similar to Rethinking Metrics of International Growth and Impact of Open Access (20)
More from Leslie Chan
More from Leslie Chan (20)
Recently uploaded
Recently uploaded (20)
Rethinking Metrics of International Growth and Impact of Open Access
- 1. Rethinking Metrics of International Growth and Impact of Open Access Leslie Chan Bioline International University of Toronto Scarborough Symposium on Innovations in AAAS 2012, “Flattening the World: Building a Reducing International Global Knowledge Society” Knowledge Isolation Vancouver, Canada
- 2. Key points • Open Access as an enabler • “Journal” no longer serves the needs of networked scholarship • From Wealth of Nations to Wealth of Networks • Need to rethink measurements of “impact” and values, especially for development • Innovations are happening in the “peripheries” but there are gatekeepers • Aligning funding and reward policies with new metrics
- 3. The problem • Access to literature from the North • Dissemination of research from the South • The barriers are structural and institutional
- 6. The World of Journal Publishing According to Thomson’s ISI Science Citation Index Data from 2002 http://www.worldmapper.org/display.php?selected=205
- 10. “… at a recent editorial team meeting, we discussed a research paper from a LMIC author. The science was well done and with a little editing for English, the paper was potentially publishable. But should we send it out for review? The question we were wrestling with was whether its findings were sufficiently new to make it worthy of page space in the journal. This is always a consideration for all manuscripts, since competition for space is intense and a priority is to publish interesting research that adds something new to the field, rather than too many replications of studies already done. So the initial response when deciding whether to send the paper out for peer review was: Reject. We already know this, don't we?”
- 11. “No journal can afford to devote all or even most of its precious page space to studies essentially finding again what others already found, with only the places changing. And this may be a good place to remind authors that we almost never publish prevalence studies, unless they are truly the first ever done (and sometimes not even then), since they tend to be of interest primarily in the countries within which they were conducted.”
- 12. So who decide on what is “new” and legitimate knowledge? And Who have access to that knowledge?
- 13. “We editors seek a global status for our journals, but we shut out the experiences and practices of those living in poverty by our (unconscious) neglect. One group is advantaged, while the other is marginalised.”Richard Horton, THE LANCET • Vol 361 • March 1, 2003
- 14. “Research or reviews that cover diseases unlikely to be encountered in the western world will not gather the citations that some editors seek. But if this commercial environment does seriously skew content away from what matters to those people the journal claims to serve, as it surely does at some journals, the culture of medicine is distorted, even harmed.” Richard Horton (2003)
- 15. The Political Economy of Knowledge Production
- 16. “Is the scientific paper a fraud?” “I mean the scientific paper may be a fraud because it misrepresents the processes of thought that accompanied or give rise to the work that is described in the paper. That is the question and I will say right away that my answer to it is ‘yes’. The scientific paper in its orthodox form does embody a totally mistaken conception, even a travesty, of the nature of scientific though”. http://contanatura- hemeroteca.weblog.com.pt/arq Sir Peter Medawar uivo/medawar_paper_fraud.pdf (From a BBC talk, 1964)
- 25. The Centro de Referência em Informação Ambiental, CRIA (Reference Center on Environmental Information) http://www.cria.org.br/
- 26. Active Bioline Journals graphed with Google Maps. Retrieved April 26, 2010 from: http://maps.google.com/maps/ms?ie=UTF8&hl=en&msa=0&msid=117328677437228595850.00044ec490b69019e64a3&z=2 26
- 29. Bioline users March 27 – April 26, 2010 Retrieved April 26, 2010 from: http://analytics.google.com 29
- 30. But
- 32. The World of Journal Publishing According to Thomson’s ISI Science Citation Index Data from 2002 http://www.worldmapper.org/display.php?selected=205
- 37. From “Big” science to Networked science Knowledge for local problem solving
- 39. The IF is negotiable and doesn’t reflect actual citation counts http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0030291
- 40. The IF cannot be reproduced, even if it reflected actual citations http://jcb.rupress.org/content /179/6/1091.full
- 41. The IF is not statistically sound, even if it were reproducible and reflected actual citations http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf
- 42. The IF are more eff http://iai.asm.org/content/early/201 1/08/08/IAI.05661- 11.full.pdf+html?view=long&pmid=2 1825063
- 45. The SURFfoundation has released Users, Narcissism and Control— Tracking the Impact of Scholarly Publications in the 21st Century.
- 46. Need for policy alignment and institutional redesign Governance of Knowledge Commons Rethink the values and reward system Social Accounting and Expanded Values
- 47. Broadening the definition of “success”, “impact”, “value” and “capital” Business value monetary return, financial capital, efficiency, competiveness Scholarly value Reputation and citation; trust; symbolic capital Institutional value Public mission, community outreach, intellectual capital Social value Equity, participation, diversity, social capital Political value Evidence based policy, transparency, accountability, civic capital
- 48. Institutional Design Sustainability as a set of institutional structures and processes that build and protect the knowledge commons (after Sumner 2005, Mook and Sumner 2010)
- 49. Conclusions • Open Access is just the substrate, but an essential one • Metrics drive behaviour, but we have been using the wrong metrics
- 50. http://www.openoasis.org http://www.bioline.org.br http://www.openaccessmap.org Thank You! chan@utsc.utoronto.ca
Editor's Notes
- Impact of Open Access and Alternative Metrics of International GrowthLeslie Chan, University of Toronto Scarborough It is generally acknowledged that researchers and institutions in the Global South suffer from knowledge isolation because of poor infrastructure and lack of access to key resources, including the current literature. The remedy is therefore capacity building and the transfer of not only knowledge, but also the institutional framework of knowledge creation from the North to the South. In this context, Open Access to the scholarly literature is seen as a means of bridging the global knowledge gap. In this presentation, I argue that a key contributor to the continual knowledge divide and the invisibility of knowledge from the Global South is the persistence and dominance of Northern frameworks of research evaluation and quality metrics, coupled with outmoded national and international innovation policies based on exclusion and competitiveness. These narrow measures have tended to skew international research agenda and undermine locally relevant research. A great opportunity that Open Access provides is the means to develop alternative metrics of research uptake and impact that are more inclusive of knowledge from the South, particularly those with development outcomes. In particular, it is important to re-conceptualize and re-design the metrics of research impact to reflect new scholarly practices and the diverse means of engagement enabled by OA and the new wave of social media tools. At the same time, appropriate policies need to be developed to reward open scholarship and to encourage research sharing — issues of particular importance for ending knowledge isolation. Examples of the new kinds of “invisible college” enabled by networking tools and OA will be presented, and particular attention will be paid to innovations emanating from the periphery.
- metrics of total publications and citations.Top 15 countries account for 82% of total publicationsAuthor with African institutional affiliation account for less than 1% of global output, and S. Africa has the highest output. The rest are “invisible”Consequence of trying to publish in “International” journal results in neglect of important local problems and solutions that are appropriate for local conditions.
- information, research, and publication capacities are intimately linked. Investigators, publishers, editors, and editorial organisations all have important parts to play in solving this global information poverty.Horton R. North and South: bridging the information gap. Lancet 2000; 355: 2331–36.““institutional racism” has a very precise meaning. According to the UK’s Commission for Racial Equality, institutional racism “occurs when the policies and practices of an organisation result in different outcomes for people from different racial groups”. The term, if one accepts that it is appropriate for medical journals, does not mean that individual editors are racist. It does mean that the scientific, medical, and public-health priorities of the rich world are presented as the norm.”
- metrics of total publications and citations.Top 15 countries account for 82% of total publicationsAuthor with African institutional affiliation account for less than 1% of global output, and S. Africa has the highest output. The rest are “invisible”Consequence of trying to publish in “International” journal results in neglect of important local problems and solutions that are appropriate for local conditions.
- Consequences of publishing in “internatioanlly” indexed journals
- he New Invisible College, Caroline Wagner combines quantitative data and extensive interviews to map the emergence of global science networks and trace the dynamics driving their growth. She argues that the shift from big science to global networks creates unprecedented opportunities for developing countries to tap science's potential. Rather than squander resources in vain efforts to mimic the scientific establishments of the twentieth century, developing country governments can leverage networks by creating incentives for top-notch scientists to focus on research that addresses their concerns and by finding ways to tie knowledge to local problem solving. T
- The report concludes that web based academic publishing is producing a variety of novel information filters. These allow the researcher to make some sort of limited self-assessment with respect to the response to his/her work. However, this does not mean that these technologies and databases can also legitimately be used in research assessments. For this application, they need to adhere to a far stricter protocol of data quality and indicator reliability and validity. Most new tools do not (yet) comply with these more strict quality criteria.