Broadening the Definition of Altmetrics - 5am conference - David Sommer
•Descargar como PPTX, PDF•
1 recomendación•338 vistas
In this presentation I discuss how researchers are using offline, private channels to communicate their research in addition to online, public channels. I explore the axes of communication, digital visibility and provide examples of how researchers use Kudos to share in closed, private channels and check the effectiveness of their dissemination. Altemtrics are just the tip of the iceberg maybe we have undervalued the data we are building up about offline and closed channel coms. The data set we are building with the 250,000 researchers using Kudos helps us provide guidance and recommendations to ensure researchers are disseminating effectively and not going unrewarded.
Recomendados
Recomendados
Más contenido relacionado
Similar a Broadening the Definition of Altmetrics - 5am conference - David Sommer
Similar a Broadening the Definition of Altmetrics - 5am conference - David Sommer (20)
Más de Kudos
Más de Kudos (20)
Último
Último (20)
Broadening the Definition of Altmetrics - 5am conference - David Sommer
- 1. Broadening the definition of altmetrics David Sommer Co-Founder and Product Director, Kudos 5:am conference - 27th September 2018 @DavidLSommer @growkudos www.growkudos.com
- 2. 22 Under the radar @DavidLSommer @growkudos www.growkudos.com
- 3. 33 Axes of communication @DavidLSommer @growkudos www.growkudos.com Persistent Ephemeral Private Public
- 4. 44 Axes of communication @DavidLSommer @growkudos www.growkudos.com Persistent Ephemeral Private Public
- 5. 0% 20% 40% 60% 80% 100% Conferences / meetings Academic networking / profile sites (e.g. ResearchGate, Mendeley, Academia.edu, Google Scholar, ORCID) Conversations with colleagues Institutional websites / repositories Email Social networking sites (e.g. LinkedIn, Twitter, Facebook) Your own blog / website Subject-based websites / repositories (e.g. arXiv, SSRN) Posts on other blogs / websites Discussion lists Multimedia sharing sites (e.g. Slideshare, YouTube) Communication channels used In which of the following ways do you currently create awareness of or share materials relating to your work? (n = 2,826) @DavidLSommer @growkudos www.growkudos.com
- 6. 0% 20% 40% 60% 80% 100% Conferences / meetings Academic networking / profile sites (e.g. ResearchGate, Mendeley, Academia.edu, Google Scholar, ORCID) Conversations with colleagues Institutional websites / repositories Email Social networking sites (e.g. LinkedIn, Twitter, Facebook) Your own blog / website Subject-based websites / repositories (e.g. arXiv, SSRN) Posts on other blogs / websites Discussion lists Multimedia sharing sites (e.g. Slideshare, YouTube) Digitally visible Offline / private Communication channels used In which of the following ways do you currently create awareness of or share materials relating to your work? (n = 2,826) @DavidLSommer @growkudos www.growkudos.com
- 7. 77 Kudos Share Labels - tracking multiple channels @DavidLSommer @growkudos www.growkudos.com
- 8. 88 Tracking offline communications - posters @DavidLSommer @growkudos www.growkudos.com
- 9. 99 Closed channel communications - Sharable-PDF @DavidLSommer @growkudos www.growkudos.com
- 11. 1111 Kudos for Research Groups @DavidLSommer @growkudos www.growkudos.com Outputs Evaluation Collaboration Audiences Timelines Articles & Books Any research output or research object Articles & books metrics Output metrics & dissemination effectiveness Individuals Research groups, labs, departments One size fits all Tailored for different audience types Post-publication Grant planning, management & reporting
- 12. 1212 Upstream Research Project @DavidLSommer @growkudos www.growkudos.com The Upstream Project Exploring opportunities for publishers to add services and value for researchers upstream of the point of publication Expanding publishers’ footprint in the research lifecycle • What other kinds of object and output are researchers using to make their work public? • What effect will ‘Plan S’ and other funder initiatives have on (a) the formats in which researchers ’publish’? (b) the mechanisms by which they are evaluated? • How will these affect publishers? Where is there potential for publishers’ skills and expertise to continue to be of value? • What kinds of services and systems are researchers becoming accustomed to paying for and who are they buying them from?
- 13. 1313 Upstream Research Project @DavidLSommer @growkudos www.growkudos.com The Upstream Project • Cross-publisher research project during Q4 2018 • Up to 15 supporting partners • International researcher survey • Teleinterviews / focus groups • Customized reports in addition to aggregate strategic analysis • Consultancy to support you in considering how to apply the findings to your organization’s strategy
- 14. 1414 Axes of communication @DavidLSommer @growkudos www.growkudos.com Persistent Ephemeral Private Public
- 15. 1515 Broadening the definition of Altmetrics @DavidLSommer @growkudos www.growkudos.com • Altmetrics are only as good as the data on which they are based • Online, open channels are just the tip of the iceberg • Have we undervalued offline and closed channels? How do we broaden the datasets that shape Altmetrics to include these below-the-waterline channels? • Providing the complete picture is critical to help researchers take effective actions to disseminate their work and to ensure their efforts don’t go unrewarded
- 16. Broadening the definition of altmetrics David Sommer Co-Founder and Product Director, Kudos 5:am conference - 27th September 2018 @DavidLSommer @growkudos www.growkudos.com
Notas del editor
- When we talk about altmetrics, and sharing and attention, many people think about online mentions/ online media. They think about twitter, facebook and LinkedIn. But there is so much more! There is a huge amount of offline sharing, closed channel sharing, under the radar sharing happening every day…
- It is useful to think of the 2 axes of communication. There is an axis of visibility ranging from public or open through to private or closed. This describes who is able to access. And there is an access of persistency - from persistent, often online through to Ephemeral, or events - often offline, real world interactions. This describes how people can access. Mostly when we talk about altmetrics we are thinking about public and persistent channels. For example Twitter, blogs and news sites. But there is so much more! People also share online but in but in closed private communication channels for example: Sending email to colleagues Sharing on closed LinkedIn Groups Posting private FB groups And then we have people also share at real world events, offline channels to reach a specific audience, fleeting ephemeral occasions: Attending a poster session Speaking at a conference Producing handouts And people may share more publicly at ephemeral events to reach a broader audience: Attending science fairs Leafleting health centres Giving a museum talk Giving media interviews
- There is still a huge level private, offline under the radar level of sharing that is not being tracked. Offline sharing and sharing in closed channels. And it is these 3 quadrants that we are really interested in. This COULD be really pivotal to building engagement with the work as well as the impact that work is having… …and if it ISN’T pivotal, researchers and funders need to know that as a lot of time and money being spent here..
- This data is from a project we undertook in 2016. We did an online survey, which was completed by about 3,000 researchers, and we asked them among other things about what kinds of things they would do to create awareness of, or share materials relating to, their work – so we were asking them about dissemination and engagement, but without constraining their thinking by using those terms! You can see that the most common way in which people were disseminating their work was via conferences, but it’s interesting that academic networking and profile sites (such as ResearchGate) are just a couple of percentage points ‘behind’ in terms of the proportion of respondents that are using those sites as channels for circulating their work, and I’ll talk more about those sites in a moment. But meanwhile, here, we see: conversations with colleagues, institutional websites or repositories, email (note that email discussion lists is separate, and further down), social media, blogs and personal websites, subject-based sites and repositories, posts on other people’s blogs or websites, and multimedia sharing sites. And I think there are two interesting lenses through which to consider this data further. Firstly, the audiences that will typically be reached by each of these channels, or that the researcher might be targeting with each of these channels, and Secondly, the digital visibility of each of these channels and I’ll explain what I mean by that in a moment. And what I’m taking away from this is just how few of the most commonly used channels do have potential for reaching a broader audience, or to put that the other way around, the channels that have the potential for broader audiences are the least used.
- If I switched the colours to show those forms of dissemination that are “digitally visible” - things that you could track down in retrospect, things you could measure using services like altmetrics, things that are easily logged and counted in an automated way – those are in green; Meanwhile in red, we have the types of dissemination that people do that are really under the radar; the things that don’t automatically generate a nice record of their existence, or the engagement with them – so if an academic sends an email to a policy maker, it’s only possible to capture the evidence or the results of that if you have access to that academic’s mail client, and if the conversation is tagged such that you can track it and interpret its outcome. So, a lot of dissemination is happening in private, closed channels.
- So, let me give you a few examples to illustrate what I’m talking about… At Kudos, we have always been aware that researchers will want to communicate in a variety of ways - open and closed channels, online and offline. Our aim is to provide tools to allow researchers to communicate through ANY channel and then measure the results so they know which actions are effective and which are not. As an example of how we support that, we recently launched Share Labels. Allowing users to create a trackable link that can be shared in different ways. Users can label their shares and then track the most effective channels. In this example you can see a user has shared links for use in handouts, email groups and posters. One even uses it their email signature file.
- But how can you track offline channel usage? Aren’t we reliant on trackable links, clicks resulting in events detectable on a web server? Well yes and no! Let’s take an example. Let’s look at a poster. Ladies and gentlemen, you can’t get much more offline that a poster! It’s a piece of paper! By creating a QR code for the trackable link, researchers at a poster session can just point their phone at the poster and link through to more information, in this case a plain language summary on Kudos with additional information and a direct link to the publication. You can try it now and point your phones at this slide! The researcher is then able to measure the attention that specific poster at that specific event generated. That is extremely useful data.
- What about tracking activities in closed channels? When we started Kudos, we anticipated lots more use of social media. We knew it was a nascent thing but anticipated it as being something that would grow faster. We see a lot of illegal sharing activity through ResearchGate and other platforms - researchers uploading full text PDFs. That inspired us to develop the Sharable-PDF, enabling researchers to create a PDF containing their PLS created on Kudos and a trackable link. They can post these on any channel, e.g. RG, Academia.edu, Mendeley and then understand which channels are most effective. When we look at the data, it is really interesting - 61% of all sharing on Kudos is S-PDF where we make it available, with social media making up less than 10%. We see continuing and unabated use of offline and private channels, and the more trackable we make sharing in those channels, the more we are seeing people using them and them being effective.
- We allow researchers to add links on their Kudos publication profile pages to related resources. This is really valuable data as it helps us understand what types of work and outputs researchers want to communicate. Typically we see researchers adding links to data sets, videos, project sites, interviews. All of this points to unmet demand. Researchers want to help more people find different types of outputs, related to their work and they want to do this in a trackable way. We were almost taken by surprise of the quality, breadth and uniqueness of the data set we are building at Kudos as we help researches track the effectiveness of their communications.
- I’m excited now to unveil some new areas of development for Kudos. Next year we will launch Kudos for Research Groups Expanding our current model and moving upstream in the research cycle. Outputs, Evaluation, Collaboration, Audiences, Timelines This will give a vastly expanded data set on what researchers are communicating, how they are communicating and how effective they are being. Analysis of the formats, channels and results will enable us to make effectives recommendations and give them guidance. We think this is going to be very exciting!
- As well as developing our product to capture more data about this, about to launch our next big research project. What other formats are gaining traction and what is the use of alternative formats and evolving funder requirements.
- If you’d like to learn more - get in touch or let’s catch up later.
- Back to our axes of communication… We need to remember that researchers are not communicating in isolation. There are overlapping responsibilities, different players with different activities all in pursuit of the same goal The researchers, the lab, the institution, the publisher, the funder. It is about efficiency and effectiveness - to maximize both of these we need to get better at surfacing what works and dedupe effort and measuring it and evaluate it so we get better shape our strategies
- Are we at a point where metrics are used to evaluate research and researchers, are we at an inflection point, and if not what is going to provide that? We are building a lot of data that we think will be useful. Altmetrics are only as good as the data on which they are based. There are some well established AM providers who have been very successful at building of data sets looking at online discussion of work. But Altmetrics is just the tip of the iceberg! Maybe we (Kudos and the community) have undervalued the data we are building up about offline and closed channel coms. With Kudos now at over 250k user, we have a statistically signifies data set. It shows by and large people are sharing through private channels and offline, so having realised that we are really pursuing growth of that data set. The more people are requiring dissemination plans and directing researchers away from the old ways of evaluation, the more important it is to have this data set and use it to help people undertake effective dissemination activities and to help ensure that their efforts aren’t going unrewarded Thank you!