Social Media Metrics and Scholarly Communication
•Descargar como PPTX, PDF•
1 recomendación•300 vistas
Tweet or Publish? Presentation at the 19th Nordic Workshop on Bibliomterics and Research Policy
Recomendados
Recomendados
Más contenido relacionado
Destacado
Destacado (7)
Similar a Social Media Metrics and Scholarly Communication
Similar a Social Media Metrics and Scholarly Communication (20)
Último
Último (20)
Social Media Metrics and Scholarly Communication
- 1. Timothy D. Bowman, Ph.D. Candidate | 19th Nordic Workshop on Bibliometrics and Research Policy
- 2. WHY INVESTIGATE SCHOLARLY ACTIVITY IN SOCIAL MEDIA ? - New technology allows for reassessment and reevaluation of academia (Baldwin, 1998) - Social media use provides insight into customs and traditions (Greenhow, 2009) - Social media use unveiling once invisible backstage activity (Priem, 2014) CRC.EBSI.UMONTREAL.CA
- 3. HAVE WE MOVED “BEYOND BIBLIOMETRICS”? - We’ve moved beyond simply measuring citations CRC.EBSI.UMONTREAL.CA (Cronin & Sugimoto, 2014). - New tools and data allow for new kinds of metrics measuring wide array of indicators (Cronin, 2014) - Electronic publishing magnifies a scholar’s awareness of own performance (Wouters, 2014) - Evaluation of bibliometric indicators by novices allows for wide use of various ad hoc indicators (Gingras, 2014)
- 4. WHY CONSIDER “ALTMETRICS” OR “INFLUMETRICS” OR SIMPLY “SOCIAL MEDIA METRICS”? - “Altmetrics” is the measure of scholarly communication and dissemination within social media contexts (Priem & Hemminger, 2010; Priem, Taraborelli, Groth & Neylon, 2010) - Perhaps a better term is Influmetrics (Rousseau & Ye, CRC.EBSI.UMONTREAL.CA 2013) or simply “social media metrics”? - Social media indicators may measure immediate assessment of academic impact and social impact (Thelwall, Haustein, Larivière & Sugimoto, 2013) - “Products,” not “publications” (Piwowar, 2013)
- 5. CRC.EBSI.UMONTREAL.CA DO SCHOLARS USE TWITTER? - 92% of Semantic Web scholars had Twitter account and rated it as favorite for spreading scientific information (Letierce, Passant, Decker, & Breslin, 2010) - Total of 367 scholars reported increasing acceptance for blogs and microblogs for consumption and dissemination of scientific information (Gruzd, Goertzen, & Mai, 2012) - Scholars’ tweets tend to share information about (a) professional discussions, (b) network with others, (c) offer help / request help, (d) call attention to other social media involvement, and (e) personal discussions, and (f) impression management (Veletsianos, 2012)
- 6. CRC.EBSI.UMONTREAL.CA DO SCHOLARS USE TWITTER? (CONT.) - 43% scholars at 2012 STI Conference using Twitter; it was used privately and professionally, to distribute professional information, and to improve visibility (Haustein et al., 2013) - 80% DH scholars ranked Twitter as relevant for consumption and 73% for dissemination of DH information (Bowman et al., 2013) - Differences by discipline found regarding the way scholars used Twitter (Holmberg & Thelwall, 2014)
- 7. CRC.EBSI.UMONTREAL.CA RESEARCH QUESTIONS 1. What differences exist between the tweeting behavior of scholars in the natural and social sciences? 2. What kind of relationships exists between tweeting and publication behavior? 3. How does Twitter affordance use differ across disciplines?
- 8. CRC.EBSI.UMONTREAL.CA WHAT DATA IS IN THIS SAMPLE? - 16,862 Associate, Assistant, and Full professors from webpages at 62 AAU-member universities - The faculty belonged to either Physics, Biology, Chemistry, Computer Science, Philosophy, English, Sociology, or Anthropology departments. - 60 of the 62 universities rank in the top 125 according to 2014 CWTS Leiden Ranking - Survey sent January and February 2014 with a response rate of 8.5% (1,910 responses) - Of these responders, 32% (615) reported having at least one Twitter account - 289,934 tweets of 585,879 from 445 accounts (391 scholars) were collected.
- 9. CRC.EBSI.UMONTREAL.CA HOW WAS THE DATA COLLECTED? - Twitter API, Local WoS Database, Manual cleaning of authors - Twitter: - tweets, # of tweets, followers, friends, retweets, created date - affordances: @mention, #hashtag, URLs, media, symbols, retweets -WoS - publications, citation averages
- 10. ALL 1,910 SURVEY RESPONDENTS :: HAVE TWITTER ACCOUNTS? 42.65% by ACADEMIC AGE 36.42% 38.89% 40.82% 24.96% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Less than 1 Year 1 to 3 Years 4 to 6 Years 6 to 9 Years 10 Years of More I'm not 38.10% 45.09% 38.27% 34.31% 35.75% 29.68% 26.58% 19.81% 16.34% 60% 50% 40% 30% 20% 10% 5.26% 2.38% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 26 to 30 years 31 to 35 years 36 to 40 years 41 to 45 years 46 to 50 years 51 to 55 years 56 to 60 years 61 to 65 years 66 to 70 years 71 to 75 years Over 75 years by AGE 33.33% 29.11% 40.38% 25.00% by ETHNICITY 29.11% 50.00% 28.00% 0% American Indian / Native American Asian Black / African American Hispanic / Latino White / Caucasian Pacific Islander Other 60% by DISCIPLINE 28.10% 27.52% 37.46% 36.90% 50.00% 20.71% 28.99% 23.64% 50% 40% 30% 20% 10% 0%
- 11. ONLY 391 SCHOLARS WITH TWITTER ACCOUNTS :: MEAN OF TWEETS PER DAY 1.06 0.53 1.96 by DEPARTMENT by GENDER 1.41 0.67 0.52 0.73 1.18 1.14 0.80 1.02 Other Female Male N=232 SD=2.3 N=122 SD=2.1 N=3 0.89 1.11 1.39 0.67 0.85 I'm Not 10 Years or More 7 to 9 Years 4 to 6 Years 1 to 3 Years Less than 1 Year by ACADEMIC AGE N=2 N=207 SD=2.4 N=53 SD=2.2 N=35 SD=2.6 N=39 SD=0.9 N=21 SD=1.1 by PROFESSIONAL TITLE 0.92 0.98 1.03 Professor Associate Professor Assistant Professor N=116 SD=2.1 N=116 SD=1.7 N=156 SD=2.9
- 12. BY DISCIPLINE :: RELATIONSHIP OF MEAN TWEETS PER DAY TO # OF ARTICLES 12 10 8 6 4 2 500 450 400 350 300 250 200 150 100 50 0 Anthropology Physics 0 2 4 6 8 0 0 5 10 15 25 20 15 10 5 0 Philosophy 0 5 10 15 20 25 60 50 40 30 20 10 0 Sociology 0 5 10 15 20 Articles N=40 N=30 N=66 Tweets per Day N=19
- 13. BY DISCIPLINE :: RELATIONSHIP OF MEAN TWEETS PER DAY TO ARTICLES (CONT.) R² = 0.0133 30 25 20 15 10 5 6 5 4 3 2 1 0 Chemistry N=20 English 0 5 10 15 20 Biology R² = 0.0291 45 40 35 30 25 20 15 10 5 0 N=40 0 2 4 6 8 10 R² = 0.0118 0 0 1 2 3 4 Tweets per Day Computer Science R² = 0.0026 70 60 50 40 30 20 10 0 0 5 10 15 20 Articles N=73 N=82
- 14. SCHOLARLY IMPACT? :: MEAN TWEETS PER DAY BY MEAN OF IAC AVERAGE 2.5 2 1.5 1 0.5 0 Anthropology N=66 N=82 0 5 10 15 25 20 15 10 5 0 English 0 5 10 15 20 7 6 5 4 3 2 1 0 Philosophy N=30 N=19 0 5 10 15 20 25 8 7 6 5 4 3 2 1 0 Sociology 0 5 10 15 20
- 15. SCHOLARLY IMPACT? :: MEAN TWEETS PER DAY BY SUM OF IAC AVERAGE Biology R² = 3E-05 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 N=40 N=20 0 2 4 6 8 10 Chemistry R² = 0.0126 8 7 6 5 4 3 2 1 0 0 1 2 3 4 Physics R² = 0.0089 18 16 N=73 N=61 14 12 10 8 6 4 2 0 0 2 4 6 8 Computer Science R² = 1E-05 16 14 12 10 8 6 4 2 0 0 5 10 15 20
- 16. MEAN PERCENTAGE OF TWEETS WITH AFFORDANCE PER PERSON BY DEPARTMENT AND MEAN OF REWTEETS BY COLLECTED TWEETS 7.44% 6.41% 8% 7% 6% 5% 4% 3% 2% 1% 0% HASHTAGS Anthropology Biology Chemistry Computer Science English Philosophy Physics Sociology 16.28% 20.06% 25% 20% 15% 10% 5% 0% MENTIONS Anthropology Biology Chemistry Computer Science English Philosophy Physics Sociology 1.16% 0.72% 1.12% 0.25% 1.69% 0.53% 1.09% 0.77% 2% 1% 0% URLs Anthropology Biology Chemistry Computer Science English Philosophy Physics Sociology 353 3291 3,500 3,000 2,500 2,000 1,500 1,000 500 0 RETWEETS Anthropology Biology Chemistry Computer Science English Philosophy Physics Sociology
- 17. CRC.EBSI.UMONTREAL.CA SUMMARY • As expected, the data reflected differences of those who reported having Twitter accounts based on academic age and actual age. • Of the 391 scholars (445 Twitter accounts) that were collected, the data did reflect differences in mean tweets per day based on gender, discipline, and academic age and title • Finally, it was found that the data reflected no strong relationships between mean tweets per day and publication output or • There was no real relationship between average citations and mean tweets per day (scholarly impact) • The data did reflect small differences in affordance use by discipline, especially the differences in retweets but theses differences in retweets are not an accurate representation of the retweets by the scholar
- 18. ONGOING WORK • Further analysis of retweets needed attempting to focus solely on retweets made by the scholars themselves • Using linguistic tools, the text of the 289,934 tweets will be used to compare terms used in tweets with article titles at the level of the scholar and discipline • A social network analysis will be completed reflecting the mentions used in tweets at the scholarly and discipline levels • A closer examination of the actual affordances (unique hashtags, unique URLs, unique mentions) used • A categorization of tweets as either personal or professional by Turkers • A general discussion on what these social media metrics are actually measuring including any correlations between social media use and publication activity CRC.EBSI.UMONTREAL.CA
- 19. CRC.EBSI.UMONTREAL.CA THANK YOU This work was partially funded by a grant by The Alfred P. Sloan foundation and a Canada Research Chair grant DO YOU HAVE ANY QUESTIONS?
- 20. REFERENCES Baldwin, R. G. (1998). Technology’s Impact on Faculty Life and Work. New Directions for Teaching and Learning, (76), 7–21. doi:10.1002/tl.7601 Bowman, T. D., Demarest, B., Weingart, S. B., Simpson, G. L., Lariviere, V., Thelwall, M., & Sugimoto, C. R. (2013). Mapping DH through heterogeneous communicative practices. In Digital Humanities 2013. Lincoln, NE. Cronin, B. (2014). Scholars and scripts, spoors and scores. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 3-22). Cambridge, Mass.: MIT Press. Cronin, B. & Sugimoto, C.R. (2014). Preface. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. vii). Cambridge, Mass.: MIT Press. Gingras, Y. (2014). Criteria for evaluating indicators. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 109-126). Cambridge, Mass.: MIT Press. Greenhow, C. (2009). Social scholarship: applying social networking technologies to research practices. Knowledge Quest, 37(4), 42–47. Retrieved from http://aasl.metapress.com/index/r282223126950757.pdf Gruzd, A., Goertzen, M., & Mai, P. (2012). Survey results highlights: Trends in scholarly communication and knowledge dissemination (p. 10). Retrieved from http://socialmedialab.ca/?p=4308 Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2013). Coverage and adoption of altmetrics sources in the bibliometric community. arXiv, 1–12. Digital Libraries. Retrieved from http://arxiv.org/abs/1304.7300 Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics. doi:10.1007/s11192-014- 1229-3 Understanding how Twitter is used to spread scientific messages. In Web Science Conference. Raleigh, NC. Moran, M., Seaman, J., & Tinti-Kane, H. (2011). Teaching, learning, and sharing: How today’s higher education faculty use social media. Piwowar, H. (2013). Altmetrics: Value all research products. Nature, 493(159). doi:10.1038/493159a Priem J., & Hemminger B.M. (2010) Scientometrics 2.0: Toward new metrics of scholarly impact on the social web. First Monday 15. Available: http://firstmonday.org/htbin/cgiwrap/bin /ojs/index.php/fm/article/view/2 874/257. Accessed 2011 December 7. Priem, J., Taraborelli, D., Groth, P., Neylon, C. Alt-metrics: a manifesto. 2010. Available from http://altmetrics.org/manifesto/ Priem, J. (2014). Altmetrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 263–288). Cambridge, Mass.: MIT Press. Rousseau, R., & Ye, F. (2013). A multi-metric approach for research evaluation. Chinese Science Bulletin, 58(3290), 1–7. doi:10.1007/s11434-013-5939-3 Thelwall M., Haustein S., Larivière V., Sugimoto, C.R. (2013) Do Altmetrics Work? Twitter and Ten Other Social Web Services. PLoS ONE 8(5): e64841. doi:10.1371/journal.pone.0064841 Veletsianos, G. (2012). Higher education scholars’ participation and practices on Twitter. Journal of Computer Assisted Learning, 28(4), 336–349. doi:10.1111/j.1365-2729.2011.00449.x Wouters, P. (2014). The citation: From culture to infrastructure. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 47–66). Cambridge, Mass.: MIT Press.
- 21. APPENDIX: UNIVERSITY DISTRIBUTION Yale University Washington University in St. Louis Vanderbilt University University of Washington University of Virginia University of Toronto University of Southern California University of Rochester University of Pittsburgh University of Pennsylvania University of Oregon University of Missouri-Columbia University of Minnesota University of Michigan University of Maryland University of Illinois at Urbana-Champaign (1908) University of Florida University of Colorado Boulder University of California, Santa Barbara University of California, San Diego University of California, Los Angeles University of California, Irvine University of California, Davis University of California, Berkeley University of Arizona University at Buffalo, The State University of New York (1989) Tulane University The University of Wisconsin-Madison The University of Texas at Austin The University of North Carolina at Chapel Hill The University of Kansas The University of Iowa (1909) The University of Chicago The Pennsylvania State University (1958) The Ohio State University (1916) Texas A&M University Stony Brook University-State University of New York (2001) Stanford University (1900) Rutgers, The State University of New Jersey (1989) Rice University (1985) Purdue University (1958) Princeton University (1900) Northwestern New York University MIT Michigan State University (1964) McGill Johns Hopkins Iowa State Indiana University Harvard Georgia Institute of Technology Emory University Duke University Cornell Columbia University Case Western Reserve University Carnegie Mellon University California Institute of Technology Brown University Brandeis University Boston University University of Maryland (3.32%) University of Wisconsin-Madison (4.85%) Indiana University (4.08%) 0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0%
- 22. APPENDIX: 62 AAU-MEMBER UNIVERSITIES CRC.EBSI.UMONTREAL.CA Boston University, Brandeis University, Brown University, California Institute of Technology, Carnegie Mellon University, Case Western Reserve University, Columbia University, Cornell, Duke University, Emory University, Georgia Institute of Technology, Harvard, Indiana University, Iowa State, Johns Hopkins, McGill, Michigan State University, MIT, New York University, Northwestern, Princeton University, Purdue University, Rice University, Rutgers, The State University of New Jersey, Stanford University, Stony Brook University-State University of New York, Texas A&M University, The Ohio State University, The Pennsylvania State University, The University of Chicago, Tulane University, University at Buffalo, The State University of New York, University of Arizona, University of California, Berkeley, University of California, Davis, University of California, Irvine, University of California, Los Angeles, University of California, San Diego, and University of California, Santa Barbara ,The University of Iowa, The University of Kansas, The University of North Carolina at Chapel Hill, The University of Texas at Austin, The University of Wisconsin-Madison, University of Colorado Boulder, University of Florida, University of Illinois at Urbana- Champaign, University of Maryland, University of Michigan, University of Minnesota, University of Missouri- Columbia, University of Oregon, University of Pennsylvania, University of Pittsburgh, University of Rochester, University of Southern California, University of Toronto, University of Virginia, University of Washington, Vanderbilt University, Washington University in St. Louis, Yale University
Notas del editor
- We are living in what many view as the age of “big data”, thus some of us want to understand what the flood of data is telling us about the way we present ourselves online, how we disseminate and consume information, and how we acquire and make use of various types of capital in these contexts. If we look at recent history beginning post-1996 and the advent of the graphical web, some interested in studying science and scholarly activity took a closer look at new technologies because they could see that it might allow for a reexamination of academia. As the web evolved and scholars more frequently contributed to listservs, blogs, data repositories, microblogs, social network sites, and other types of social media, it became clear that the traces of scholarly activities in these new environments were valuable in that they could provide insight into the structure and norms of the academy and to previously invisible activities of scholars. The construction of these various social media tools and the foundation (like programming languages and database systems) these tools are built upon allow us to more easily collect, store, retrieve, and evaluate the large amounts of data produced within these contexts. These are just a few of the reasons that there has been an influx of research on scholarly activity in social media
- Technological innovations and the evolution of these online social media tools has led many to look beyond the traditional metrics in an attempt to provide a better measure of scholarly impact. The tools, platforms, and changing access to data has allowed for the identification of various metrics attempting to detect and measure novel types of indicators In addition to the these tools, platforms and data access, the shift toward electronic publishing has enhanced the ability of scholars to measure their own performance and the performance of their colleagues; this phenomenon can create a desire for additional forms of metrics and indicators in order to further distinguish oneself from others. Also, the ability to evaluate bibliometric indicators has moved from the domain to experts to the realm of novices who are using online tools employing hidden algorithms to report on their own academic impact
- This influx of metrics used to evaluate online contexts has led some to label them as altmetrics, a concept defined as “the measure of scholarly communication and dissemination within social media contexts” It seems that altmetrics is a term that no longer serves to adequately explain what it is that we are measuring because these indicators are not measuring phenomenon alternative to something else such as citations or journal impact, but instead measure the traces of activity in the context of social media and other tools that were once either unavailable or invisible. Instead I think of these as simply social media metrics. One of the appeals of the measure of social media indicators is that it might provide immediate insight into immediate academic and social impact; this has been compared to citations that both take a longer period of time to accumulate and only measure those who cite Another reason social media metrics are important today is that organizations such as the National Science Foundation in the U.S. are stipulating that scholars submit a list of their “products,” not just a list of relevant “publications”, when applying for funding. This indicates that a scholar’s publications are no longer enough to determine productivity, impact and overall value. These are just some of the reasons why social media metrics are an important and interesting area of research
- What do we know about scholar’s use of social media in contexts such as Twitter? Work examining this phenomena has focused on a variety of disciplines including one study that surveyed 61 Semantic Web scholars finding that 92% had a Twitter account and rated it as their favorite service to spread scientific information In another work, a survey of 367 primarily social science scholars reported an increasing acceptance for blogs and microblogs as trustworthy and legitimate sources for the consumption and dissemination of scientific information In a content analysis of scholarly tweets it was discovered that scholars tended to (a) share information about their professional practice, (b) attempt to network, (c) offer help and request help, (d) call attention to other social media involvement, (e) engage in personal discussions, and (f) iin impression management
- Another survey of 71 scholars at the 2012 STI Conference found that 43% reported using Twitter and that they used it privately, professionally, to distribute professional information, and to improve their visibility Over 200 Digital Humanities scholars surveyed with 80% reporting Twitter as relevant for consumption of DH and 73% reported it as relevant for dissemination of DH information (Bowman et al., 2013) Finally, scholars from 10 different disciplines (astrophysics, biochemistry, digital humanities, economics, history of science, cheminformatics, cognitive science, drug discovery, social network analysis, and sociology) were analyzed and it was found that there were differences in the way they used Twitter (Holmberg & Thelwall, 2014)
- For this work I was interested in three exploratory questions: What differences exist exist across these humanities and natural science disciplines? What kind of relationships exist between tweeting and publication behavior? And finally, how does Twitter affordance use differ across disciplines? Here I define an affordance as a relation between an object or environment (in this case a tweet) and an organism (human) that affords the opportunity for that organism to perform an action (such as categorizing a tweet with #, referencing someone using an @ symbol, or providing additional information with a URL).
- Information of 16,862 Associate, Assistant, and Full professors from eight departmental webpages from 62 universities belonging to the Association of American Universities was harvested between September 2013 and January 2014. The faculty belonged to either Physics, Biology, Chemistry, Computer Science, Philosophy, English, Sociology, or Anthropology. According to the 2014 CWTS Leiden Ranking website that lists universities by scholarly impact, 60 of the 62 universities included in this sample rank in the top 125 of this ranking http://www.leidenranking.com/ranking/2014 A survey was sent to all of the faculty between January and February 2014 with a response rate of 8.5% (1,910 responses). A Of these, 32% (615) reported having a Twitter account Of the 615 scholars who reported having a Twitter account, 289,934 tweets of 585,879 from 445 accounts were collected. Note that the Twitter API restricts the collection of tweets the most recent 3,200 tweets per account. The missing 170 accounts were either private or could not be found. There were 41 scholars with 2 accounts, 11 scholars with three account, and 1 scholar with 5 --- leaving 391 scholars
- Twitter API was used in combination with PHP and MySQL to collect and store the tweets from the 445 scholar accounts Local WoS SQL Server database containing data provided by Thomson Reuters was searched with SQL to match author names with scholar names from the collected Twitter account 321,033 publication records retrieved from initial WoS search, author name disambiguation resulted in 7,734 articles across 391 authors When a media
- Using the data from the survey and comparing those who self-reported having a Twitter account with those who did not, we see that when comparing respondents by academic age there is a big drop off after 9 years. There is no clear differences by ethnicity and we see that there is definitely a drop off by age, with those 31 to 35 years old reporting the most uses.
- IAC Average is the average of citation impact of the person
- Again, an affordance is defined as a relation between an object (tweet), or an environment, and an organism (human) that affords the opportunity for that organism to perform an action (categorize tweet with hashtag). I want to give each researcher the same weight, so I want to look at the affordance use per person. The overall mean is XXX. This is what we expect the scholar to do… we see that… The problem with this analysis is that there is what I consider a bug in the Twitter API; the giant number in physics reflected here is from the fact that one physicst retweeted the Ellen Degeneris oscar selfie tweet and Twitter counts the original tweets ‘retweet_count’ when it delivers the tweet count for each user. It seems to me it should provide the retweet count of the current Twitter user, but this is not the case.
- In response to research question one which asked about relationship amongst scholarly Twitter use, there were differences found between scholars reporting having a Twitter account with a difference by academic age and actual age. It was also found that there was differences in mean tweets per day by discipline, academic age, academic title, and gender with the caveat that there were high standard deviations. With regards to question two thinking about any coorrelations between Twitter use and publication activity and Twitter use and scholarly impact, it was found that there were no real relationaships between these two activities. Scholars who tweeted frequently were not high publishers and tweet behavior did not seem to have an impact on scholarly impact Finally, there were small differences found between departments regarding affordance use in Twitter, but further analysis is needed