Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Learning Analytics: Opportunities & Dilemmas

2.307 visualizaciones

Publicado el

Presentation for an e/merge Africa Webinar on 4 May 2017 - http://emergeafrica.net/4-may-learning-analytics-opportunities-and-dilemmas/

Publicado en: Educación
  • Inicia sesión para ver los comentarios

Learning Analytics: Opportunities & Dilemmas

  1. 1. Webinar 4 May 2017 – e/merge Africa http://emergeafrica.net/4-may-learning-analytics-opportunities-and-dilemmas/ Paul Prinsloo (University of South Africa, Unisa) Learning analytics: Opportunities and dilemmas
  2. 2. Acknowledgements I do not own the copyright of any of the images in this presentation. I acknowledge the original copyright and licensing regime of every image used. This presentation (excluding the images) is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License 2
  3. 3. Confession(s) I am a bit embarrassed with the fact that this presentation is so ‘text- heavy’ as I normally enjoy designing with less text and rely more on visual elements to convey meaning. The reason for the amount of text in this presentation is to allow this presentation to stand on its own as a possible resource for whoever may find the content or parts of the content usable/informative Image credit: https://www.flickr.com/photos/sheila_sund/23835028019 3
  4. 4. 1. Is bigger data better data? What evidence can such data provide and what are some of the shortcomings? 2. What are some of the ethical dilemmas involved in uses of student data? 3. Is the hype over learning analytics based on idealism rather than reality? How can we move beyond the hype of learning analytics? 4. And… In preparation for this presentation, the organisers posed the following questions: 4
  5. 5. 4. Are lessons learnt from the Global North about uses of learning analytics a useful starting point for educators in African higher education? What do we adopt and where do we adapt? Of these questions, the fourth one fascinates me and exposes some of the dilemmas but also unique opportunities for learning analytics in the Global South/developing world context… 5
  6. 6. • We, in the Global South, see ‘data’ differently – and that we are very aware of how certain variables and characteristics were selected and used as basis for dehumanising many identified individuals and groups • We are very aware of the challenge of dealing with the inter-generational legacy of classification systems where the potential for advancement and access to resources were determined based on assumptions and stereotypes of race, gender and culture • We ask how learning analytics functions in environments where online access and participation are unevenly distributed and where login, download and participation data say more of the legacy of apartheid and colonialism, than of students’ potential, motivation or aspirations • We wonder how to engage with the number of logins and downloads, and the number and quality of online engagements where evidence suggests that these numbers correlate with socio-economic and prior learning circumstances, and not, per se, with student potential, motivation or aspirations Considering our locations and histories (past and present)… 6
  7. 7. • Our geopolitical and institutional contexts and (in)efficiencies impact more on individual performance than individual students’ agency • Higher education institutions in the Global South often (mostly?) lack the resources and infrastructure to establish and maintain integrated systems, processes and policies to enable appropriate and ethical data collection, analysis and use • Many (most?) institutions in the Global South do not have capable and well- resourced human resources (e.g. data scientists) to optimise the potential of learning analytics or have the resources to respond ethically and appropriately to identified needs • The gender and race of those who code and develop algorithms have internalised the assumptions and beliefs of the North Atlantic and perpetuate these assumptions and beliefs through code 7 Considering our locations and histories (past and present)… (cont.)
  8. 8. African higher education institutions are for sale to the highest bidder – those commercial vendors and the apostles of Silicon Valley who regard a free IPad and a dumbed-down version of the internet as a fair exchange for access to students’ data? And finally, considering our locations and histories (past and present), is it possible that... Image credit: https://pixabay.com/en/photos/for%20sale/ 8
  9. 9. Image credits: https://pixabay.com/en/photos/spaghetti/; https://www.flickr.com/photos/stevendepolo/30801087140; http://www.offutt.af.mil/News/Art/igphoto/2000395404/; https://www.flickr.com/photos/stevendepolo/4312332671 https://www.flickr.com/photos/donhomer/12862086683; https://www.flickr.com/photos/tim_norris/1216942456 Making sense of the collection, analysis and use of learning analytics is [increasingly] like… 9
  10. 10. Overview of the presentation • A short introduction to learning analytics • Mapping the socio-technical imaginary of evidence, (student) data and the need for more data • How do we engage with the potential, dangers and challenges in the collection, analysis and use of data when…? • Disclosing my own position regarding technology [data] • Mapping student data – what data do we have, don’t have but can collect, and what data we may never have… • Pointers for considering the potential, limitations and challenges in learning analytics • What about the ethics of [not] collecting, analysing and using student data? • (In)conclusions 10
  11. 11. “Learning analytics refers to the measurement, collection, analysis and reporting of data about the progress of learners and the contexts in which learning takes place” (https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education) 11
  12. 12. Page credit: https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education 12
  13. 13. Page credit: http://ignatiawebs.blogspot.co.za/2017/04/using-learninganalytics-to-inform.html 13
  14. 14. “Learning analytics refers to the measurement, collection, analysis and reporting of data about the progress of learners and the contexts in which learning takes place” (https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education) It is about ‘learning’ and informed about our beliefs about what constitutes learning… What do we measure? How do we measure? When do we measure? What don’t we measure? Where do we collect, what, how often, for what purpose and who does the collection? To whom do we report? What then? Who will act? What if we cannot act? What if students don’t act? Who analyses the data, using what, what skills are required, who verifies the analysis? How do we define ‘progress’? How do we involve students in making sense of the data, of their progress, of their journeys? Do we assume that all their learning takes place on the LMS? What do we assume about their logins, their downloads, their clicks? 14
  15. 15. … has become saturated with data – ranging from automatically collected, analysed and used, purposefully collected, analysed and used and volunteered on social media; in exchange for (perceived) benefits despite concerns about privacy, the uncertainty of how the data will be used downstream and combined with other sources of data; and in the context where our trust in the collectors of data is often misplaced, irrational or wishful thinking (See Kitchen, 2013, pp. 262-263) How do we talk about the collection, analysis and use of student data in a world that… Image credit: https://commons.wikimedia.org/wiki/File:Big_Hand_-_geograph.org.uk_-_644552.jpg 15
  16. 16. Page credit: http://www.govtech.com/education/How-Data-Driven-Learning-is-Changing-Education.html “Success here derives from access to data, or big data as it’s sometimes called.” “Data matters.” 16
  17. 17. Page credit: http://www.baltimoresun.com/news/maryland/education/bs-md-college-analytics-20160611-story.html 17
  18. 18. Imagecredit:https://www.flickr.com/photos/haydnseek/2534088367 Student data is seen as the ‘new black’, as a resource to be mined, scraped & abused/misused Image credit: http://fpif.org/wp-content/uploads/2013/01/great-oil-swindle-peak-oil-world- energy-outlook.jpg 18
  19. 19. Page credit: http://insider.foxnews.com/2016/01/31/oklahoma-college-forcing-students-wear-fitbits 19
  20. 20. Page credit: https://inews.co.uk/essentials/news/education/university-monitor-student-social-media-gauge-well/ 20
  21. 21. Higher education is mesmerized/seduced by the potential of the collection, analysis and use of student data Image credit: http://en.wikipedia.org/wiki/Medusa (Student) data as Medusa – techno-solutionism in action 21
  22. 22. Page credit: http://www.ft.com/cms/s/2/634624c6-312b-11e5-91ac-a5e17d9b4cff.html#slide0 22
  23. 23. Page credit: http://hechingerreport.org/students-worry-education-technology-might-predict-failure-chance-succeed/ 23
  24. 24. How do we talk about the collection, analysis and use of student data when… • We have access to more data (often/increasingly real-time, granular and very personal) of [some] students while a lot of our data about [most?] students are but proxies for their circumstances [e.g. addresses] and potential [e.g. prior learning experiences] • We collect, measure, analyse and use student data in isolation from issues surrounding epistemological access, institutional operational (in)efficiencies, pedagogical approaches, faculty and support staff (in)experience and (dis)engagement • Current research suggests that many [most?] of the variables impacting on students’ chances of success (i.e. socio-economic circumstances, political and economic (in)stability] fall outside the locus of control of students, faculty and the institution 24
  25. 25. How do we talk about the collection, analysis and use of student data when… (cont.) • We already have a lot of data that is scattered all over the institution, in a variety of formats, ranging in quality, collected for a variety of purposes [with and without ethical clearance], accessed by a variety of staff for a variety of purposes, and combined with other sources of data collected for other purposes and used for (un)related needs • We value quantitative data and ignore qualitative data; we celebrate big data and ignore small data; and we value surface-level trends rather than thick descriptions. We are obsessed with ‘what?’ and not ‘why?’ • Students don’t know that/how we collect, analyse and use their data and don’t have access to their digital dossiers to question, verify, add context and/or opt out • The effectiveness and appropriateness of our responses to our analyses depend on our own locus of control, our resources, our understanding of the data and a political will [or lack of] 25
  26. 26. Imagecredit:https://pixabay.com/en/binary-code-man-display-dummy-face-1327512/ How do we collect, analyse and use student data while recognising that their data are not indicators of their potential, merit or even necessarily engagement but the results of the inter-generational impact of the skewed allocation of value and resources based on race, gender and culture? 26
  27. 27. A good place to start… 27
  28. 28. “Science [and technology] increases human power – and magnifies the flaws in human nature. It enables us to live longer and have higher living standards than in the past. At the same time it allows us to wreak destruction – on each other and the Earth – on a larger scale than ever before” John Gray – Straw dogs (2002, p. xiii-xiv) 28
  29. 29. “… ‘educational technology’ needs to be understood as a knot of social, political, economic and cultural agendas that are riddled with complications, contradictions and conflicts”… (Selwyn, 2014, p. 6) If we accept that …what are the implications for the collection, analysis and use of student data? 29
  30. 30. Data always mattered… • The origins of the word "census" can be traced back to Rome from the Latin word censere - "to estimate.“ Used to determine taxes, counted ‘citizens’ (and defined citizens); used to determine citizens’ and “others’”; used to allocate rights and privileges; and used to assess the number of arms-bearing men (sic) in preparation for war • India, 300 BCE – Census during the reign of Emperor Chandragupta Maurya • China, ACE 2 – the Han Dynasty “one of the world's earliest preserved censuses” • Bible – a number of censuses – to determine taxes, to count the number of ‘foreigners’ living among Israelites 30 Source: https://en.wikipedia.org/wiki/Census
  31. 31. Data = Power Throughout the ages …those who had the power, decided on what data was needed, for what purposes, how the data was defined, what the data meant, who had access to the data, and who would verify the correctness and the meanings of the data Data has always been used to • Control • Solve (or perpetuate) societal problems/horrors • To allocate (or withhold) resources/support • To safeguard the survival of those who collected the data; and ensure adherence to the assumed rules and conventions determined by those who collected the data 31
  32. 32. Examples include… Image credit: http://commons.wikimedia.org/wiki/File:Penetentiary_Panopticon_Plan.jpg ‘Panopticon’ Jeremy Bentham, 1873 Greek mythology – Argus Panoptes – A giant with 100 eyes 32
  33. 33. Image source: https://www.mpiwg-berlin.mpg.de/en/news/features/feature14 Copyright could not be established • 1749 Jacques Francois Gaullauté proposed “le serre-papiers” – The Paperholder – to King Louis the 15th • One of the first attempts to articulate a new technology of power – one based on traces and archives (Chamayou, n.d) • The stored documents comprised individual reports on each and every citizen of Paris The technology will allow the sovereign “…to know every inch of the city as well as his own house, he will know more about ordinary citizens than their own neighbors and the people who see them everyday (…) in their mass, copies of these certificates will provide him with an absolute faithful image of the city” (Chamayou, n.d) The Paperholder – “le serre papiers” (1749) 33
  34. 34. Image credit: http://iconicphotos.wordpress.com/2010/07/29/the- great-ivy-league-photo-scandal/ “… a person’s body, measured and analysed, could tell much about intelligence, moral worth, and probably future achievement… The data accumulated… will eventually lead on to proposals to ‘control and limit the production of inferior and useless organisms’” (Rosenbaum, 1995) The great Ivy League photo scandal 1940- 1970 34
  35. 35. Imagecredit:https://pixabay.com/en/binary-code-man-display-dummy-face-1327512/ Image credit: http://za.geoview.info/apartheid_museum_entrance,83879989p The collection, analysis and use of data are political acts and serve declared and hidden assumptions about the purpose of higher education and the masters it serves (Apple, 2004, 2007; Grimmelman, 2013; Watters, 2015) 35
  36. 36. Revisiting (our beliefs re)Data Page credit: https://randomdatablog.com/2017/04/17/yes-data-can-lie/ “Data do lie on occasion. They can lie for a whole bunch of reasons, from the simple to the complex. The lies can begin at point of collection and continue on through aggregation and analysis.” 36
  37. 37. “Data are like Play-Doh and can take all sorts of shapes and dimensions. It can be worked and reworked for endless variety. But, it can only stretch so far before it breaks and becomes separate pieces. This is what happens with data when you stretch the definition and structure too far, original meaning is lost and the provenance is broken. Small pieces can be lost during this shaping, or blended with other “colours” creating something new, but increasingly more abstract than the original data.” (Tod Massa) Page credit: https://randomdatablog.com/2017/04/17/yes-data-can-lie/ 37
  38. 38. What student data do we currently have, where is it stored, in what format, what is the quality of this data, who has access to the data under what circumstances, and for what purposes? 38
  39. 39. Data from student inquiries prior to (application for) registration - telephonic, in- person, via email, via social media Data gathered during the registration process – location of registration, demographic/ socio- economic data, prior learning, uploaded (or not) documents, financial status LMS data – how long after registration did they log in, what resources did they download, how often, who did they interact with (if), for what purpose, what time of day, for how long , how many requests for password resets… ‘Class’ data – attendance registers, Clicker data Library data – physical or online – what resources downloaded, access, requested, how often Academic support data – phone calls/emails/visits to lecturers/tutors Administrative support data – phone calls/emails/visits – re payments, assignment marks, examination dates, remarksAffective support data – phone calls/emails/visits to support staff - counselling advisory services Internal/external requests for student data – who wanted to know what, for what purpose, under what conditions, ethical clearance required? Social media – Twitter, Facebook, Linkedin, Own Your Own Domain 39
  40. 40. Where is this data stored, in what format, what is the quality of this data, who has access to the data under what circumstances, for what purposes? 40
  41. 41. Student data [in all its varieties, and in different combinations] are [currently] used to… • Access - What are the assumptions about and the purpose of the data? How sure are we that the data means what we think it means? What are the purpose of controlling access – resources, placements, reputation? • Allocation of resources – placements in programmes, staff: student ratios, educational triage • Personalisation of curricula, assessment, feedback, support and shortened/extended programs • Marketing • Reporting – Internal/External • Curriculum (re)development • Quality assurance – who determines quality? • Addressing strategic objectives – e.g. addressing the legacy of colonialism and apartheid or dancing to the drum beat of the market? 41
  42. 42. Pointers for a way forward • Students’ digital lives and our data sets are but a minute part of a bigger whole – so we should not pretend as if our data represent the whole • The data we collect are never ‘raw’, ‘uncontaminated’, or just ‘scraped’… Our samples, choices, timing and tools change and impact on data. “Data are in fact framed technically, economically, ethically, temporally, spatially and philosophically. Data do not exist independently of the ideas, instruments, practices, contexts and knowledges used to generate, process and analyse them” (Kitchen, 2014, p. 2) • Data have contexts. To re-use data outside of the original context and purpose for which it was collected impacts on the contextual integrity. • Knowing ‘what’ is happening, does not necessarily tell us the ‘why’… • Education is an open, recursive system (Biesta 2007, 2010) where multiple variables not only intersect but often also constitute one another. Let us therefore tread carefully between correlation and causation… 42
  43. 43. Caught between correlation and causation Image credit: http://www.tylervigen.com/spurious-correlations 43
  44. 44. Caught between correlation and causation (cont.) Image credit: http://www.tylervigen.com/spurious-correlations 44
  45. 45. The collection, analysis and use of student data: some pointers 1. What are our (management, administrative, faculty and support staff’s) beliefs about knowledge, learning, assessment, data, and evidence? 2. What student data do we already have, why was it collected, in which format is it stored, who has access to the data, how is the data used by whom, and do students know this, have access to it, and know how it influences our and their choices? 3. What data do students currently have access to about their learning and about our choices pertaining to their learning? 45
  46. 46. The collection, analysis and use of student data: some pointers (cont.) 4. What data don’t students currently have access to, but we have, that will help them to plan their time and resources in order to maximize their chances of success? 5. What student data don’t we have, but need in order to teach better, allocate resources, and support students? Is this data available, under what conditions will we be able to access it, how will we govern its-- storage, combination with other sources of data, who will have access to it and under what conditions? 46
  47. 47. 1. There must be no personal-date record-keeping systems whose very existence is secret. 2. There must be a way for an individual to find out what information about him/her is in the record and how it is used. 3. There must be a way for an individual to prevent information obtained about him/her for one purpose for being used or made available for other purposes without his/her consent. 4. There must be a way for an individual to correct or amend a record of identifiable information about him/her. 5. Any organisaton creating, maintaining, using, or disseminating records of indentifiable personal data must assure the reliability of the data for their intended use and must take reasonable precautions to prevent misue of the data. 1973 Code of fair information practices 47
  48. 48. What are the ethical implications for the collection, analysis and use of student (digital) data? 1. The duty of reciprocal care • Make TOCs as accessible and understandable (the latter may mean longer…) • Make it clear what data is collected, when, for what purpose, for how long it will be kept and who will have access and under what circumstances • Provide users access to information and data held about them, to verify and/or question the conclusions drawn, and where necessary, provide context • Provide access to a neutral ombudsperson (Prinsloo & Slade, 2015) 48
  49. 49. What are the ethical implications …? (2) 2. The contextual integrity of privacy and data – ensure the contextual integrity and lifespan of personal data. Context matters… 2. Student agency and privacy self-management • The fiduciary duty of higher education implies a social contract of goodwill and ‘do no harm’ • The asymmetrical power relationship between institution and students necessitates transparency, accountability, access and input/collaboration • Empower students – digital citizenship/care • The costs and benefits of sharing data with the institution should be clear • Higher education should not accept a non-response as equal to opting in… (Prinsloo & Slade, 2015) 49
  50. 50. What are the ethical implications …? (3) 4. Future direction and reflection • Rethink consent and employ nudges – move away from thinking just in terms of a binary of opting in or out – but provide a range of choices in specific contexts or needs • Develop partial privacy self-management – based on context/need/value • Adjust privacy’s timing and focus - the downstream use of data, the importance of contextual integrity, the lifespan of data • Moving toward substance over neutrality – blocking troublesome and immoral practices, but also soft, negotiated spaces of reciprocal care (Prinsloo & Slade, 2015) 50
  51. 51. • Knowing • Not knowing • Knowing what we don’t know • Knowing what we may never know • Knowing more? The solution is not only (or necessarily?) in knowing more, but ensuring that once we know, we respond in ethical, caring, discipline and context-appropriate ways What are the ethical implications of … 51
  52. 52. (In)conclusions 52 “Technology is neither good or bad; nor is it neutral… technology’s interaction with social ecology is such that technical developments frequently have environmental, social, and human consequences that go far beyond the immediate purposes of the technical devices and practices themselves” Melvin Kranzberg (1986, p. 545 in boyd & Crawford, 2012, p. 1)
  53. 53. Thank you Paul Prinsloo (Prof) Research Professor in Open Distance Learning (ODL) College of Economic and Management Sciences, Office number 3-15, Club 1, Hazelwood, P O Box 392 Unisa, 0003, Republic of South Africa T: +27 (0) 12 433 4719 (office) prinsp@unisa.ac.za Skype: paul.prinsloo59 Personal blog: http://opendistanceteachingandlearning.wordpress.com Twitter profile: @14prinsp 53
  54. 54. Useful resources 54 Source: https://press.anu.edu.au/publications/series/centre-aboriginal-economic-policy-research-caepr/indigenous-data- sovereignty
  55. 55. Useful resources (cont.) 55 Source: http://www.nature.com/news/south-africa-s-san-people-issue-ethics-code-to-scientists-1.21684
  56. 56. Useful resources (cont.) 56 Source: http://journals.sagepub.com/doi/abs/10.1177/0002764213479366
  57. 57. Useful resources (cont.) 57 Source: http://oro.open.ac.uk/47251/
  58. 58. Useful resources (cont.) 58 Source: https://the-other-jeff.com/wp-content/uploads/2017/02/Structural-Justice-in-Learning-Analytics-ESS.pdf
  59. 59. Useful resources (cont.) 59 Source: http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005399
  60. 60. Useful resources (cont.) 60 Source: https://na-production.s3.amazonaws.com/documents/Promise-and-Peril_4.pdf
  61. 61. Useful resources (cont.) 61 Source: https://na-production.s3.amazonaws.com/documents/Predictive-Analytics-GuidingPractices.pdf
  62. 62. Useful resources (cont.) 62 Source: https://www.slideshare.net/abelardo_pardo/the-role-of-data-in-the-provision-of-feedback-at-scale

×