Keynote presentation for the ILIAS conference

1. ETHICAL CONSIDERATIONS
ABOUT THE DATAFICATION OF
EDUCATION
Dr. Javiera Atenas

2. OUR SOCIETY IS ONE NOT OF
SPECTACLE,
BUT OF SURVEILLANCE
Michel Foucault.
Panopticon, 1975

3. SURVEILLED LEARNING
The desire to measure the quantity and quality of learning
are not new - traditionally education has made use of
assessment in order to check student understanding and
progress.
With the advent of ‘big data’ of student activity we have
moved into an age of surveilled learning, in which every click
seemingly counts.

4. ORWELLIAN TALES
Australian schools are being asked to trial technology which will
allow them to phase out morning roll call and spy on students
throughout the school day. Start-up LoopLearn is hopeful its
advanced facial recognition technology and “small,
unobtrusive devices” which scan campuses for students in real
time, will be embraced by
schools across the country
(Corp, 2018).

5. A DATAFIED SOCIETY
We live in a datafied society where almost everything is transcribed into data,
quantified and analysed (Schäfer & Van Es, 2017).
For Giroux (2010), education must develop and improve people's ability to
recognise and challenge power dynamics, and Schäfer & Van Es (2017) argue
that “students need to be educated to become critical data practitioners who
are both capable of working with data and of critically questioning the big myths
that frame the datafied society”.
If students cannot understand data, they become merely objects of study,
rendering their points of view and their lives invisible, making them just data
(Atenas & Havemann, 2015).

6. WEAPONS OF MATH DESTRUCTION
Algorithms can favour discrimination, stigmatisation as due to its obscure
nature can portray learners in an unfair manner, but also, it can unfairly evaluate
educators only against the performance of their students.
Algorithms use performance data to rank individuals against a series of metrics,
however, how this data is stored, managed, shared and accessed is still a
mystery, as schools and universities provide corporations with students
performance data, and the government provides corporations with socio-
economic data alongside with students background information and this data
can be used in the future to inform potential employers, banks, insurance
companies and also, health providers.

8. ORWELLIAN TALES
Technology being proffered to schools may be more likely to
misfire on language used by black youth, potentially causing
them to experience greater scrutiny from school administrators.
(Wired, 2018)
A ProPublica investigation challenged COMPAS as “likely to
falsely flag black defendants as future criminals, wrongly
labelling them this way at almost twice the rate as white
defendants” (Chander, 2017)

9. DATA GOVERNED EDUCATION
Education seems to be governed by data, and this needs to be critically
questioned by scholars and researchers to understand and examine the
methods and approaches used by algorithmists and data scientist as
their claims and reports can have an impact in the development of
policies, putting at risk vulnerable groups (Williamson, 2016).
The measurement of everything is central to the modern educational
experience, whereby success is framed in terms of targets achieved and
performance is evaluated through ever more complex metrics (Grek,
2009; Grek, 2015; Ozga, 2009).

10. DATAFIED EDUCATION
The transformation of complex educational processes into data points that can
be used to sort, order, benchmark, compare, and rank. Numbers, and “data,”
become increasingly significant in framing the working lives and experience of
teachers (Stevenson, 2017)
Children are becoming the objects of a multitude of monitoring devices that
generate detailed data about them (Lupton & Williamson, 2017).
Databases, reinvent teachers and children into data that can be measured,
compared, assessed and acted upon, children become reconfigured as miniature
centres of calculation (Williamson, 2014)

11. ORWELLIAN TALES
The "intelligent classroom behavior management system" used at
Hangzhou No. 11 High School incorporates a facial recognition
camera that scans the classroom every 30 seconds. The camera is
designed to log six types of behaviors by the students: reading,
writing, hand raising, standing up, listening to the teacher,
and leaning on the desk. It also records the facial expressions of
the students and logs whether they look happy, upset, angry,
fearful or disgusted.
(Liang Jun, 2018).

12. LEARNING AND ALGORITHMS
Algorithms are or can be broken and biased, as they are
obscure, secretive, complex and oppose to the conceptions of
participation and transparency that are promoted in the current
political landscape.
When students’ performance is measured through algorithms it
can have an impact in their lives, by stigmatising them or by
portraying them in an unfair manner

14. ETHICAL CONUNDRUMS
● Can we predict learning through interaction with devices?
● Can we predict students’ potential through their demographic
background?
● Is it ethic to monitor student from Afro-Caribbean descent over
white students?
● Is it ethically accepted to surveil female students?
● Is it ethically accepted to evaluate educators’ quality against
students’ performance?

15. ORWELLIAN TALES
Florida’s economy had been buoyed by the housing bubble and
suffered immensely during the crash. To manage a shrinking budget,
the legislature made a fundamental change. In 2013, it passed a law
establishing “performance funding” for Florida’s public universities,
directly tied to a school’s score on certain criteria, including
persistence and graduation rates. The three worst-performing
schools would miss out on this funding entirely. (Carey, 2018)

16. ETHICAL CONUNDRUMS
Why should students from a poor background be targeted to
monitor their learning due to predictions’ because of their heritage?
Why students coming from deprived neighbourhoods should be
surveilled because of the rankings of their schools?
Why machines are telling us how students learn through their
interactions with machines that are coded to learn using machine
learning techniques?

17. ORWELLIAN TALES
Turnitin will monitor and learn the writing styles of individual
students and flag up content which shows considerable
divergence from their previous work. (Warner, 2018)
The University of Kentucky released a plan to install 2,000
surveillance cameras on campus and give students new ID cards
that will contain chips that can track student movements in and
out of buildings (The Guardian, 2013).

18. STUDENTS’ RIGHT TO PRIVACY
Educational data, which includes performance, social background
data, educational budget, is normally analysed through algorithms,
and it affects education governance because its social, institutional,
political and economic contexts, therefore ethical aspects need to be
reviewed
(Williamson, 2016).
It is important that institutions recognise that data and algorithms
can contain and perpetuate bias (University of Edinburgh, 2018)

19. ORWELLIAN TALES
Social Sentinel provides a structured process to mitigate
risks pro-actively and Geo-Listening pitches the powerful
benefits of a service that "help you better meet the social
and emotional needs of your students that they'll know
about because they will "monitor, analyze and report"
student social network postings (Forbes, 2018).

20. LA - EDUCATIONAL CONTEXT
The Society for Learning Analytics Research defines
learning analytics as ‘the measurement, collection,
analysis and reporting of data about learners and their
contexts, for purposes of understanding and optimizing
learning and the environments in which it occurs’

21. LA - EDUCATIONAL CONTEXT
The techniques used in learning analytics are based on standard
statistical methods, but typically involve the development of complex
models, the full working of which will only be apparent to those familiar
with the data and with the statistical methods employed. It is likely,
however, that users will want to understand how the models produce
the outcomes which they then deploy. Students will want to
understand why they have been selected for an intervention and, in
some cases, may want to challenge the basis for their selection
(EU-JRC, 2016)

22. ORWELLIAN TALES
A week after students begin their distance learning courses at the UK’s Open
University this October, a computer program will have predicted their final
grade. An algorithm monitoring how much the new recruits have read of their
online textbooks, and how keenly they have engaged with web learning forums,
will cross-reference this information against data on each person’s socio-
economic background. It will identify those likely to founder and pinpoint when
they will start struggling. Throughout the course, the university will know how
hard students are working by continuing to scrutinise their online reading
habits and test scores. (Forbes, 2015)

24. HOW IS ‘LEARNING’ MEASURED?
Can learning be measured through frequency of clicks?
Can algorithms provide substantial information about quality of learning,
learners and educators?
Reducing human behaviour, performance and potential to algorithmic
analysis is indeed quite risky.
Analysing learning through algorithms (normally outsourced to Ed-Tech
corporations) needs to be handled with care, due to ethical challenges

25. ALGORITHMIC SURVEILLANCE
Predictive modelling in Learning Analytics can lead us to algorithmic
discrimination of learners, we cannot predict the future, machines
cannot either
Allowing the proliferation of algorithmic surveillance as a substitution
for human engagement and judgment helps pave the road to an ugly
future where students spend more time interacting algorithms than
instructors or each other
(Warner, 2018)

26. ASSISTIVE OR PUNITIVE INTERVENTIONS
“The overarching purpose of analytics in education today is
merely to punish those who get bad data, and to reward
those who get good data by leaving them alone. We are
squandering the power of 21st-century data analytics in
education by deploying it firmly inside a 19th-century
Skinner box of basic rewards and punishments”
(Kuhn, 2016).

28. POLICY RECOMMENDATIONS
● Ensure everyone is aware which data is being collected, for what
purposes
● Don’t collect/retain/share data unnecessarily
● Ensure data collected will not affect social mobility
● Carefully select who will be analysing your data
● Consider bias and their potential effects
● Create a Data Governance committee

30. “A right to privacy is neither a right to secrecy nor a right to
control but a right to appropriate flow of personal information
… Privacy may still
be posited as an important human right or value worth
protecting through law and other means, but what this
amounts to is contextual integrity and what this amounts to
varies from context to context”.
(Nissenbaum, 2010)