An example of introducing an analytics tool targetting a specific high impact problem, with brief reflections on key success factors to date.

1. Introducing
Analy0cs
at
UTS
Example:
Academic
Wri0ng
Analy0cs
Simon Buckingham Shum
Director, Connected Intelligence Centre
Professor of Learning Informatics
@sbuckshum #LearningAnalytics
utscic.edu.au

2. Stakeholders to build writing analytics tool
2
Ágnes Sándor
Xerox Research Centre Europe
Grenoble
(linguist: modelling
academic writing)
Xerox
Incremental
Parser
Rosalie Goldsmith
Institute for Multimedia Learning
(researcher in reflective writing
in engineering)
Philippa Ryan
Faculty of Law
(academic teaching civil law)
Reflective
writing
rubrics
Civil Law
student
writing
Simon Buckingham Shum
Shawn Wang, Simon Knight
Connected Intelligence Centre
(learning analytics researchers and web developer)
User requirements, interface design and
evaluation, data collection & analysis

3. AWA: Analytical Academic Writing
3
Roll over sentences with Fkeys for a
popup reminding you of their meaning

4. Analytical academic writing (Civil Law)
4

5. Engaging Law academic in tuning AWA
5
1.
Trigger
curiosity
through
CIC
briefings
2.
Submit
collabora0on
proposal
to
CIC
3.
Agree
0melines
4.
Think
aloud
user
interface
walkthrough
5.
Preliminary
text
analyses
6.
Systema0c
text
analyses
7.
Refine
parser
8.
Design
student
pilot
9.
Gather
student
feedback

6. 6
From informal rubric to formal rhetorical pattern

7. AWA: reflective academic writing
7

8. Validating the tool: metrics
8
human highlighting automated highlighting

9. Structured annotation from the civil law
academic to tune the parser for her discipline
9

10. Validating the tool: student feedback
10
“takes
the
emo+on
out
of
having
your
work
scru+nized”
“it
was
not
embarrassing
in
the
way
that
it
can
be
when
a
tutor
or
marker
gives
feedback”
“I
definitely
found
it
useful.
It
also
made
me
realise
that
I
tend
to
use
bold,
certain
language
in
making
my
point
towards
the
end
of
each
paragraph
rather
than
up
front
at
the
beginning
(when
introducing
my
point).”
“I
realise
now
what
descrip+ve
wri+ng
is
-­‐
the
soCware
had
quite
a
bit
to
say
about
my
lack
of
jus+fica+on
-­‐
also
true
-­‐
pressed
for
+me
and
difficult
circumstances
have
caused
this
for
me
in
this
instance
-­‐
good
to
see
it
sampled.”

11. Validating the tool: academic trust
11
An issue of “Algorithmic Accountability”
Critical to forge trust between all the stakeholders: Educators, Analysts, Students
We aim to ensure that academics feel that:
their expertise is respected
their feedback has an impact
they understand the tool sufficiently to trust it
ideally they get esteem and research outputs from engaging with analytics