Results from the student digital experience tracker pilot project - Tabetha Newman and Helen Beetham

1. Student digital experience tracker 2016
Results from the pilot project -Tabetha Newman and Helen Beetham
13/06/2016

2. Executive summary
» Jisc developed a set of 11 questions, designed to provide a snapshot of students' digital
experience while engaged in a course of study
» In February-April 2016 Jisc delivered two versions of the questions (HE, and FE and skills)
as a pilot tracker service for institutions
» The questions were delivered in BOS and piloted across 24 institutions (12 HE and 12 FE
and skills, the latter including two specialist colleges)
» Pilot sites were supported through the process with identical guidance to simulate a
real-world service delivery setting and to ensure equivalence of data
» A total of 10,753 students completed the tracker questions
» We have a reliable snapshot of the situation with respect to device ownership and use,
access to digital services, typical digital activities on course, and how students rate
different aspects of their digital experience
» The data reveal significant differences between students in HE and in FE and skills
13/06/2016 Student digital experience tracker pilot results 2016 3

3. Contents
» Introduction
» Devices used to support student’s learning
» Access to institutional digital resources
» Digital skills within the course experience
» Institutional digital guidance and support
» Student digital experience
» Digital technologies in learning: what students want to start, stop and keep
13/06/2016 Student digital experience tracker pilot results 2016 4

4. Introduction
13/06/2016 Student digital experience tracker pilot results 2016 5

5. Development of the tracker
» A series of consultations carried out by Jisc during
2014-2015 found a demand for a better understanding
of students' digital experiences and expectations
» The tracker questions were based on research by the
Jisc Digital student project including studies of
students in HE, FE and skills
» A live consultation workshop with staff and students
refined the proposed questions from 36 to 12
» Cognitive interviewing with FE and HE students, and
early beta testing further refined the questions
for piloting
13/06/2016 Student digital experience tracker pilot results 2016 6

6. Implementation of the tracker
» The survey was implemented in BOS onlinesurveys.ac.uk,
a system developed for UK HE and FE and already in use
at many of the pilot sites
» 50+ education providers applied to take part in the pilot:
24 were chosen based on a spread of institutional sizes,
types and locations
» Pilot sites were required to engage stakeholders and
explore their reasons for taking part in the pilot
» Pilot sites were responsible for customising the survey,
releasing it to learners, engaging learners to complete it,
and analysing their own data
13/06/2016 Student digital experience tracker pilot results 2016 7

7. Implementation of the tracker
» Six separate guides were produced to support the
pilot process and to ensure parity of the data collected
» Sites had only the guides and collective email/blog
postings for support, to reproduce service
delivery conditions
» The project negotiated for benchmarking data to be
made available to pilot sites within the BOS system
13/06/2016 Student digital experience tracker pilot results 2016 8

8. Evaluation of the tracker
» Synthesis data has undergone preliminary analysis (this report)
» Further analysis is planned by this and other teams within Jisc
» Each pilot site is supported to analyse, understand and respond to its own data, with the
involvement of key stakeholders
» Standardised feedback about the pilot process has been collected from the lead contact at
all 24 pilot sites
» The experience of six contrasting sites is being explored in more detail, including feedback
from learners and other stakeholders
13/06/2016 Student digital experience tracker pilot results 2016 9

9. Sample sizes and response rates
» A total of 10,753 students across the 24 pilot institutions answered at least one question in
theTracker: a median average of 245 responses per institution
» 7,425 students from twelve HE and 3,328 students from twelve FE and skills institutions
completed the tracker (an average of 285 responses per HE institution, and 219 per FE
institution)
» The maximum number of responses from one institution was 3,592 (from a university with
about 20,000 students); the minimum number of responses was 25 (from a specialist
college with about 40 students)
» Questions were optional, hence the exact sample size differs for every question
» A total of 11 of our 24 institutions (46%) collected over 300 responses, and 19 (79%)
collected at least 100 responses from students
13/06/2016 Student digital experience tracker pilot results 2016 10

10. Analysis and statistics
» Data is presented graphically where possible, usually as percentages of students choosing
each option
» Where a “don’t know” option was available, this data is usually included unless:
› HE student data is compared to FE and skills student data where possible
› Comparisons between proportions (eg between FE and skills versus HE students) were
carried out using a ZTest calculator set at a 99.9% confidence level (higher than the
minimum level of 95% in order to reflect the number of times that comparisons were
run with this data set)
» The magnify glass (see below right) is positioned at the bottom right of pages where a
particularly interesting result has been found
13/06/2016 Student digital experience tracker pilot results 2016 11

11. Q1: What digital devices do you use to support your learning?
13/06/2016 Student digital experience tracker pilot results 2016 12

12. Headline Q1 findings
With regards to use of digital devices to support learning:
» Of the devices listed, students are most likely to use institutional desktops and printers to
support their learning
» When it comes to personal devices, students are more likely to use laptops, smartphones
and tablets than desktops to support their learning
» 90% of HE students use personal laptops, in comparison with only 66% of FE and
skills students
» 40% of FE and skills students use personal desktops, in comparison with only 30% of
HE students
» FE and skills students are significantly more likely to use institutional desktops, laptops,
tablets and smartphones to support their learning in comparison with HE students
13/06/2016 Student digital experience tracker pilot results 2016 13

13. Personal devices
13/06/2016 Student digital experience tracker pilot results 2016 14
» HE, and FE and skills students are more likely to use personal laptops, smartphones and
tablets than desktops to support their learning
30%
90%
47%
84%
43%
6%
40%
66%
45%
80%
47%
5%
Desktop Laptop Tablet Smartphone Printer Other
HE FE and skills

14. Personal devices – HE v FE differences
There were two significant differences
between HE versus FE and skills students in
terms of personal use of digital devices to
support their learning:
» 90% of HE students use personal
laptops, in comparison with only 66% of
FE and skills students
» 40% of FE and skills students use
personal desktops, in comparison with
only 30% of HE students
13/06/2016 Student digital experience tracker pilot results 2016 15

15. Institutional devices
» Of all the devices listed, HE and FE and skills students are most likely to use institutional
desktop computers and institutional printers to support their learning
» “Other digital devices” used personally or in institutions included Kindles, scanners, 3D
printers, iPods, cameras and games consoles
13/06/2016 Student digital experience tracker pilot results 2016 16
74%
21%
7% 5%
71%
6%
84%
36%
12% 12%
71%
4%
Desktop Laptop Tablet Smartphone Printer Other
HE FE and skills

16. Personal devices – HE v FE differences
FE and skills students are significantly more
likely to use institutional desktops, laptops,
tablets and smartphones to support their
learning in comparison with HE students:
» Desktops: 84% of FE and skills students in
comparison with 74% of HE students
» Laptops: 36% of FE and skills students in
comparison with only 21% of HE students
» Tablets: 12% of FE and skills students in
comparison with only 7% of HE students
» Smartphones: 12% of FE and skills students
in comparison with only 5% of HE students
13/06/2016 Student digital experience tracker pilot results 2016 17

17. Q2: How often can you access the following digital services at
your institution?
13/06/2016 Student digital experience tracker pilot results 2016 18

18. Headline Q2 findings
With regards to accessing institutional digital services:
» Significantly more FE and skills students could frequently access computers and printers in
comparison with HE students
» Significantly more HE students could frequently accessWi-Fi in comparison with FE and
skills students
» Significantly more HE students could frequently access personal information (eg. grades,
module choices) online in comparison with FE and skills students
» A significantly higher % of FE and skills students said they could never access personal
information online in comparison with HE students
13/06/2016 Student digital experience tracker pilot results 2016 19

19. Computers and printers
» 64.1% of HE students could access
university computers and printers most
of the time
» 73.2% of FE and skills students could
access college/learning provider
computers and printers most of the time
» Significantly more FE and skills students
could frequently access computers and
printers in comparison with HE students
13/06/2016 Student digital experience tracker pilot results 2016 20
64%
31%
3%
2%
73%
24%
2%
2%
Most of the time
Sometimes
Never
Don't know
HE FE and skills

20. Wi-Fi in buildings
» 79.5% of HE students could accessWi-Fi
in university buildings most of the time
» 70.0% of FE and skills students could
accessWi-Fi in college/learning provider
buildings most of the time
» Significantly more HE students could
frequently accessWi-Fi in comparison
with FE and skills students
13/06/2016 Student digital experience tracker pilot results 2016 21
79%
16%
4%
1%
70%
22%
6%
3%
Most of the time
Sometimes
Never
Don't know
HE FE and skills

21. E-books and e-journals
» 89.7% of HE students could access e-
books and e-journals some or most of
the time; 5% said they could never
access them and 5% said they
didn’t know
» 53.0% of HE students could access e-
books and e-journals most of the time
» A further 36.7% could access e-books
and e-journals sometimes
» This question was not asked to FE and
skills students
13/06/2016 Student digital experience tracker pilot results 2016 22
53%
37%
5%
5%
Most of the time
Sometimes
Never
Don't know
HE

22. Online course materials
» 71.2% of HE students could access online
course materials most of the time
» 60.4% of FE and skills students could
access online course materials most of
the time
» Significantly more HE students could
frequently access online course materials
in comparison with FE and skills students
13/06/2016 Student digital experience tracker pilot results 2016 23
71%
24%
2%
2%
60%
30%
4%
5%
Most of the time
Sometimes
Never
Don't know
HE FE and skills

23. Other learning resources
» 87.6% of FE and skills students could access
other learning resources materials most or
some of the time
» 51.6% of FE and skills students could access
other learning resources most of the time
» 4% said they could never access other
learning resources, and 9% chose “don’t
know” (perhaps this option was
too ambiguous?)
» This question was not asked to HE students
13/06/2016 Student digital experience tracker pilot results 2016 24
52%
36%
4%
9%
Most of the time
Sometimes
Never
Don't know
FE and skills

24. Personal information online
» 70.9% of HE students could access
personal information (eg grades, module
choices) online most of the time
» 54.9% of FE and skills students could
access personal information (eg grades,
module choices) online most of the time
» Significantly more HE students could
frequently access personal information
(eg grades, module choices) online in
comparison with FE and skills students
» A significantly higher % of FE and skills
students said they could never access
personal information online in
comparison with HE students
13/06/2016 Student digital experience tracker pilot results 2016 25
71%
26%
2%
1%
55%
31%
7%
8%
Most of the time
Sometimes
Never
Don't know
HE FE and skills

25. Online file storage
» 80.0% of HE students could access online
file storage (eg shared or Google drives,
or inThe Cloud) most or some of
the time
» 82% of FE and skills students could
access file storage most of the time
» There was no statistical difference
between HE and FE and skills students in
terms of their access to online
file storage
13/06/2016 Student digital experience tracker pilot results 2016 26
51%
29%
9%
11%
55%
28%
8%
10%
Most of the time
Sometimes
Never
Don't know
HE FE and skills

26. Q3: As part of your course, in the last six weeks have you done
the following activities?
13/06/2016 Student digital experience tracker pilot results 2016 27

27. Headline Q3 findings
» HE students are significantly more likely than FE/skills students to have produced work in
a digital format, and to have worked online with others
During the last six weeks of their course:
» Nine in every 10 students found information online
» Seven in 10 students produced work in a digital format
» Five in every 10 students worked online with others
» Three in every 10 students created a personal record of their learning
13/06/2016 Student digital experience tracker pilot results 2016 28

28. Found information online
» As part of their course, 96.0% of HE and
94.1% of FE and skills students had been
asked to find information online in the
past six weeks
» Although the percentage difference here
is very small, there is a significant
difference, with HE students being asked
to complete this action more than FE and
skills students in the last six weeks
» This result is possible because of the
large volume of data collected, allowing
for a robust comparison between the
two groups of students
13/06/2016 Student digital experience tracker pilot results 2016 29
96% 94%
HE FE&S
Found info' online
HE FE and skills

29. Worked online with others
» As part of their course, 62.8% of HE and
46.6% of FE and skills students had been
asked to work online with others in the last
6 weeks (eg using discussion boards,
Skype, Facebook groups,Twitter, Google+)
» There is a significant difference, with HE
students being asked to complete this
action more than FE and skills students in
the last six weeks
13/06/2016 Student digital experience tracker pilot results 2016 30
63%
47%
HE FE&S
Worked online with others
HE FE and skills

30. Produced work in digital format
» As part of their course, 79.1% of HE and
71.7% of FE and skills students had been
asked to produce work in a digital format
in the past six weeks (eg using
presentations, web pages,
info-graphics, animations)
» There is a significant difference, with HE
students being asked to complete this
action more than FE and skills students
in the last six weeks
13/06/2016 Student digital experience tracker pilot results 2016 31
79%
72%
HE FE&S
Produced digital work
HE FE and skills

31. Created personal record of your learning
» As part of their course, 34.9% of HE and
32.6% of FE and skills students had been
asked to create a personal record of their
learning in the last six weeks (eg using a
blog or e-portfolio)
» There is no significant difference
between the proportion of HE and FE
and skills students who had been asked
to create a personal learning record in
the last six weeks
13/06/2016 Student digital experience tracker pilot results 2016 32
35% 33%
HE FE&S
Created personal learning record
HE FE and skills

32. Q4: Do you get enough guidance and support to help you with
the following?
13/06/2016 Student digital experience tracker pilot results 2016 33

33. Headline Q4 findings
With regards to guidance and support:
» Less than half of all students say they have received guidance on: modifying devices to
suit their individual needs, and creating a positive online presence
» Less than two thirds of all students say they have received guidance on: bringing your own
device, and developing digital skills relevant to their course
» Significantly more FE and skills students than HE students had received guidance on:
bringing your own device, developing digital skills relevant to their course, behaving safely
and respectfully online, and modifying devices to suit their needs
13/06/2016 Student digital experience tracker pilot results 2016 34

34. Using own device (bring your own - BYO)
» 58.5% of HE students and 66.1% of FE
and skills students agreed that yes, they
had received enough guidance and
support to help them use their own
device (BYO) for learning in the
institution (eg using their laptop, tablet
or phone)
» Significantly more FE and skills students
received guidance to BYO in comparison
with HE students
» About two in ten students had never
tried to ask for BYO support
13/06/2016 Student digital experience tracker pilot results 2016 35
59%
13%
29%
66%
13%
21%
Yes No Never tried
HE FE and skills

35. Develop digital skills relevant to course
» 56.1% of HE students and 67.1% of FE
and skills students agreed that yes, they
had received enough guidance and
support to help them develop the digital
skills relevant to their course
» Significantly more FE and skills students
received this guidance in comparison
with HE students
13/06/2016 Student digital experience tracker pilot results 2016 36
56%
24%
20%
67%
17% 16%
Yes No Never tried
HE FE and skills

36. Behaving safely/respectfully online
» 63.7% of HE students and 81.9% of FE
and skills students agreed that yes, they
had received enough guidance and
support to help them behave safely and
respectfully online
» Significantly more FE and skills students
received this guidance in comparison
with HE students
13/06/2016 Student digital experience tracker pilot results 2016 37
64%
13%
23%
82%
6%
12%
Yes No Never tried
HE FE and skills

37. Modify devices to suit individual needs
» 32.0% of HE students and 39.3% of FE
and skills students agreed that yes, they
had received enough guidance and
support to help them modify digital
devices to suit their learning needs (eg
text-to-speech, assistive software)
» Significantly more FE and skills students
received this guidance in comparison
with HE students
13/06/2016 Student digital experience tracker pilot results 2016 38
32%
27%
41%
39%
26%
35%
Yes No Never tried
HE FE and skills

38. Create positive online profile
» 46.3% of HE students and 46.5% of FE
and skills students agreed that yes, they
had received enough guidance and
support to help them create a positive
online profile (eg LinkedIn, CV,
e-portfolio)
» There was no significant difference
between HE and FE and skills students
13/06/2016 Student digital experience tracker pilot results 2016 39
46%
28%
25%
47%
26%
28%
Yes No Never tried
HE FE and skills

39. Q5: How much do you agree with the following statements about
the student digital experience?
13/06/2016 Student digital experience tracker pilot results 2016 40

40. Headline Q5 findings
13/06/2016 Student digital experience tracker pilot results 2016 41
With regards to the overall student digital experience:
» Approximately seven in ten of students believe that when technology is used by teaching
staff it enhances their learning experience
» Around six in 10 students believe that digital assessments are delivered well
» FE and skills students are significantly more likely than HE students to say that they know
how their personal data is used by their provider, and that they feel involved in decisions
about the digital environment

41. Technology use by teaching staff
» 72.2% of HE students and 70.1% of FE
and skills students agreed that when
technology is used by teaching staff, it
helped their learning experience
» There was no significant difference in the
opinions of HE and FE and skills students
13/06/2016 Student digital experience tracker pilot results 2016 42
72%
25%
2%
70%
28%
2%
Agree
Neutral
Disagree
HE FE and skills

42. Technology use by support staff
» 65.8% of HE students and 59.7% of FE
and skills students agreed that support
staff (learning resources, admin. etc.) use
digital technologies confidently
» Significantly more HE students agreed
with this statement in comparison with
FE and skills students, who were more
likely to have a neutral opinion
13/06/2016 Student digital experience tracker pilot results 2016 43
66%
31%
4%
60%
35%
5%
Agree
Neutral
Disagree
HE FE and skills

43. I’m told how personal data is handled
» 31.5% of HE students and 44.5% of FE and
skills students agreed that they are told
how their personal data is stored and used
» Significantly more FE and skills students
agreed with this statement in comparison
with HE students, who were more likely
to disagree
13/06/2016 Student digital experience tracker pilot results 2016 44
32%
35%
33%
45%
34%
22%
Agree
Neutral
Disagree
HE FE and skills

44. Online assessments are delivered well
» 58.1% of HE students and 55.8% of FE
and skills students agreed that online
assessments are delivered and
managed well
» There was no significant difference
between HE and FE and skills students
13/06/2016 Student digital experience tracker pilot results 2016 45
58%
32%
10%
56%
36%
9%
Agree
Neutral
Disagree
HE FE and skills

45. Learners are involved in decision-making
» 37.5% of HE students and 43.2% of FE
and skills students agreed that learners
are given the chance to be involved in
decisions about digital services
» Significantly more FE and skills students
agreed with this statement in
comparison with HE students, who were
more likely to disagree
13/06/2016 Student digital experience tracker pilot results 2016 46
38%
37%
26%
43%
36%
21%
Agree
Neutral
Disagree
HE FE and skills

46. Q6, 7, 8: What should we start, stop and keep doing when it
comes to digital technologies?
13/06/2016 Student digital experience tracker pilot results 2016 47

47. Headline Q6,7,8 findings
13/06/2016 Student digital experience tracker pilot results 2016 48
» Students were asked to describe that they would like their institution to start, stop and
keep doing with regards to using digital technologies
» The main issues are summarised in the following slides

48. HE: What to start doing
The following themes were often
mentioned as something that HE
institutions should start doing:
» Recording lectures
» Better use ofVLEs: standardise use by
staff, add presentations, teach students
how to use it effectively, and improve
access (eg mobile access)
» Improve online services: more online
resources/activities, assessment
submissions
» Access to better/more computers
13/06/2016 Student digital experience tracker pilot results 2016 49

49. FE and skills: What to start doing
The following themes were often
mentioned as something that FE and skills
institutions should start doing:
» Access to better/more computers
and laptops
» BetterWi-Fi access
» Teach students and staff how to best use
technology (hardware and software)
effectively and efficiently
» Put more resources online and make
them easy to find
13/06/2016 Student digital experience tracker pilot results 2016 50

50. HE: What to stop doing
The following themes were often
mentioned as something that HE
institutions should stop doing:
» Stop sending cluttering emails! (LOTS of
students said this – is there a better way
to match students with relevant emails?)
» Early morning lectures
» “Death by PowerPoint” or very long
lectures
13/06/2016 Student digital experience tracker pilot results 2016 51

51. FE and skills: What to stop doing
The following themes were often
mentioned as something that FE and skills
institutions should stop doing:
» Blocking access to websites
» Giving multiple assignments at the
same time
» Stop using Chrome machines, and
google docs
13/06/2016 Student digital experience tracker pilot results 2016 52

52. HE: What to keep doing
The following themes were often
mentioned as something that HE
institutions should keep doing:
» Online 24/7 access to as much content
as possible
» 24/7 library access
» Putting as much a possible (lecture
notes, slides etc) on Blackboard, and
making it accessible any time any place
» Using technology, and embracing new
technology services and resources
13/06/2016 Student digital experience tracker pilot results 2016 53

53. FE and skills: What to keep doing
The following themes were often
mentioned as something that FE and skills
institutions should keep doing:
» Providing support for learners
» Allowing access to social media and
the internet
» Allowing home access to course materials
» GoodWi-Fi connectivity
» Embedding technology in the teaching
learning experience
» Using google classroom
13/06/2016 Student digital experience tracker pilot results 2016 54

54. jisc.ac.uk
For more information contact
Sarah Knight
Senior co-design manager
sarah.knight@jisc.ac.uk
13/06/2016 Student digital experience tracker pilot results 2016 55