Higher education faces challenges in addressing economic and readiness problems for students from disadvantaged backgrounds. While free educational content helps, it does not solve complex readiness issues. For-profit models also struggle with serving students in remote, underserved areas. Analytics and technologies show promise in helping institutions address these "last mile" challenges through personalized, adaptive learning approaches. However, significant organizational changes and integration of disparate data sources will be required for institutions to fully leverage these tools. Open discussion is needed around ensuring insights into learning and student success remain available as public goods, not proprietary to any private vendor.

1. Analytics & Higher Education
Vince Kellen, Ph.D.
Senior Vice Provost
Analytics and Technologies
University of Kentucky
Vince.Kellen@uky.edu
January, 2014
This is a living document subject to substantial revision.

2. Higher education has a ‘last mile’ problem
 Education in any form is struggling to address families and communities with economic and
other readiness problems
 Free or low-cost educational content does not easily solve readiness problems which have a
multitude of factors
 For profit models rightfully struggle with ‘last-mile’ problems. Public policy matters!
2

3. What would Abraham Lincoln think of things?
Abraham Lincoln
• Autodidactic
• Books, books, books
• Became a skilled military strategist
• Penchant for poetry, Shakespeare,
politics and history
My nephew
• Not an autodidact
• Good worker, smart kid, but…
• It takes a village
• After a few low-security colleges
and much money borrowed
• He has found an intellectual home
3

4. Analytics in higher education is hot right now
 A number of vendors, old and new are making sales now
• Knewton, Starfish, Civitas, Education Advisor Board, Banner (Signals) and
others
• Value proposition: we collect lots of data, we have data scientists who can
analyze, we collect lots of best practice that we can share with you, we host
the data and systems so you don’t have to
 Adaptive learning got a boost this past year
• Gates Foundation spurred implementations with funding for colleges and
universities to implement pilot programs
• Reports by Education Growth Advisors, Foundation report identified and
evaluated eight adaptive learning vendors
• APLU formed a Personalized Learning Consortia (also partially funded by the
Gates Foundation) to spur collaboration between universities for personalized
learning content and platform development
• Industry press has picked up coverage (Chronicle, InsideHigherEd) and
vendors are making acquisitions
4

5. Analytics have been around awhile
 Thousands of studies and research on many aspects of student success
and the psychology of learning over several decades
 Most institutions understand the common causes
• High school GPA, test scores, family familiarity with higher education, personal and
family expectations, wealth, high identification with an academic discipline, high
motivation, conscientiousness, involvement in academics and co-curricular activities,
strength of and placement within a social network, problems in course progressions and
degree choice
 Most institutions understand and are starting to act upon the signals
• First semester performance, mid-term grades, the first few weeks of progress in
academic classes, identification of gaps in needed skills and remediation
 Many institutions are taking steps to reform the teaching and support
• Hybrid designs, active learning designs, guide-on-the-side versus sage-on-the-stage
• Early intervention, live and learn communities, peer mentoring, professional advising
5

6. What is different now?
 The technology for analytics has undergone a recent renaissance
• Different forms of high-speed, big-data processing are coming forward
– Structured and unstructured data can be rapidly analyzed
– Queries can be run against both structured and unstructured stores simultaneously
– The hardware is now more parallel enabling ‘scale-out’ designs to handle big data
– Apple Siri, IBM Watson and others have grabbed mainstream attention
• Data visualization is well established
– Many tools offer interesting ways of visualizing data, enabling better communication of insights
• MOOCs have brought attention to eLearning opportunities
– MOOCs are expected to incorporate analytics to help improve learning outcomes
 Organizations and vendors are facing some critical transitions
• How do old-line database and analytic vendors change their tools to compete
with new approaches (e.g., the Hadoop bandwagon, SAP HANA, etc.)
• How do institutions adopt and take advantage of the new tools? What skill sets
are lacking? What organizational pieces need to be put into place? Can
institutions integrate the data needed?
6

7. What we have done and what we would like to do
 First steps over the past year
• Mobile micro-surveys: Learning from the learner
• Student enrollment, retention, demographics, performance, K-Score, facilities
utilization, instructor workload and more
• High speed, in-memory analytics architectural differences
• Open data and organizational considerations
 Coming down the road?
• Micro-segmentation tool to enhance user and IT productivity, develop
personalized mobile student interaction/intervention
• Models for learner technographics, psychographics, in addition to behaviors,
performance, background
• Advanced way-finding for streaming content like lecture capture
• Content metadata extraction and learner knowledge discovery
• Real-time measures of concept engagement and mastery
• Real-time learner recommendations and support engine
• Use graphing algorithms to perform more sophisticated degree audit what ifs
7

8. 8
Model Description
Enrollment Enrollment in a class, midterm and final grades, credit hours attempted and
earned, instructor teaching the class
Student retention and
graduation
Student demographics and cohort identification (e.g., John Doe is in the 2009
entering first-year student cohort)
Student demographics Demographics, such as age, high school GPA, entrance test scores (SAT, ACT)
and subcomponent scores. Also, in a secure location, additional personally
identifiable demographic details such as name, address, email, etc.
Student performance Present the enrollment data in such a way as to easily show the student’s
performance for each term, including credit hours earned, term GPA, cumulative
GPA for that term, etc.
Student academic career Keep a list of the majors and minors for each student and degrees awarded. Also,
include details on students who transfer in and out, including transfer institution,
credit hours transferred in, etc.
Productivity The room utilization model contains every building, every potential classroom and
lets users analyze the room capacity and enrollments for the class or event in the
room at five minute intervals. The faculty stats per term model pulls together the
number of students and sections taught per term and will contain other important
data such as research expenditures per term and grant proposals submitted and
won.
Micro-surveys Capture questions and answers from the My UK Mobile micro-survey feature
Student involvement Interaction history with various applications including the learning management
system, clickers, course capture and playback, academic alerts. Provide the basis
for calculating the student’s K-Score.

9. My UK Mobile
usage stats
9

10. 10

11. 11

12. Predictive analytics at its finest
Our Ph.D. data scientists locked themselves in a room and worked
very hard on an approach to more reliably predict first-year student
retention from fall to spring term.
What they came up with amazed us…
12

13. A question sent via MyUK Mobile to freshman who are not doing so
well just prior to midterms:
“Do you plan on coming back in the spring?”
13

14. 14

15. Academic Health Notifications:
View in student mobile app

18. 18

19. K-Feed: Intelligent, personalized alerts, news, reminders
19

20. Taxonomy? Automatic metadata? Automatic
atomic metadata?
 Let learners navigate an
audio/visual stream
 Let the system learn what are top
terms. Let the system map terms
to concepts. Let instructional
designers lightly ‘bump’ the
taxonomy, post production
 Record student engagement with
specific terms / concepts
 Deliver personalized messages to
students 20
See http://p.uky.edu

21. University of Kentucky

22. Key questions
• Can the audio and slides be reliably converted into ‘useful’ text?
• Can a concept map be derived automatically from the text generated
or easily edited by an instructor?
• How easy will it be for designers-instructors to create an assessment
and guide its placement in the right location in the video?
• Can we personalize the recommendations to reflect prior knowledge,
student ability and individual differences in information processing?
• Can the interface support real-time integration with high-speed
analytic back-ends (e.g., HANA)?
• Can advising, learning and general support processes be integrated?
• Can this be cost-effective for existing courses?
This is just one conceptualization.
What other interface designs might exist? How effective will they be?

23. Personalize learning and support in one architecture
• Real-time personalized interactions
• Target on-demand peer tutoring based on student’s profile
• Deliver micro-surveys and assessments to capture additional information
needed to improve personalization
• Give students academic health indicators that tell students where they can
improve in study, engagement, support, etc.
• Let students opt their parents in to this information so the family can support
the student
• Tailor and target reminder services, avoid over messaging, enable timing of
message delivery based on user temporal proclivities
• Allow for open personalized learning
• How content gets matched to students is psychologically complex
• Several theories of how humans learn give many insights
• Students differ in the following abilities and attributes: visual-object, visual-
spatial, reasoning, cognitive reflection, need for sensation, need for cognition,
various verbal abilities, confidence, persistence, prospective memory, etc.
• We need an open architecture to promote rapid experimentation, testing and
sharing of what works and what doesn’t
University of Kentucky

24. Is learning analytics a sustainable opportunity?
 Let’s look at the pieces to the puzzle of the value proposition
• We have the expertise, you don’t
– Researchers have been examining many aspects of students success and learning,
but do all vendors have “experts” knowledgeable of the breadth of this literature?
– Research into how the brain learns is yielding much recent insight, but a single
‘theory of mind’ for learning has not emerged, nor will one soon. There is
complexity, counter-intuitiveness and much more to understand regarding how the
brain functions
– While data science is a scarce skill sets, universities, especially research
universities, tend to have these skills
• We have large data across multiple institutions that you don’t
– Does one need large data to gain valid insights? Exactly where does large data
provide learning analytics benefits? For inductive, atheoretical approaches, perhaps
big data can help (e.g., millions of rows of clickstream).
– Do students brains vary that much that very large sample sizes are needed?
– Have we exhausted existing and emerging theoretical approaches?
• We have access to multi-institutional best practices that you don’t
– Good point
24

25. Where are we going? Question 1
 Should knowledge of how students learn be considered a private or
public good?
• Learning analytics are nonrival, that is, if I gain knowledge of how
students learn that does not simultaneously deprive you of the same
opportunity
• Learning analytics are excludable, that is, if I have a piece of software
that collects data on how students learn, I can prevent you from
getting it
 How excludable are learning analytics?
• Most if not all learning analytic companies base their analytics off of
published research. While a single vendor’s knowledge of learning
analytics might be excludable, the ‘prior art’ is often commonly
available
• How easy would it be for a competitor or a customer to reverse-
engineer an approach or design an alternative?
25

26. Questions 2 and 3
 Is making knowledge about how students learn excludable the right thing to do?
• Image a company that has figured out how to improve learning for 90% of human
beings by increasing learning outcomes over any time frame by 100% while maintaining
or reducing costs of instruction? Suppose this approach is available at the following
price: $75,000 per student
• Who gets to benefit? Who does not?
 How would you feel about the same scenario, but now regarding a difficult, life-
saving, heart surgical procedure?
• While in the U.S. medical procedure patents have been permissible, many countries ban
them. Most, if not all medical procedures are based on ‘prior art.’ Many in the medical
community are opposed to these types of patents as they can interfere with educating
doctors, and impede public health objectives
 What does this mean for vendors?
• Learning analytic procedures (algorithms) would have to be free from reliance on ‘prior
art’ which might be difficult for most vendors
• While copyright law can protect the software written regarding learning analytics,
copyright law cannot compel anyone from revealing what aids learning
• Vendor goals and university goals are not always aligned
26

27. Is this something to worry about?
 Perhaps not
• Cognitive psychology, neuroscience and learning theory are rapidly evolving. Recent
brain imaging and sensor advances are expanding knowledge quickly
• How humans learn is amazingly complex, and even harder to apply in ‘single event
cases.’ How many of you have family in college you just can’t seem to help?
• Universities can easily create ‘open source’ versions of learning analytic tools, and
sharing specific knowledge about their students with others
• As learning analytic knowledge diffuses, universities will then shift the competitive effort
on to those activities that their faculty and staff perform (skill of the doctor) versus
gaining access to the analytically-power learning tool (medical procedure)
• We live in a connected world. Countries might demand learning insights be public goods
hurting companies relying on keeping knowledge excludable. Research globally can
undermine vendors locally
 Perhaps so
• Universities sometimes move in haste and en masse, and don’t have the expertise to
build their own tools, thus will be reliant on vendors
• Vendors, by design, are motivated to make data and insights into data excludable
• Regulation in a single nation can encourage further privatization of education, thus
‘locking up’ insights into things universities must purchase (and not generally use)
27

28. Scarce / Not scarce
 Scarce
1. Management ability to know how to build an organization to take
advantage of analytics
2. Enterprise architecture and data science skills
3. Ability to integrate from disparate sources quickly
4. Order of magnitude improvement in cost-effectiveness
 Not scarce
1. Ideas for analytics
2. Raw data (dark data)
3. Tools
4. Willing students
28

29. Déjà vu?
29
http://www.thelongtail.com/conceptual.jpg
MOOCs
Large
lectures
PHI 698
???

30. Questions?
30
http://www.independent.co.uk/life-style/gadgets-and-tech/features/from-fighter-jets-to-google-glass-headup-displays-make-the-jump-to-mainstream-gadgets-8854485.html