The document discusses how Elasticsearch and Kibana can be used to build embedded, interactive learning dashboards. It presents a case study of integrating Elasticsearch and Kibana into a learning platform called GRAASP to enable analytics on user interactions. User events are captured using xAPI and indexed in Elasticsearch for analysis. Kibana is then used to visualize interactions and create dashboards that provide insights for various user types. This embedded analytics approach allows contextualizing insights within the interaction environment to promote awareness and reflection.

1. EMBEDDED INTERACTIVE
LEARNING DASHBOARDS WITH
ELASTICSEARCH AND KIBANA
The copyright of images belongs to their authors. Drop me a message at andrii@vozniuk.com to remove
Talk description: http://www.meetup.com/elasticsearch-switzerland/events/233201724/
ANDRII VOZNIUK
REACT-EPFL
Elastic Meetup SwitzerlandLausanne, September 2016

2. GOALS
OF THE TALK
• Present the case of using Elasticsearch & Kibana to
build learning dashboards
• Demonstrate the interactive analytics tools in action
• Motivate thinking on how the tools can be useful in
your particular cases

3. HAVE YOU EVER USED
ELASTICSEARCH OR
KIBANA?

4. ANDRII VOZNIUK
https://about.me/vozniuk
RESEARCHER
CS PhD Candidate
SOFTWARE ENGINEER
Web, Mobile, Data
ENTREPRENEUR
Knowledge Sharing Systems
TODAY
Data Engineer

5. BACKGROUND

6. REACT - EPFL
L AU S A N N E

7. REACT =
COORDINATION &
INTERACTION
SYSTEMS GROUP
We Research Computer-Mediated Interaction
And Build Interaction Systems
Focus On
• Educational Technology
• Humanitarian Knowledge Sharing

8. SPEAKUP
is a co-located social
media to improve
audience interaction
speakup.info

9. GRAASP
a social media as a personal and collaborative
learning environment
graasp.net
graspeo.org

10. GRAASPBOX
facilitating
knowledge delivery
into underconnected
environments

11. GRAASP IS
A MEAN WEB APP
M MongoDB
E Express.js
A AngularJs
N Node.js
Front-end
mongoose
express
Server
Database

12. WE USE ELASTIC
TOOLS A LOT
SEARCH
RECOMMENDATIONS
NLP
CONTENT ANALYTICS
INTERACTION ANALYTICS
MONITORING

13. GRAASP SHORT DEMO

14. LEARNING ANALYTICS &
DASHBOARDS

15. Allow platform end-users to gain insights by
exploring the Interactions
TEACHERS
STUDENTS
KNOWLEDGE MANAGERS
CONTENT CREATORS
FOR AWARENESS & REFLECTION
THE GOAL OF
EMBEDDED
INTERACTIVE
DASHBOARDS

16. APPROACH

17. ELASTICSEARCH
MAPS
INDEXES
KIBANA
VISUALISES
INTERACTIONS
ANALYTICS PIPELINE
PLATFORM
CAPTURES
INTERACTIONS

18. ANALYTICS PIPELINE
PLATFORM
CAPTURES
INTERACTIONS

19. LOGS ARE
NOT THE
BEST FORMAT
FOR ANALYSIS

20. ESPECIALLY,
WHEN
COMING FROM
MULTIPLE
SYSTEMS

21. GETTING
TIMESTAMP
SHOULD
BE EASY, RIGHT?
[19/FEB/2015:19:00:00 +0000]
1424372400
[2015-02-19 19:00:00,000]
FEB 19 19:00:00
2015-02-19T19:00:00+02:00
APACHE
UNIX TIMESTAMP
LOG4J
POSTFIX.LOG
ISO 8601
OVER 40 FOMATS

22. CAPTURING
INTERACTIONS
WITH xAPI

23. WHAT IS THE
PURPOSE OF
xAPI?
xAPI = ExPERIENCE API
It is a standard to capture in
a unified way experiences
of the user, in our case
user-tool interactions

24. HOW DOES xAPI
WORK?
Users interact with tools
These interactions are observed and recorded
by the tools as xAPI statements
The tools [store] and send the statements to a
central system (Learning Record Store or LRS) for
further usage, for instance, Analysis

25. XAPI
Source: http://www.slideshare.net/aaronesilvers/going-with-xapi

26. TIMESTAMP
ACTOR
VERB
OBJECT
RESULT
CONTEXT
xAPI FORMAT
SPECIFICATION
WHEN?
WHO?
DID WHAT?
WITH WHAT?
WITH WHAT RESULT?
IN WHICH CONTEXT?
Today at 10:15
Andrii
answered
the question five
with the grade four
while it was raining outside

27. INTERACTION
TYPES (VERBS)
ABANDONED
ANSWERED
ASKED
ATTEMPTED
ATTENDED
COMMENTED
COMPLETED
EXITED
EXPERIENCED
FAILED
IMPORTED
INITIALIZED
INTERACTED
LAUNCHED
MASTERED
PASSED
PREFERRED
PROGRESSED
REGISTERED
RESPONDED
RESUMED
SATISFIED
SCORED
SHARED
SUSPENDED
TERMINATED
VOIDED
WAIVED
MESSAGED
INVITED
MENTIONED

28. SAMPLE
XAPI STATEMENT
TIMESTAMP
ACTOR
id
name
mbox
countryname
VERB
id
display
OBJECT
id
definition
name
description
objectType
CONTEXT
ipAddress
location
city
countryCode
countryName

29. ACTIVITY
STREAM

30. ELASTICSEARCH
MAPS
INDEXES
ANALYTICS PIPELINE
PLATFORM
CAPTURES
INTERACTIONS

31. YOU KNOW,
FOR SEARCH
BUT THERE IS MORE
THAN THAT
elastic.co/products/elasticsearch

32. BRINGING DATA TO
ELASTICSEARCH
Front-end
mongoose
express
Server
Database
mongoosastic
mongoosastic
is a mongoose
plugin
updating ES
on mongoose
events

33. ELASTICSEARCH
PROCESSING PIPELINE
MAPPING ANALYSIS QUERYDATA
FLEXIBLE SCHEMA
INTERPRETING FIELDS
DATA INTO TOKENS
TOKENISERS, FILTERS
SEARCH
ANALYTICS

34. MAPPING
TELLS ELASTICSEARCH HOW TO TREAT
THE DATA DIFFERENTLY FROM
RELATIONAL DATABASES,
WHERE SCHEMA IS STATIC
IN ELASTICSEARCH MAPPING IS FLEXIBLE
DEFAULT MAPPING IS CREATED
AUTOMATICALLY
Core types: string, integer/long, float/double, boolean, and null
Other types: Array, Object, Nested, IP, GeoPoint, GeoShape, Attachment

35. ELASTICSEARCH
MAPPING FOR
xAPI
STATEMENTS

36. ELASTICSEARCH
MAPS
INDEXES
KIBANA
VISUALISES
INTERACTIONS
ANALYTICS PIPELINE
PLATFORM
CAPTURES
INTERACTIONS

37. elastic.co/products/kibana

38. KIBANA IS FOR
INTERACTIVE
DASHBOARDS

39. KIBANA
Enables near-real-time visualisation and
analysis of streaming data
Allows interactive data exploration and
supports cross-filtering
Multiple chart types: bar charts, line and
scatter plots, histograms, pie charts, maps
No need to know programming or query
language in most of the cases
It’s open-source, there are extensions

40. ANOTHER EXAMPLE WITH
LIGHT COLOURS SCHEME

41. ANOTHER EXAMPLE
MONITORING MONGODB

42. ELASTICSEARCH
MAPS
INDEXES
KIBANA
VISUALISES
INTERACTIONS
ANALYTICS PIPELINE
PLATFORM
CAPTURES
INTERACTIONS

43. EMBEDDED &
INTERACTIVE
ANALYTICS

44. EMBEDDING DASHBOARD
INTO THE PLATFORM
Front-end
mongoose
express
Server
Database
Kibana
iframe

45. ANALYTICS
EMBEDDED INTO THE
INTERACTION CONTEXT
TRY YOURSELF AT graasp.net

46. DASHBOARD
IN GRASP

47. ELK =
ELASTICSEARCH+
LOGSTASH+
KIBANA
This is an elk as well
SUMMARY
ELK is a powerful analytics
tool that can be integrated
into your platform to create
value for your users

48. LEARNING RESOURCES

49. ELASTICSEARCH
REFERENCE
https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
ELK GUIDES
https://www.elastic.co/guide/index.html
MULTIPLE TUTORIALS
http://www.elasticsearchtutorial.com/
YOUTUBE VIDEOS
https://www.youtube.com/watch?v=Kqs7UcCJquM
https://www.youtube.com/watch?v=wHWb1d_VGp8
https://www.youtube.com/watch?v=1gnpzL9jBqY
https://www.youtube.com/watch?v=SH5hLM2asB8
GO THROUGH IT.
IT IS REALLY GOOD.

50. BOOKS

51. BOOKS

52. QUESTIONS?
FEEDBACK?
about.me/vozniuk
andrii@vozniuk.com
DESIGNED BY LINA BERZHANER