Effective use of personal data is a core utility of modern smartphones. On Android, several challenges make developing compelling personal data applications difficult. First, personal data is stored in isolated silos. Thus, relationships between data from different providers are missing, data must be queried by source of origin rather than meaning and the persistence of different types of data differ greatly. Second, interfaces to these data are inconsistent and complex. In turn, developers are forced to interleave SQL with Java boilerplate, resulting in error- prone code that does not generalize. Our solution is Epistenet: a toolkit that (1) unifies the storage and treatment of mobile personal data; (2) preserves relationships between disparate data; (3) allows for expressive queries based on the meaning of data rather than its source of origin (e.g., one can query for all communications with John while at the park); and, (4) provides a simple, native query interface to facilitate development.

1. Epistenet: Facilitating
Programatic Access & Processing
of Semantically Related Data
Sauvik Das
Carnegie Mellon
University
Jason Wiese
University of
Utah
Jason Hong
Carnegie Mellon
University
sauvik@cmu.edu wiese@cs.utah.edu jasonh@cs.cmu.edu

2. Overview
Motivation
Key features
Motivating Example
Conclusion

3. Goal
Facilitate the programmatic access to
and processing of semantically related
mobile personal data.

4. Goal
Facilitate the programmatic access to
and processing of semantically related
mobile personal data.

5. personal data other data
capturable data about
the user
capturable data not
about the user
call logs
location
browser history
activity
battery level
connectivity
processing load

6. Goal
Facilitate the programmatic access to
and processing of semantically related
mobile personal data.

7. Phone GPS People
Personal data is currently isolated in source-of-origin
silos…

8. Epistenet, instead, semantically structures mobile
personal data in knowledge graph…

9. Goal
Facilitate the programmatic access to
and processing of semantically related
personal data.

10. and provides a unified query interface to access this data.

11. Timestamp > 7/7/15
Attribute Constraint Filter
Ontology => SensorReading
Ontology Constraint Filter
Reference Node
Relative Filters
Related Node
Developers can query for personal data through
expressive filter mechanisms.

12. Usage
EpistenetObject eo = uqi.getMostRecent(
new OntologyClassFilter(OntologyClass.Communication));
Get a random correspondence
EpistenetObject[] rel = uqi.get(
new RelativeFilter(eo).createdWithinRange(5*60*1000L);
Get all other data within 5 minutes of correspondence

13. Overview
Motivation
Key features
Motivating Example
Conclusion

14. Motivation
Mobile personal data underlies many
of the most interesting application
concepts of the present and future.

15. Examples
Smart app launchersIntelligent assistants
Life loggers eHealth

16. Personal data access
Location, calendar,
interests,
contacts
Application usage,
time of day, activity,
communications
Location, time,
application usage
Heart rate, activity logs,
food logs, self-reports

17. Problem
Today, mobile personal data
application development difficult is
difficult because…

18. Queries based on
source of origin, not
meaning
Semantic relationships
between data lost
Data Silos

19. Persistence dramatically
different between log
and sensor data
Programming complexity:
event-based programming
vs. database queries
Sensor / log disparity

20. Data silos Sensor / log disparity
Queries based
on origin, not
meaning
Semantic
relationships
lost
Persistence
vastly different,
lose context
Programming
complexity
Challenges

21. Data silos Sensor / log disparity
Queries based
on origin, not
meaning
Semantic
relationships
lost
Persistence
vastly different,
lose context
Programming
complexity
Challenges
Epistenet
Knowledge
graph
Relative, semantic
and descriptive
query filters
Historical sensor
data around log
data events
Unified query
interface

22. Data silos Sensor / log disparity
Queries based
on origin, not
meaning
Semantic
relationships
lost
Persistence
vastly different,
lose context
Programming
complexity
Challenges
Epistenet
Knowledge
graph
Relative, semantic
and descriptive
query filters
Historical sensor
data around log
data events
Unified query
interface

23. Overview
Motivation
Key features
Motivating Example
Conclusion

24. Key features
Knowledge graph
Unified Query Interface
Relative, semantic and descriptive filters
Histories of sensor and log information

25. Knowledge graph
Epistenet organizes personal data into a single knowledge
graph.

26. Ontology
The Epistenet ontology organizes personal data
semantically. Data providers can contribute personal data
under any ontology class.
Message
Communic
ation
Person
Sensor
reading
Phonecall Location
Accelerometer

27. Knowledge graph objects
Epistenet objects are individual personal data points (e.g., a
phone call, a location ping, a set of accelerometer readings).

28. Knowledge graph objects
Descriptive attributes
Meta attributes
Ontology classes
(e.g., caller, duration)
(persistence, timestamp, permissions)

29. Unified query interface

30. Epistenet provides a single interface to access all personal
data in the knowledge graph.

31. How can it be used?
uqi.get(filter)
.getRandom(filter)
.getN(filter, N)
.getMostRecent(filter)
. . .

32. How can it be used?
uqi.get(filter)
.getRandom(filter)
.getN(filter, N)
.getMostRecent(filter)
. . .

33. Filters
Filters are sieves on the personal data
graph that developers use to isolate
personal data points they want.

34. Attribute-constraint filters
Sender is not unknown
new AttributeConstraintFilter().addNonEqualsConstraint(“sender”,
“unknown”)
Constrain any meta or descriptive attributes of an object.

35. Ontology-constraint filters
OntologyClass is “Location”
Constrain ontology class.
new OntologyConstraintFilter(OntologyClass.Location)

36. Ontology-constraint filters
OntologyClass is “SensorReading” or
any subclass
Constrain ontology class. Can also traverse ontology.
new OntologyConstraintFilter(OntologyClass.SensorReading,
true)

37. Relative filters
Constructs queries relative to other personal data points.
Thus, any personal data point is an index into the
knowledge graph.
Reference Node Related Node

38. + - xunion (or) intersect (and) except (xor)
Filter compositions

39. Historical Sensor Data

40. Ontology-constraint filters
Epistenet also captures sensor histories around logged data
points to bridge the persistence disparity. Also allows
developers to get sensor data from before install-time.

41. Overview
Motivation
Key features
Motivating Example
Conclusion

42. Motivating Example:
Autobiographical Authentication

44. Description
Three steps:
Find a “fact” to ask a question about
Create an answer list
Provide contextual hints

45. Find a fact
EpistenetObject fact =
uqi.getRandom(sms.intersectCompositWith(atTime));
Filter sms =
new OntologyConstraintFilter(OntologyClass.SMSMessage);
Filter atTime =
new MetaConstraintFilter().timestampWithinRange(lt, ht);

46. Create answer list
Filter similar =
new RelativeFilter(fact)
.sameOntologyClass()
.createdBefore();
EpistenetObject[] answers = uqi.get(similar);

47. Provide contextual hints
Filter hintFilter =
new RelativeFilter(fact)
.differentOntologyClass()
.createdWithinRange(10*60*1000L);
EpistenetObject[] hints = uqi.get(hintFilter);

48. Overview
Motivation
Key features
Motivating Example
Conclusion

49. Epistenet organizes mobile personal data into a
knowledge graph.
Epistenet affords easy access to subgraphs of the
knowledge graph via the UQI.
Epistenet bridges the data persistence and query
gaps between sensor and log data.
Conclusion

50. Epistenet: Facilitating Programatic
Access & Processing of Semantically
Related Data
Sauvik Das
Carnegie Mellon
University
https://gitlab.com/scyrusk/epistenet
@scyrusk || sauvik@cmu.edu
http://sauvik.me
w/ Jason Wiese and Jason Hong