xAPI-Enabled Mobile Health System with Context Awareness Recommendation Engine for Patients
•Download as PPTX, PDF•
1 like•312 views
xAPI Camp Chicago at Lurie Children's Hospital presentation by Jessie Chuang
Recommended
Recommended
More Related Content
Similar to xAPI-Enabled Mobile Health System with Context Awareness Recommendation Engine for Patients
Similar to xAPI-Enabled Mobile Health System with Context Awareness Recommendation Engine for Patients (20)
More from Megan Bowe
More from Megan Bowe (20)
Recently uploaded
Recently uploaded (20)
xAPI-Enabled Mobile Health System with Context Awareness Recommendation Engine for Patients
- 1. XAPI-enabled Mobile Health System with Context- Awareness & Recommendation Engine for Patients JESSIE CHUANG CLASSROOM AID INC. VISCA ANALYTICS XAPI CHINESE COP
- 2. by KenPan
- 4. Background and Motivation (1/2) Fact Mobile Health（mHealth）flourished. (Saxon, 2016.) 150 Health 60% Other 40% Apple Research Kit & Care Kit Google Fit SDK
- 5. Background and Motivation (1/2) Related Monitoring physiological value Health Information (Text, Picture, Video) Fact Mobile Health（mHealth）flourished. (Saxon, 2016.) Mostly focused on providing information and monitoring physiological value. (Vashist, et al, 2014. Saxon, 2016.)
- 6. Background and Motivation (1/2) 1. We need to monitor patients daily activities and motions of upper limb. (Subbarao, et al, 1995.) Gap 92% 50 - 70% Fact Related Mostly focused on providing information and monitoring physiological correlation value. (Vashist, et al, 2014. Saxon, 2016.) Mobile Health（mHealth）flourished. (Saxon, 2016.)
- 7. Background and Motivation (1/2) 2. Applications provided for Spinal cord injury patients only taught health information, but lack of monitoring and management. Solution 1. Develop an intelligent assistance system - SmartChair APP 2. Proposed Context Awareness Suggestion Engine (iCASE) Fact Mobile Health（mHealth）flourished. (Saxon, 2016.) Mostly focused on providing information and monitoring physiological correlation value in Health APPs. (Vashist, et al, 2014. Saxon, 2016.) Gap Related 1. The need for patients daily activities and rehabilitation of upper limb motion should be monitored. (Subbarao, et al, 1995.)
- 8. Background and Motivation (2/2) SmartChair APP (Dept Engineering Science, NTU) Motor power wheelchair (Dept Mechanical Engineering, NTU) Physician, occupational therapist (Dept Occupational Therapy, NTU) Spinal cord injury (Taoyuan Potential Development Center, National Taiwan University Hospital) Data Prescription
- 9. Objective Identify problems, dynamic correction, improve the system. System Developer User Related Staff Implement a mHealth APP for Patients with SCI. Purpose
- 10. Objective Click：1 Click：2 Click：3 Click：4 System Developer User Related Staff Implement a mHealth APP for Patients with SCI. Issue Frequent clicks will result in chronic injuries. Purpose
- 11. Objective Click：1 Click：2 System Developer User Related Staff Implement a mHealth APP for Patients with SCI. Issue Frequent click will result in chronic injuries. Purpose Solution Machine builds context-awareness from user history, prescription, & other contexts and prompts recommendations dynamically.
- 12. Objective Solution Using Experience API (xAPI) , data can be transferred between different services System Developer User Related Staff Implement a mHealth APP for Patients with SCI. Purpose IssueIssue Collect data from different services
- 13. System Developer User Related Staff Objective System System’ Collection Analysis FeedbackRevision System System’ Collection Analysis Feedback Revision Stable? Y N Implement a mHealth APP for Patients with SCI. Purpose System revision is usually inefficient and time-consuming. Issue
- 14. Objective Solution Add "Recommended Interface” to the current architecture to get feedback. (A/B testing) System Developer User Related Staff Implement a mHealth APP for Patients with SCI. Purpose System rebuilding is usually inefficient and time-consuming. Issue
- 15. Methodology Collection Analysis Technique Server Level Client Level The actual usage can not be completely recorded. Server Log Access easily to information, but can not collect JavaScript event. (Srivastava, 2000.)
- 16. Methodology Technique Server Level Client Level Access easily to information, but can not collect JavaScript event. (Srivastava, 2000.) Server Log Direct Access Intermediary Server-Side There will exist the problem of grammar incompatibility in migration. (Corbi and Burgos, 2014.) Leverage Escrow Services to avoid grammar migration issue. Software / APIs Collection Analysis
- 17. Methodology Technique Server Level Client Level Server Log Direct Access Intermediary Server-Side xAPI Since the Experience API (xAPI) is an open standard, so it is used as Intermediary Server. Collection Analysis Access easily to information, but can not collect JavaScript event. (Srivastava, 2000.) There will exist the problem of grammar incompatibility in migration. (Corbi and Burgos, 2014.) Leverage Escrow Services to avoid grammar migration issue.
- 18. Methodology Technique xAPI Client Level Through Escrow Services to avoid incompatibility problem with the grammar migration. Intermediary Server-Side xAPI Cross- platform Use the Activity Streams to record user experience. Actor (Who) Verb (How) Object (What) Collect and transfer data between heterogeneous platforms through Learning Record Store (LRS). Context description Parameters for different situations can be recorded as context data. Collection Analysis Data integrity
- 19. Methodology Context- Awareness Collect user behavior through xAPI Definition Categories Dynamic a person, place or object. (Dey, et al. 2001.)Static Actor behaviors (Actor, Verb, Object) (G. Chen and D. Kotz, 2000.) i.e. user profile, location(G. Chen and D. Kotz, 2000.) Collection Analysis Computing context User context Physical context Time context i.e. network connectivity i.e. time of a day, week i.e. lighting, noise level User context Time context
- 20. Methodology Methods Statistics Sequential Pattern Statistical inference (frequency, average, etc.) is the most popular. Investigate the probability that when an event appears, another event also appears. Classify old data, and then predict the future data. Cluster the data by property similarity. Analyze data pattern on timeline. Statistics Sequential Pattern Context- Awareness Collect user behavior through xAPICollection Analysis Association Rule Clustering Classification
- 22. Filter Model Context- Awareness Model Expert Knowledge Cost-Benefit Analysis iCASE xAPI Therapist System DeveloperRecommended Interface iCASE system Data Flow System BlocksContext-Awareness Model xAPI Behavior Model xAPI format Mapping Interface Segmentation Block Naming Behavior Library (LRS) Behavior Model
- 23. Context-Awareness Model xAPI Behavior Model xAPI format Mapping Block Naming Behavior Library (LRS) Behavior Model: Collecting User Data through xAPI VIPS algorithm (Microsoft, 2003.) Interface Segmentation
- 24. Context-Awareness Model xAPI Behavior Model xAPI format Mapping Interface Segmentation Behavior Library (LRS) 23 Behavior Model: Collecting User Data through xAPI Discomfort Record Activity Record Route Record Exercise Exercise Time Block Naming
- 25. Context-Awareness Model xAPI Behavior Model Interface Segmentation Block Naming Behavior Library (LRS) Behavior Model: Collecting User Data through xAPI xAPI Verb xAPI Object viewed experienced modified recorded Discomfort Record Activity Record Exercise … xAPI format Mapping Discomfort Record Activity Record Route Record Exercise Exercise Time
- 26. xAPI Verb xAPI Object viewed experienced modified recorded Discomfort Record Activity Record Exercise … Context-Awareness Model Behavior Model xAPI format Mapping Interface Segmentation Block Naming Behavior Model: Collecting User Data through xAPI Event： { “Actor”：”John Lee”, “Verb”：”recorded”, “Object”：”Discomfort Record” } Behavior Library (LRS) … xAPI
- 27. Filter Model Expert Knowledge Cost-Benefit Analysis iCASE xAPI Therapist System DeveloperRecommended Interface iCASE system Data Flow Behavior Model Context-Awareness Model Filter Model Behavior Model Context Analysis Time Context User Context …… Context Definition Context-Awareness Model Context- Awareness Model Behavior Model
- 28. Context- Awareness Model Expert Knowledge Cost-Benefit Analysis iCASE xAPI Therapist System DeveloperRecommended Interface iCASE system Data Flow Filter Model Sort Results Rule filtering engine Filter Model Context- Awareness Model Behavior Model Filter Model
- 29. Behavior Model xAPI Recommended Interface Sort Results Rule filtering engine Filter Model Context-Awareness Model Filter Model + Expert Advice # System functions Therapist prescription 135347 Route Distance 5 - 10 km 116170 Frequency of Wheelchair Repair At least once a month 99242 Rehabilitation Three times a week Hash Table Expert Knowledge Therapist
- 30. # System functions Therapist prescription 135347 Route Distance 5 - 10 km 116170 Frequency of Wheelchair Repair At least once a month 99242 Rehabilitation Three times a week Sort Results Rule filtering engine Filter Model Filter Model Adjust the weight to improve the bad habits. Time Frequency Therapist prescription (Threshold) Forcibly removed Time Frequency Therapist prescription (Upper limit) Forcibly removed Therapist prescription (Lower limit) 1) User habits 2) Increase the weight Reduce the weight
- 31. Expert Knowledge Therapist Behavior Model xAPI Sort Results Rule filtering engine Filter Model Context-Awareness Model L0.5 L1 L0 Recommend L2 L3 Login functio n1 Index functio n1.1 functio n1.3 functio n2 functio n3 functio n2.1 functio n2.2 functio n3.1 functio n3.2 functio n3.3 functio n1.2 Ln：Level n；n： clicks required to access the function 1) Sequentially outputs. 2) Until satisfy the size of the recommended list. Recommendation Interface Recommended Interface
- 32. xAPI Behavior Model xAPI format Mapping Interface Segmentation Block Naming Behavior Library (LRS) Future works (1/2) Join other devices to increase data integrity. Thoracic Vertebrae Cervical Vertebrae Lumbar Vertebrae Sacral Vertebrae User Injuries Level Feature 1 C4 No upper limb movements. Use the chin to operate APP. 2 C5 Only the elbows bent, not straight action. 3 C6 The elbow can straighten and wrist cock. 4 C6 The elbow can straighten and wrist cock. 5 C7 The fingers weakness. 6 T11 The upper limb is fine.
- 33. Context Analysis Time Context User Context …… Context Definition Context-Awareness Model Future works (2/2) Association Rule Clustering Methods Classification Parameters Contexts Computing context User context Physical context Time context Add more context data to improve the accuracy of the recommendation. Social context: peers, physician & caregiver with patient
- 34. Related work – Food Control for Cancer Patients LRS Behavior Prescription E-health brain xAPI E-Health System Human-in-the-loop Machine Learning : machine is human’s collaborator.
- 35. Physiological sensors : blood pressure; blood glucose level; temperature; blood oxygen level; and the signals related to ECG, EEG, and EMG. Biokinetic sensors : to measure the acceleration and the angular rate of rotation that results from body movements. Ambient sensors : to measure environmental factors such as temperature, humidity, light, and the sound pressure level. Self-reporting : alarm, habit, discomfort recording, survey, check- list, request help. Patient-centered “Sensor Network” XAPI records rich CONTEXT information, which is crucial for medical data.
- 36. Serve Humanity ASAP XAPI data are highly structured in a pre-designed way, can be integrated meaningfully as soon as collected, data can be put to use right away. for human to read, for machine to compute & respond (less guess), services can talk to each other & work together in real time !! (If … then … ) If data talk in different languages, we can NOT make sense out of them or use them UNTIL the time and computing power are committed to integrate and interpret them.
- 37. What if… It takes you 3 days to “see” the woman standing in front of you… Will your response be way too late…?
- 38. Dataviz as a Cognitive Agent
- 39. Takeaways XAPI is a very effective tool in enabling Apps to serve humanity ASAP, because it connects heterogenous data immediately. XAPI is about people working together. XAPI is about connecting current technologies, instead of re-inventing wheels.(API’s power) @classroomaidinc Jessie@classroomaid.org
Editor's Notes
- My first question : How many have watched this movie “Big Hero 6”? In case you don’t know, it’s a fun Disney movie, and Baymax is a personal health companion – a huggable robot that will take care of you, always. Second question: If possible, do you want to give your family member who’s sick one personal health companion? My boss of my first job taught me “you must tell the conclusion at the beginning of your report”, so here is my conclusion – xAPI is a very effective tool to help us build a Baymax! Of course I’ll tell you why later.
- A little about us…in short, Classroom Aid is our consulting company, I have been working on xAPI project for 3 years, Visca is our effort to give real-time dataviz for xAPI data, we believe returning data to learners and instructors immediately is crucial. We even provide API for you to embed dataviz inside your own App or dashboard. I am also the lead of xAPI Chinese CoP, Here are our major CoP members. We set up our AcrossX profile, recipes and vocabulary for our CoP. BTW, profile and recipe are to define how we record specific experiences. xAPI is totally different with SCORM. xAPI spec. only guarantees data structure interoperability, but it’s highly extensible and flexible, this is great, but semantic interoperability must rely on people working together to decide how to record a behavior, that’s why xAPI depends on CoP and DISC. This extensibility and community driven vocabulary/profile/recipe is also why xAPI is so powerful. Here comes the 2nd conclusion: successful xAPI implementation is about “people working together”, among the lessons learned along the way trying to solve all difficulties in implementing xAPI, this is the no. 1 factor. Today’s story is from ICAN Lab of National Taiwan University.
- According to research, average Americans check their mobile phones 150 times a day. 60% is for health purpose, like tracking fitness goal, they said they will use health App more in the next 5 years…. we all agree mobile phone are now our most intimate friend. Apple offers ResearchKit and CareKit for healthcare support and research, and Google has Google Fit SDK, too. Now hospitals also use mobile portal to engage and serve patients.
- 2/3 of mHealth App only provides health information, 1/3 is to monitor physiological values, it’s not enough for special need patients.
- 92% of spinal cord injury patients need to use wheelchairs，within 10-15 years, 50-70% have upper limb overuse problem, the probability of chronic illness increases significantly.
- The solution is to develop Smartchair App that provides instructional videos of rehab. For patients to follow along, medical knowledge for this type of patients. It also include social media platform for patients to connect with peers. From the data collected, the system behind it, we called Context Awareness Suggestion Engine(iCASE), builds context-awareness and then provides recommendations dynamically to support patients. .
- At the center is Smartchart App, data from the wheelchair and the prescription from physician or therapist are sent here to integrate with data collected by Smartchair. This App is now offered to spinal cord injury patients in these 2 organizations.
- Let’s identify problems to be solved regarding these 3 aspects. Notice that this is how we should use xAPI – pinpoint problems first, not sending whatever xAPI data we can send without purposes in mind.
- For user’s need, frequent clicks will result in chronic injuries.
- The solution is to build context-awareness from user’s history records, the prescription and other context data to prompt the recommended action for user. This reduces clicks and provides guidance at the same time.
- The 2nd issue is we need to communicate between different services – for example, the site for therapists, and wheelchair vendor’s system, in order to solve this, xAPI is leveraged for data transfer and integration.
- And, system developers need a more efficient way to collect user behavior to feedback to improve the system. Also after revision, users also need effort to re-learn newly-designed interface. Is there a more flexible and agile way?
- The solution is a recommendation engine to prompt recommended action for supporting users as well as collecting feedback, developers will update the system design only after the hit rate of recommendation is higher than a threshold.
- Smartchair is a hybrid App with its UI based on web language, here we discuss how to collect interaction data.
- XAPI is a user-centered data format, use Actor-Verb-Object along with other rich contextual and result data, the experience can be captured with necessary details. You might know better than me that for medical data, context information is crucial. This way allows collecting and transferring data in between heterogeneous platforms through intermediary server – the Learning Record Store, LRS.
- Now let’s look at context-awareness modeling. Contexts could be static or dynamic, and belong to many categories that matter to our analytics. You can imagine sensors are like our 5 senses, and computing is like our brain. Only with xAPI, we can record heterogenous data in a coherent way, so those 5 senses data can be put together immediately for computing brain to build context-awareness. For example, it’s like Baymax observes when, how long, with whom, the patient did exercises.
- So how does computer model data? it’s like Baymax can reason that the patient walk longer when it’s morning than afternoon, or exercise longer when exercising together with peers. There are many well-developed data science / machine learning methods and even open-source tools to let machine do reasoning and predictions. For 1st phase, statistics and sequential pattern are used to model the behavior data here.
- This is the conceptual system architecture, mostly you can have an idea about how computer “brain” works to process data to support users. xAPI collects data from functionalities interface into building behavior model, which will be sent to context-awareness model for calculation, and then combine with expert knowledge, these data will be processed by filter model to sort out suggested next steps and offer to the patient. The cost-benefit analysis is to analyze if this recommendation engine really improve user experience and solve the issue.
- This block “behavior model” is where xAPI works. So let’s go into more details here.
- This is a way to model interaction data from this mobile App – first, interface segmentation.
- 2nd, block naming, it’s suggested to make it meaningful for both humans and machine to read.
- 3rd, map and design xAPI statements
- A simple example is like this : John recorded discomfort record. That’s the most basic sentence of an xAPI statement, along with it context information can be recorded as well – such as timestamp (when did this happen), location (where did this happen), result (what is the discomfort level), or even the environment temperature.
- Then user’s behavior records will be processed along with context awareness model, using those mentioned machine learning methods to model data, also taking into consideration of contexts such as current time and location.
- This filter model is to filter and sort the recommended next steps for the user.
- Usually this kind of recommendation engine only factors in user’s habits, but for a patient, we need to put therapist’s prescription into filter model. All recommended actions are referring to the prescription list, the priority is adjusted according to user’s history and context.(for example, time)
- It adjusts the weight for recommending actions according to user’s history data, for behaviors with lower frequency, the weight will be increased, for behaviors with higher frequency, the weight will be reduced, or totally remove an action if the frequency exceeds the threshold.
- The recommendation engine prompts recommended action at 0.5 layer, on top of the current hierarchy. As I mentioned, this reduces clicks, and provides guidance as well.
- Future work: we can add voice recognition and voice control, and wearables for monitoring purpose, for example, measuring quantified data of patient’s movement, or monitoring eyeballs, brainwave…. Depending on the pinpointed purposes.
- If we have more sensors, those data might help us get more context information, then we can answer more questions. But some context data will be collected by designing the functionalities such as enabling interactions between patients and caregivers, physicians, and we can analyze the impact of such social context.
- Based on the same system architecture, this related work is to help cancer patients choose the right foods and balanced between preferences and nutrition requirements. Actually another related work undergoing is in learning and training domain, it uses the same system architecture too. The strategy here is the collaboration relation between machine and experts (therapist or teacher). First the machine builds recommendations from expert’s prescription and user’s behavior history to balance both dynamically, the output of machine learning is under monitoring of expert, the expert can modify the recommendation if needed, and then machine will adjust its modeling to fit to that label every time. That means, machine learns from its human collaborator and real data continuously. The benefit for experts to use this system is that collected data can give them a better picture about their patients or students, they can interact with them whenever needed; and with enough training, machine’s recommendation can be very helpful and share the expert’s work loading.
- Now we can have tons of medical devices and sensors to collect data around a patient, even swallow a pill and take pictures inside the body. Data, data, data everywhere, 2 things we care here. As mentioned before, for medical data the related contexts of a data point is very important, and xAPI can record that in a standard way.
- 2nd, now all these data talk in different languages…
- Can you imagine? What if… But each data from different sources are like a piece of the puzzle, if without xAPI, we can not see the real-time whole picture.
- Here we go. Dataviz can help you see the whole picture of integrated xAPI data IN REAL TIME, zoom in to see a tree or zoom out to see a forest, for learners, instructors, patients, family, and therapists.
- 3 takeaways: … … Usually driven by people who care about patient-centered or learner-centered service. Then talk to an xAPI expert about your questions needed to be answered, work together to design xAPI implementation. To reduce technological barrier, xAPI wrappers and connectors are available. … maybe you’ve heard of API economy, XAPI is API for experience data, if with proper xAPI profile/recipe design, it can be plugged with machine learning APIs or our iCASE brain as introduced in this presentation. Think of the fact that we have such “personalized advertisements” from Google and Facebook etc., can we bring that “personalization” technology to learners and patients? Up to all of us.