1. Converting Paper to Digital
Carolinas Health Informatics Program
Practicum Project
John Horan

2. Carolinas Comfort Scale
Presentation Outline
Introduction of the Carolinas Comfort Scale (CCS)
Project Scope and Requirements
Research and Development
Application Demonstration
Potential Impact of Application
Questions and General Discussion

3. What is it?
• CCS is survey used to help physicians and patients
assess Quality of Life both pre and post hernia repair
with mesh
• Developed by physicians and researchers from
Carolinas Laparoscopic and Advanced Surgery
Program (CLASP)
• CLASP has become a leader in minimally invasive
surgery by focusing on Patient Care, Research, and
Education.

4. How it was developed
• Developed by comparing Carolina Comfort Scale
results against the SF-36 questionnaire which is
considered the gold standard for measuring Quality of
Life for patients who have undergone hernia repair
surgery
• Results of the study published in the Journal of
American College of Surgeons
http://www.ncbi.nlm.nih.gov/pubmed/18387468
• CCS was found to better assess the quality of life
and satisfaction of patients who have undergone
surgical hernia repair better than the generic SF-36.

5. Carolinas Comfort Scale
CSS Usage
The CCS survey is currently
used by surgeons in 19 US
states, 39 Countries, and
has been translated into
multiple languages.

6. Carolinas Comfort Scale
A physician from the United Kingdom will
use the CCS in Nottingham City Hospital
which covers 64 general practices and
provides care to 330,000 patients annually.
The CCS will be used as a support tool for
screening eligibility for inguinal hernia
repair surgery funded by the Nottingham
City Hospital.
French physicians will launch a research
trial using the CCS involving 30 university hospital
teams in France. The study is sponsored by the
French government.

7. Carolinas Comfort Scale
Current Workflow of Data Collection

8. Carolinas Comfort Scale
How the Survey Works
• Patients fill out the paper
based survey
• Question responses are
scored using an algorithm
developed by CLASP
surgeons and researchers
that score five endpoints
• Activity, Sensation, Pain,
Movements and the Total
Score.

9. Carolinas Comfort Scale
SAS Scoring Algorithm Sample
Primary Endpoint #1: Activity Score PROC SORT data= CCS.Responses; by SYMPTOMATIC; RUN;
Compare mean/median scores for each of the 23 questionnaire items (1a, 1b, 2a, PROC MEANS NOPRINT data= CCS.Responses;
2b, 2c, etc) for lightweight vs. heavyweight mesh. This will provide a summary for VAR q2c q3c q4c q5c q6c q7c q8c;
all of the 23 questions for each comparative group. OUTPUT OUT=CCS.AvgMOVEMENT MEAN=m1-m7;
PROC SORT data= CCS.Responses; by CURRENTMESHTYPE; RUN; BY SYMPTOMATIC;
PROC MEANS NOPRINT data= CCS.Responses; WHERE FOLLOWUPTIMEPOINT = ‘POSTOP’;
VAR q1a q1b q2a q2b q2c q3a q3b q3c q4a q4b q4c q5a q5b q5c q6a q6b RUN;
q6c q7a q7b q7c q8a q8b q8c; Primary Endpoint #5: Total Score
OUTPUT OUT=CCS.AvgAll MEAN=m1-m23; Compare mean/median scores for QOL scores collapsed across
by CURRENTMESHTYPE ; pain/sensation/movement & activity for symptomatic vs. non-symptomatic
RUN ; patients. This will provide an overall report based on the mean for all 23
Primary Endpoint #2: Sensation Score questionnaire items.
Broader Endpoint #2: Compare mean/median scores for sensation collapsed across /*Combine result data files from examples #2, #3 and #4. This combination represents
activity (1a, 2a, 3a, 4a, 5a, 6a, 7a, 8a) for symptomatic vs. non-symptomatic patients. all scores collapsed across pain/sensation/movement.*/
This will provide a report that specifically focuses in on the sensation of mesh post PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgPAIN FORCE; run;
implantation pertaining to a specific activity for each comparative group. PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgMOVEMENT FORCE;
PROC SORT data= CCS.Responses; by SYMPTOMATIC; RUN; run;
PROC MEANS NOPRINT data= CCS.Responses; PROC MEANS NOPRINT data= CCS.avgSENSATION;
VAR q1a q2a q3a q4a q5a q6a q7a q8a; VAR m1 m2 m3 m4 m5 m6 m7 m8;
OUTPUT OUT=CCS.AvgSensation MEAN=m1-m8; OUTPUT OUT=CCS.QOLall MEAN=m1-m23;
BY SYMPTOMATIC; by SYMPTOMATIC; RUN ;
RUN; Compare mean/median scores for QOL scores collapsed across
Primary Endpoint #3: Pain Score pain/sensation/movement & activity for symptomatic vs. non-symptomatic
Broader Endpoint #3: Compare mean/median scores for pain collapsed across patients. This will provide an overall report based on the mean for all 23
activity (1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b) for symptomatic vs. non-symptomatic questionnaire items using the Wilcoxon signed-rank test.
patients. This will provide a report that details the level of pain that is experienced PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgPAIN FORCE; run;
by the patients pertaining to a specific activity either pre or post-operatively for PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgMOVEMENT FORCE;
each comparative group. run;
PROC SORT data= CCS.Responses; by SYMPTOMATIC; RUN; DATA CCCS.meanALL;
PROC MEANS NOPRINT data= CCS.Responses; SET CCS.avgSENSATION;
VAR q1b q2b q3b q4b q5b q6b q7b q8b; RUN;
OUTPUT OUT=CCS.AvgPAIN MEAN=m1-m8; PROC NPAR1WAY WILCOXON data=ccs.meanALL;
BY SYMPTOMATIC; 8
WHERE FOLLOWUPTIMEPOINT = ‘PREOP’; CLASS SYMPTOMATIC;
RUN; 7 VAR m1 m2 m3 m4 m5 m6 m7 m8;
Primary Endpoint #4: Movement Score OUTPUT OUT = CCS.wilcoxonresults;
Compare mean/median scores for movement collapsed across activity (2c, 3c, 4c, RUN;
5c, 6c, 7c, 8c) for symptomatic vs. non-symptomatic patients. This will provide a PROC SORT data=ccs.wilcoxonresults; by SYMPTOMATIC; RUN;
report that details movement limitations either pre or post-operatively for each PROC MEANS
comparative group.

10. What CLASP wanted
• An electronic version of the CCS survey
• Electronic version of survey needs to be easily
distributed to current and future users on a variety of
platforms
• To minimize risk survey needs to function as a
standalone application so CLASP does not take on the
risk or expense of data collection and data
management.
• Current users need to have the ability to print PDF
versions of the completed survey with results and have
the ability to export data for patient records and further
analysis.

11. Additional Considerations
• Security of patient information
• Data collection and storage
• Branding of application to bring recognition to the
research and work that CLASP has done to create the
Carolina Comfort Scale
• Ability to control the use and monitor usage of the
application
• Ability to update application as future changes are
developed

12. Additional Considerations Continued
• Decrease turn around time of data analysis
• Make data collection and sharing easier
• Create application that would require very little if any
additional support
• Make application and results generation easy to use
and export for current users

13. Development options
• Research into existing tools and solutions
• REDCap
• Survey Monkey
• WebSurvey Toolbox
• To meet customer requirements separate versions of
the electronic survey would have to be developed for
each platform using several different tools
• Android: Android SDK or Eclipse
• Apple: XCODE
• Web: Eclipse, Netbeans, etc.
• Development costs and management would be
expensive and time consuming if separate platform
development was undertaken

14. Carolinas Comfort Scale
A solution exists to meet all
the needs!
• Adobe leader in development of
Rich Internet Applications
• Using Adobe’s development
tools applications can be
developed and then distributed to
a variety of platforms

15. Carolinas Comfort Scale
Adobe workflow for application development

16. Carolinas Comfort Scale
• Is Flash a good choice for development
of this application?
• Benefits of Adobe Flash vs. HTML5
• Popularity: estimated that world
wide over 90% of internet users
have Flash installed on their
desktop computers
• Many users find it to be easier to
vs. use than other development options
that exist with HTML5
• Drawbacks of using Flash
• Not support on Apple devices,
however Adobe provides several
tools to convert flash based
application for use on IOS

17. Application Development
• A tested and successful design already existed in the paper format of
the survey
• Screen elements were designed in Photoshop and then imported into
Adobe Catalyst. In Catalyst the screen elements were given functionality
and page transitions were defined
• The project was then imported into Flash builder where additional
functionality was added and it was then compiled for testing on the
Blackberry Playbook and Adobe Air platforms
• The customer was sent an Adobe Air version of the prototype
application to evauluate and additional edits to the screen layout were
made
• The Adobe Air and Playbook tablet prototype was then demonstrated
before the Department of Minimally Invasive Surgery at there monthly
team meeting

18. Carolinas Comfort Scale
Proposed workflow for application data
collection and analysis

19. Carolina Comfort Scale
Application Demonstration

20. Results
• Response to the application was very positive
• When prompted for feedback on the application the only question was
could it have the ability to export the data. I had missed demonstrating
the data export screen in my presentation
• When scoring the paper survey the algorithm has to take into account
that patients may neglect to answer a question or that they may select
more than one choice for a question. These issues can be prevented in
the application by limiting question responses to one selection and by
requiring that users answer each question before the results are
generated.
• Converting the survey to electronic forms give the customer multiple
options for distribution and the ability to manage their intellectual
property

21. Results Continued
• Use of the application would allow Physicians to have immediate result
that that they could share with their patients
• Provides patients with additional information
• Streamlines the workflow of data collection and analysis
• Provides an example of how other paper based surveys and forms can
be converted into applications in a cost effective manner
• Next steps develop the full scoring and export functionality into the
application

22. Questions