Presentation at Qatar Computing Research Institute, 13/11/2017. Read more: https://persona.qcri.org

1. Automatic Persona
Generation (APG)
News & Social Media Analytics Team
Social Computing Group
Qatar Computing Research Institute
Hamad Bin Khalifa University

2. Information
architecture: Choosing
the correct information
elements and layout for
a given user or industry.
Comments: Finding
representative, relevant
and non-toxic comments
describing the persona.
Evaluation: Validating
accuracy, consistency,
and usefulness of
personas for individuals
and organizations.
Topics of interest:
Classifying topics of online
content and discovering
probable interests across
social media platforms.
APG: Platform for research
Description: Generating
fluent text descriptions of
the personas.
Discovering better ways to computationally process
and choose useful representations from vast
amounts of online data (”giving faces to data”).
Image: Using neural
networks to generate
persona profile pictures.
Story selection: predicting
and choosing content for
personas or content
creators.
Temporal analysis:
Observing change in
personas over time.

3. Information
architecture: Choosing
the correct information
elements and layout for
a given user or industry.
Comments: Finding
representative, relevant
and non-toxic comments
describing the persona.
Evaluation: Validating
accuracy, consistency,
and usefulness of
personas for individuals
and organizations.
Topics of interest:
Classifying topics of online
content and discovering
probable interests across
social media platforms.
APG: Platform for research
Description: Generating
fluent text descriptions of
the personas.
Discovering better ways to computationally process
and choose useful representations from vast
amounts of online data (”giving faces to data”).
Image: Using neural
networks to generate
persona profile pictures.
Story selection: predicting
and choosing content for
personas or content
creators.
Temporal analysis:
Observing change in
personas over time.

4. Validating personas
1. Personas as ”objective truth”
2. Personas as ”subjective truth”
For objective truth => accuracy metrics,
predictive ability, stability analysis of personas
For subjective truth => case studies,
ethnography, surveys

5. Validating personas
1. Personas as ”objective truth”
2. Personas as ”subjective truth”
For objective truth => accuracy metrics,
predictive ability, stability analysis of personas
For subjective truth => case studies,
ethnography, surveys

6. Persona Perception Scale:
Toward systematic measurement of personas
Research Objective: Personas are notoriously difficult to
validate and evaluate (Chapman & Milham, 2006). As fictive
user representations based on real data, personas can be seen
subject to interpretations from end-users of personas. These
interpretations can be multi-dimensional and include
elements such as empathy, perceived accuracy, and liking.
We develop a persona perception scale and validate it among
business professionals.
Chapman, C. N., & Milham, R. P. (2006). The Personas’ New Clothes: Methodological and
Practical Arguments against a Popular Method. Proceedings of the Human Factors and
Ergonomics Society Annual Meeting, 50(5), 634–636.

7. Persona Perception Scale:
Toward systematic measurement of personas
Plan: Develop the scale, validate with pilot
study, and then conduct a large-scale survey for
persona users. Once the scale has been
published in journal, it can be referred to in
other studies.
How does a system change affect persona perception?

8. Research process
1. Selecting dimensions and items => Literature review
2. Pruning them down => Expert feedback (content
validity)
3. Pruning them down => Pilot survey
4. Validating the scale => Large-scale survey (N > 300)
5. Publishing
6. Measuring impact of system changes to persona
perception.

9. Research process
1. Selecting dimensions and items => Literature review
2. Pruning them down => Expert feedback (content
validity)
3. Pruning them down => Pilot survey
4. Validating the scale => Large-scale survey (N > 300)
5. Publishing
6. Measuring impact of system changes to persona
perception.

10. Tips for scale development

11. Construct
Dimensions
Items
Remember: you’re trying use
questions to reveal a hidden thing
you can’t directly observe

12. Remember: you’re trying use
questions to reveal a hidden thing
you can’t directly observe
Persona Perception
Construct
Credibility
Consistency
Completeness
x1
x2
x3
x4
x5
x6
x7
Dimensions
Items

13. Tips for scale development
1. Start by collecting more items than you need in the end
2. Most sources seem to suggest a 7-point scale (but you
can use a 5-point scale as well)
3. Figuring out what you’re measuring is important (e.g.,
what is ’persona perception’?)
4. ”Would people answer these items in a similar way?”
(simulate factor loadings in your head :)
5. Wording matters a lot…

14. 1. Don’t mix positive and
negative statements
For example,
”This persona seems useful”
”I don’t think this persona is very useful.”
 Better is to have all the questions pointing at the
same ”direction”.

15. 2. Avoid long sentences and
complicated words
Even your mother should be able to answer with ease!
For example,
”I think I would have no trouble engaging in exhilarating
discussions with this particular persona.”
 ”I would enjoy talking to this persona.”

16. 3. Avoid redundancy
(=asking the same thing twice)
For example,
”The persona seems like a kind person.”
”I would see this persona as a kind individual.”
(However, the initial pool of items can have redundancy.)

17. 4. Avoid combining two or more
statements in one sentence
For example,
”This persona seems credible because it has the right picture and
text content.”

”This persona has credible picture content.”
”This persona has credible text content.”

18. Checklist to consider for each
construct/item
• Is it relevant? ✓
• Is it simple? ✓
• Is it clear? ✓
So that
1 = Not relevant
2 = Needs some revision
3 = Needs minor revision
4 = Relevant
 Can be used for calculating
content validity index (CVI)

19. Good constructs give you basis
for hypothesis testing
E.g., structural equation modeling (SEM)
Persona
Perception
Credibility
Consistenc
y
Completen
ess
x1
x2
x3
x4
x5
x6
x7
Usefulness
Industry
Use case
Willingness to
use

20. Too many tips to share…
A good reference book:
DeVellis, R. F. (2016). Scale Development: Theory and
Applications (4 edition). Los Angeles: SAGE
Publications, Inc.
Best is to try it out yourself!

21. And now… survey time!
Please go to: __________
(also have print copies if someone doesn’t have a laptop.)

22. The beautiful team :)
Prof. Jim Jansen Dr. Jisun An Dr. Haewoon Kwak
Dr. Joni SalminenMSc. Soon-Gyo Jung