1. M i S d b U biliMapping Study about Usability
Requirements ElicitationRequirements Elicitation
Yeshica Isela Ormeño, Jose Ignacio Panach

2. Index
1. Introduction
2. Mapping Study Designpp g y g
3. Results
4 Conclusions4. Conclusions

3. 1. Introduction
How to capture functional requirements?

4. 1. Introduction
How to capture functional requirements?

5. 1. Introduction
How to capture interaction requirements?
Create suscriberCreate suscriber
Process suscriberCategory
[ ]>>
g y

6. 1. Introduction
How to capture usability requirements?

7. 1. Introduction
• Usability requirements aim to improve the system
usability.
• The goal of developing usable systems has been dealt
with the Human Computer Interaction (HCI) and
Software Engineering (SE) fieldsSoftware Engineering (SE) fields.
– HCI has proposed several usability guidelines, but they are too
abstract.
– SE has defined sound methods to capture functional
requirements, but usability is relegated to last stages.
I thi i t d i t id tif th i ti• In this mapping study we aim to identify the existing
methods for capturing usability requirements.

8. 1. Introduction
• MS provides an objective procedure for identifying the
t d t t f th h th t i il bl tnature and extent of the research that is available to
answer a particular question.
We aim to identify gaps in current research and to• We aim to identify gaps in current research and to
suggest areas for further investigation.
• In particular we focus on finding works to capture• In particular, we focus on finding works to capture
usability requirements in the MDD methods.

9. Index
1. Introduction
2. Mapping Study Designpp g y g
3. Results
C4. Conclusions

10. 2. Mapping Study Design
• The MP has been defined according to Kitchenham’s
kworks.
• There are previous MP related to usability but none of
them focuses on usability requirementsthem focuses on usability requirements.

11. 2.1 Research Question
• Research question: “What are the proposals to elicit
bilit i t th h t th ftusability requirements throughout the software
development process?”
1 Methods to elicit usability requirements– 1.Methods to elicit usability requirements
– 2.Methods to elicit interaction requirements
– 3.Usability guidelines to elicit usability requirementsy g y q

12. 2.1 Research Question
• Research question: “What are the proposals to elicit
bilit i t th h t th ftusability requirements throughout the software
development process?”
1 Methods to elicit usability requirements– 1.Methods to elicit usability requirements
– 2.Methods to elicit interaction requirements
– 3.Usability guidelines to elicit usability requirementsy g y q
– 4 Tools to support usability requirements elicitation
– 5 Notations to elicit usability requirements
– 6 Empirical validation environment

13. 2.2 Search Strategy
• Search strategy: We have used the following search
IEEE l ACM Di it l Lib S i Li ksources: IEEExplore, ACM Digital Library, Springer Link,
and Science Direct.
• We have used Scopus as search tool• We have used Scopus as search tool.
• The sources explored were the proceedings of
conferences journals books and workshops (from 2000conferences, journals, books, and workshops (from 2000
to 2011).
RESEARCH SEARCH SELECTIONRESEARCH
QUESTION
SEARCH
STRATEGY
SELECTION
CRITERIA
PUBLICATIONS
SELECTION
QUALITY
ASSESSMENT
DATA
EXTRACTION
STRATEGY

14. 2.2 Search Strategy
• Search strategy: The search string is a set of terms to
obtain the publications that answer the research questionobtain the publications that answer the research question.
• Search string = (Usability Requirement) AND (Software
Engineering)Engineering)
– Usability Requirement = (usability requirement OR user
requirement OR usability elicitation OR interaction requirement OR
f ti l OR bilit id li )non-functional OR usability guidelines).
– Software Engineering = (MDD OR model-driven OR MDA OR
notation OR tool OR interface OR engineering OR test).g g )

15. 2.3 Selection Criteria
• Selection criteria: Inclusion and exclusion criteria are
d t i ddetermined
– Inclusion criteria:
• Publications that define how to extract usability requirementsPublications that define how to extract usability requirements.
• Approaches applied to an environment based on MDD conceptual
models.
P blications that define ho to represent the sabilit req irements• Publications that define how to represent the usability requirements.
RESEARCH SEARCH SELECTIONRESEARCH
QUESTION
SEARCH
STRATEGY
SELECTION
CRITERIA
PUBLICATIONS
SELECTION
QUALITY
ASSESSMENT
DATA
EXTRACTION
STRATEGY

16. 2.3 Selection Criteria
• Selection criteria: Inclusion and exclusion criteria are
d t i ddetermined:
– Inclusion criteria:
• Publications that define how to extract usability requirementsPublications that define how to extract usability requirements.
• Approaches applied to an environment based on MDD conceptual
models.
P blications that define ho to represent the sabilit req irements• Publications that define how to represent the usability requirements.
– Exclusion criteria:
• Publications focused on guidelines, notations, and tools whereg
usability has not been considered or has not been included.
• Publications that consider only functional requirements.
• Publications written in a language that is not EnglishPublications written in a language that is not English.

17. 2.4 Publication Selection
• Publication Selection with a systematic process:
– Reading the title and the abstract. From 150 papers, we chose
65 (potential publications).
– Reading the whole publication. From 65 papers, we chose 27Reading the whole publication. From 65 papers, we chose 27
(initial selected publications).
– Searching in references. We added two new publications,
res lting in a total of 29 papers (selected p blications)resulting in a total of 29 papers (selected publications).
RESEARCH SEARCH SELECTIONRESEARCH
QUESTION
SEARCH
STRATEGY
SELECTION
CRITERIA
PUBLICATIONS
SELECTION
QUALITY
ASSESSMENT
DATA
EXTRACTION
STRATEGY

18. 2.5 Quality Assessment
• Quality assessment: to specify the quality of the
l t d bli tiselected publications.
• We use a Likert-questionnaire to specify the quality.
RESEARCH
QUESTION
SEARCH
STRATEGY
SELECTION
CRITERIA
PUBLICATIONS
SELECTION
QUALITY
ASSESSMENT
DATA
EXTRACTION
SELECTIONASSESSMENT
STRATEGY

19. 2.5 Quality Assessment
Subjective Questions 1=Yes 0=Partially -1=No
1. Is the method to elicit the usability requirements clear?
2. Are the guidelines to elicit requirements comprehensible?
3. Are the guidelines to elicit requirements useful in other contexts?
4. Are the publications tools downloadable?
5. Is there a clear case study or example illustrating the proposal?
6. Is the whole proposal empirically validated?
7. Are the results clearly explained?
8. Is the notation to elicit requirements easy to learn?
Objective Questions
1 Has the publication been published in journal or conference1. Has the publication been published in journal or conference
proceedings?
1=Very important 0=Important -1=Not important
2 Has the publication been cited by other authors?2. Has the publication been cited by other authors?
1= More than 4 0=Between 2 and 4 -1=Less than 2

20. 2.6 Data Extraction Strategy
• Data extraction strategy: it consists of classifying the
ibl f h h b tipossible answers for each research subquestion.
RESEARCH
QUESTION
SEARCH
STRATEGY
SELECTION
CRITERIA
PUBLICATIONSQUALITY
DATA
EXTRACTION
SELECTIONASSESSMENT
EXTRACTION
STRATEGY

21. Index
1. Introduction
2 Mapping Study Design2. Mapping Study Design
3. Results
4. Conclusions

22. 3. Results
• SQ1 Methods to elicit usability requirements.
– Existing methods deal with usability requirements together with
other NFR.
– NFR are elicited at the same time the system functionality andNFR are elicited at the same time the system functionality and
architecture are defined.
– A holistic quality model that fits every context does not exist, and
th d t li it NFR l id b i i tmethods to elicit NFR only provide basic requirements
management by means of extensions.
– A few approaches deal with usability requirements at earlypp y q y
stages.

23. 3. Results
• SQ2 Methods to elicit interaction requirements.
– Most methods are based on patterns or models.
– Some methods improve usability by means of applying formal
modellingmodelling.
– However, further work is needed to deal with dynamic
specifications that depend on system functionality.

24. 3. Results
• SQ3 Guidelines to elicit usability requirements.
– There is a huge variety of guidelines.
– The application of these guidelines is not easy.
A few works propose a definition of new guidelines– A few works propose a definition of new guidelines.
– Some representations are based on patterns, templates, or
models and they can be adapted according to which usability
requirements are being captured.
– Nowadays, the guidelines do not provide precise support to
address usability requirements elicitation at the early stagesaddress usability requirements elicitation at the early stages.

25. 3. Results
• SQ4 Tools to support usability requirements elicitation
– Selected publications present tools to support: frameworks,
structured styles, scenarios, notations and methods.
– Some tools incorporate a mechanism of transformation (forSome tools incorporate a mechanism of transformation, (for
example, from task flow diagrams to formal representations).
– A few tools are model-based and they can resolve traceability
b t i t d t tbetween requirements and next stages.

26. 3. Results
• SQ5 Notations to elicit usability requirements:
– More than one notation is usually applied to the development
method.
– The user requirements specifications are usually presented toThe user requirements specifications are usually presented to
end-users in normal text, even though the analyst works with
languages based on models.
S l i t i t t f ti l i t d– Some proposals aim to integrate functional requirements and
NFR in the same elicitation process.

27. 3. Results
• SQ6 Empirical validation environment:
– There is not explicit metrics to evaluate the usability
requirements elicitation.
– Existing validations are focused on quantitative and qualitativeExisting validations are focused on quantitative and qualitative
usability requirements.
– Studied publications are focused on evaluating a few usability
f tfeatures.
– Patterns, scenario management, checklists, work sessions, and
templates are the most common artefacts used to evaluatep
usability.

28. 3. Results
Studied Publications Quality of Publications
13
15
POTENTIAL PUBLICATIONS
SELECTED PUBLICATIONS
12
16
14
16
18
8
6 6
8
7 7
8
5
10
12
6
8
10
12
2 2 2
3
12 1 1
5
3 3 3 3
0
1
0
2
4
Very good Good Bad

29. 3.1 Discussion
• Usability requirements are elicited once all functional
requirements have been capturedrequirements have been captured.
• Existing approaches do not propose a precise and
unambiguous notation to represent usabilityunambiguous notation to represent usability
requirements.
• Guidelines are not defined such a way end-users canGuidelines are not defined such a way end users can
understand them.
• Guidelines are not a holistic support to develop usablepp p
systems.
• The methods are inflexible and require considerable
effort to be applied in contexts that are different from the
contexts where they have been defined.

30. Index
1. Introduction
2 Mapping Study Design2. Mapping Study Design
3. Results
4. Conclusions

31. 4. Conclusions
• We have explored the development methods that
consider usability as a requirement.y q
• A total of 29 publications were selected from an initial set
of 150 publications returned by the search string.
• 97% of selected publications have a good quality.
• If we focus on MDD, there are few works in this field
• We plan to develop a framework to elicit usability
requirements in such a way that it could be used in any
MDD method.
• We plan to develop transformation rules from the
bilit i t t t l i d d iusability requirements to generate analysis and design
models

32. Th k !Thanks!
Questions?Questions?