Learn more on software architecture, UML, DSLs,... at http://modeling-languages.com

1. How do Software Architects
consider Non-Functional
Requirements

2. Outline
• Motivation and objective
• Background
• The study
• Observations
• Conclusions and related work
2

3. Motivation
NFRs SA
“the rationale behind each architecture decision is
mostly about achieving certain NFRs”
“[NFRs] play a critical role during system
development, serving as selection criteria for choosing
among myriads of alternative designs”
“quality attribute requirements strongly influence a
system’s architecture”
Little evidence from empirical
studies support these statements
3

4. Objective
To conduct an exploratory study to investigate
the following research question:
How do software architects
deal with NFRs in practice?
4

5. Background
Reqs. structure
Prd. managers
Sw. architects
+ SLR in [Svensson
Programmers
Arch. design
RE activities
Prj. leades
NFR types et al. 2010]
Users
Analysis Companies
Interviews 5
Interviews 11
Interviews 2
e-Survey 25
e-Survey ?
Questionnaire 15
Questionnaire ?
Group discus. 10
Interviews 12
5

6. Background
Observations:
• No clear view in NFR elicitation
• Cost-driven NFR quantification
• Role-dependent view on NFR type’s importance
• Lack of knowledge about NFR management
• Vagueness of NFRs
• Difficulty to test NFRs
• FRs still prevale
• Need for more empirical studies 6

7. The Study
• Type:
 Exploratory study using qualitative research
 Seven RQs
• Target population:
 Software architects (or practitioners with that role)
 We invited 21 software organizations (Spain)
 We recruited 12 of them
13 interviews made
7

8. The Study
SCC: Software Consulting Company; SH: Software House; ITD: IT Department
8

9. The Study
• Data collection:
 Semi-structured interviews
 Focus: one single project
 Duration: ~1 hour
• 7 questions about architect profile and development
methodology used in the project
• 10 questions about decision-making and NFRs
 Audio-taped  Transcribed  Checked
9

10. The Study
• Analysis:
 Two authors coded data independently
tabulation
 Categories generation
NVivo Software
 Consolidation
Group meetings
 Presentation
Counting technique
10

11. The Study
• Limitations and mitigations:
 Evaluation apprehension  Confidentiality
 Understandability  Piloting (4)
 Influential factors  Single project
 Bad memory  Questionnaire sent in advance
 The best project  Ask for the most familiar
 Omitting data  Results overviewed by respondants
 Researcher bias  Two separated interpretations
 Generalize the findings  We do not attempt to
make universal generalizations
11

12. Observations
• RQ1: What is the role of the software
architect?
 Nobody had the “software architect” position
 Nomination: (experience, tec. knowledge) > architect skills
 12 played other roles played in the project:
Project
3 Manager
4
Both
5
Developer
12

13. Observations
• RQ2: Are there terminological confusions on
NFRs?
 Inability. “availability”, “accuracy”, “sustainability”
 Confusion. “ergonomic” instead of “easy to use”
 Incorrectness. “Maintainability is very important, because
when something is working, we can’t make changes”
 Worsened by the Spanish translation
13

14. Observations
• RQ3: What types of NFRs are relevant to
software architects?
10
9
8 49 Software qualities
7
6
5
4
Domain
3 Tacit
2
1
0
14

15. Observations
• RQ3: What types of NFRs are relevant to
software architects?
10
9
8
33 Non-technical reqts.
7
6
5
4
3
2
1
0
15

16. Observations
• RQ4: How are NFRs elicited?
User and Mainly
3
architect architect
10
Outsourced
Projects
 All architects considered elicitation as a gradual
process… also never-ending
16

17. Observations
• RQ5: How are NFRs documented?
Plain text 1 No
documentation
Some kind
3 9
of strategy
“I rarely document
Volere my projects because
ISO/IEC 9126-based it costs money”
Ad-hoc
 Documentation not always evolved! 17

18. Observations
• RQ6: How are NFRs validated?
 NFRs had been met by the end of the project?
Not “yes” 2 Yes
11
“We wait for the Very vague
client to complain” justifications
18

19. Observations
• RQ6: How are NFRs validated?
 What NFRs were validated?
None Efficiency
1
Accuracy
Usability
Unknown 4 Reliability
8
19

20. Observations
• RQ7: What type of tool support for NFRs is
used?
 Architects did not use any specific tool support for
NFR management
20

21. Observations
• RQ7: What type of tool support for NFRs is
used?
 What about NFR-driven MDD (cf. [Ameller RE’10])
Not answer
2
Not at all
Just support 2 5
“I would not trust”
4
Not viable
21

22. Conclusions
• Contributions
 Corroborating previous evidences
Architects…
performed other activities simultaneously
did not share a common vocabulary
showed some misunderstandings and lack of knowledge
considered performance and usability as most important
mainly elicited NFRs iteratively
more often than not, didn’t document
validated just a few types
were not enthusiastic about advanced tool support
22

23. Conclusions
• Contributions
 Finding previously unreported observations
Architects…
considered NTRs almost as important as quality reqts.
mainly elicited NFRs by themselves
claimed that NFRs were satisfied at the end of project
did not use any specific tool for NFR management
23

24. Conclusions
• Contributions
 Finding some misalignments or even contradictions
with previous studies
Architects…
did not exist as a differentiated role
quantification of NFRs was poor
24

25. Conclusions
• Future work
 Replication; consolidation with other studies
• E.g., study on OSS domain [REFSQ’12]
 Changing the conditions
• E.g., influence of an starting architecture (Ferrari et al, REJ10)
• More research needed on:
 Cost-benefit analysis (e.g., documentation)
 Highly customizable NFR management (e.g., existence
of architect role)
 Exploring other communication channels (e.g., blogs)
25

26. Hope you
liked it!