Devnology Back to School: Empirical Evidence on Modeling in Software Development

Zin en Onzin van Modeleren
in Software Development

Dr. Michel R.V. Chaudron
Leiden Institute of Advanced Computer Science
chaudron@liacs.nl

Leiden University. The university to discover.

Outline of talk
Introduction
- LIACS & Software Engineering
Modeling in Software Development
- How are models being used in practice?
- What does it cost/bring to do modeling?
Experiments
- Hafeez
- Dave (announcement)

Dr. Michel R.V. Chaudron, Devnology, June 2012, Leiden Leiden University. The university to discover.

Leiden Institute of Advanced
Computer Science
Algorithms Software Engineering
Prof. J.N. Kok, Prof. T. Bäck Dr. M.R.V. Chaudron,
Prof. F. Arbab,
• Novel Algorithms
• Data Mining • Software Design and Modeling 4 years Leiden
• Natural Computing • Software Composition
• Applications • Formal Methods
10 years TU Eindhoven
• Drug Design • Coordination / Concurrency 2 year IT-company
• Medicine
• Engineering
• Logistics
• Physics

Imagery and Media Core Computer
Dr. M. Lew, Dr. F. Verbeek
Technologies
• Computer Vision and Audio/Video Prof. H. Wijshoff, Dr. T. Stefanov
• Bioimaging
• Multimedia Search • Embedded Systems
• Internet Technology • Parallel / Distributed Computing
• Computer Graphics • Compiler Technology

3

Software Engineering Researchers

Werner Heijstek Hafeez Osman Dave Stikkolorum Ana Fernandes Pieter Kwantes
Models in Global SE Updating UML Teaching SW Design UML in maintenance Domain Specific
Modeling Languages

Ramin Etemadi Bilal Karasneh Peng Ye Javier Hernandez
Optimizing Reasoning about Modeling Sw- Dr. Michel R.V. Chaudron
Sw. Arch. Design Sw. Arch. Design Product Lines chaudron@liacs.nl


Automated Updating of Class Models
time
source code

change  change ’ change ’’

learn design update with
abstraction:  abstraction of change: () (’) (’’)
UML design model

A change  can be an addition, modification, removal.


Extracting Model from Images Bilal

Recognizing:
- Rectangles
- Lines
- Text,
- arrow-heads

Input: .jpeg, .png Output:
XMI for the class model


History of UML

Other modeling languages: BPML, Archimate, …


4+1 Views Representation
of System Architecture
What can/does
How is the system the system do ? How to build /
structured? configure ?
Logical View Development View

System Architect End-user Programmers
Functionality (Decomposition) Configuration management
Use Case View
How does the Where to install ?
system behave? What hwnw is used?
Process View Deployment View
System Architect System engineering
System topology
Concurrency, Communication, Delivery, installation, maintenance
Synchronization How does the Performance, Scalability, Throughput
system perform ?


UML modeling notations for 4+1 Views
Structure model : components,
Config. Mngnt model
packages, interfaces
versioning policies
A B file ownership
Stakeholders & …
Use cases view
C D

Behaviour model : Deployment model :
MSC, state-diagrams physical model + mapping
A B C D
A C D
B
TCP/IP over Ethernet
BC/WC e2e-response times, freq. bandwidth, availability


Opinions on Value of Modelling

Value
Agile

MDA “Loose
UML”
Formal Methods

100%
Panel
completeness


Introduction: Research Interest
- What are the pay-offs of investing in
early design/architecture/modeling?
Fewer defects?
Cheaper maintenance? …

- Effectiveness of software modelling
- Quality of Modeling
When is a model good (enough)
- Analysis of Non-Functional
Quality Properties of architectures

- Many modeling approaches around;
focus on UML in custom software development

Increasing Complexity of Software

Blow, J. "Game Development: Harder Than You Think." ACM Queue
12

Empirical Research
… is a way of gaining knowledge by means
of observation or experience.

Theory Practice
SE body of knowledge People/Skills, Processes, Methods,
Technology, Artefacts
Observation
Hypothesis

Prediction/Test
Validation

Topics: CMM, OO, Agile, MDA, …
Methods: Experiment, Case study, Survey, Project Repositories …

contact
Dr. Michel R.V. Chaudron
Associate Professor
Leiden Institute of Advanced Computer Science
chaudron@liacs.nl
www.liacs.nl/~chaudron

If you have UML models you would like to assess?

If you want advice on Quality Assurance for modeling?

If you want to sponsor our research?


Empirical Life-cycle

Initial Idea
Exploratory Interviews
Survey

Method
Development

Experiment
Industrial Case Studies


Phase 1: Explore via Case Studies

Your project here? Mail me: chaudron@liacs.nl

Industrial use of UML diagrams

How UML is used B Dobing, J Parsons - Communications of the ACM, 2006
Class


Styles of Modeling
Sketch

Communication
Recipe for construction


Modelling style and model purpose

Architecture and Design Description. IEEE Software 23(2): 40-46 (2006)
C.F.J.Lange, M.R. V. Chaudron, J.Muskens: In Practice: UML Software
understanding

communicating

managing
complexity
implementation
blueprint

Styles of using UML
– as a sketch – thinking tool/understanding
– for communicating system design
– as a blueprint – guide the implementation work
– as a implementation (MDA) - code generation

The Modeling Spectrum

Model driven architecture: Principles and practice, Brown, A.W., Software and Systems Modeling, 2004


Uses of Software Models
+ Understanding Doing
+ Analyzing / Predicting modeling is
a quality
+ Communicating improving
activity
+ Guiding


UML in practice
Sketch

Communication

More effort  More expensive
Recipe for construction


Modeling & Documentation
in Agile Development
- Agile principles:
working software over comprehensive documentation
Survey under
75+ agile developers
Tending towards:
we need a bit more
Modeling is
compatible with
agile development
Christoph J. Stettina and Werner Heijstek, Necessary and Neglected? An Empirical Study of Internal Documentation in Agile Software Development Teams 29th ACM Int. Conf. on Design of
Communication , Pisa, Italy


Does use of UML improve software quality?

Ariadi Nugroho, Michel R. V. Chaudron: A Survey of the Practice of Design -
Code Correspondence amongst Professional Software Engineers. ESEM 2007:
50.0

45.0
Reduce
Somewhat Reduce
45%
Neutral 41%
40.0 Somewhat Improve
Improve
Percentage of the respondents

35.0

30.0

25.0

20.0

15.0

10.0

5.0

0.0

Cover Correctness Modularity Testability Understandability
Requirements

A large number of developers indicated the use of UML
improves understandability and modularity

Economic Model for Cost of Quality

From: H. Krasner, Cost of Quality, 1998
Cost of SQ = Achievement Cost + Non-conformance Cost


Cost of Modeling
Focus on effort

“Is there a correlation between class-count and the
effort spent in modeling?”

Sources:
‐ Experiment
‐ Set of industrial case studies


Empirical Data
 Experiment
 106 MSc students in (TU/e)
 Organized in 35 teams
 Task: Model Car Navigation System
 Duration: 6 weeks assignment
 Three treatments were applied:
1. No modeling conventions
2. With modeling conventions
3. Tool-supported modeling conventions


Experiment Results: Effort ~ Size?
40,00

35,00

30,00
Effort (hours)

25,00
Spearman’s Correlation Coefficient
20,00

15,00 Effort (hours)
10,00 Class-count .302
5,00 Sig. (2-tailed) .171
0,00
0 10 20 30 40 50 60 70
Class-count
Number of classes in design

There is no significant correlation between class-count and
the effort spent in modeling.

 class-count is not a good measure for the effort spent
on modeling?

Measurement of Modeling/Design
in RUP Effort Distribution

modeling designing

Impact of MDD on Effort Distribution
as % of total project effort
construction

constr.
elaboration
transition

transition
elaboration
architecture & code generator
incep.

‘traditional’ Model-driven
(avg. of 17 projects) (1 project)

Does Quality of Modeling Matter?
An Industrial Case Study
Focus on detail in a UML Model
attributes

operations

associations

Low detail High detail


Relation between UML-LoD and Code Quality
Select ‘defects’
in defect DB

Find classes in source component C2 defect
component C2

code that were
requires I2
requires I3I2
requires
uses requires I3
I2.g
uses I2.g
density
uses I3.h

repaired for solving this uses I3.h

defects

O rder
OrderDetail
* 1 -date P ce or
repro ss Parser A er
nalys DC
B reator DF
B iller Inherita R
nce elator DC
B hecker S tis Filte
ta tic r StatisticC lculator
a
-quantity

Find corresponding
-status
-taxS tatus SA A
A T.B T
+calcTax()
+calcS ubTotal()
+calcTotal()
+calcW eight() +calcTotalW eight()
Paym ent 1

-am ount 1

classes in UML models Credit
-num ber
-type
-expDate
Cash
-cashTendered
Check
-nam e
-bankID
+authorized()
+ authorized()

OrderDetail
O rder LoD per
class
* 1 -date P ce
repro ssor P rse
a r A lys
na er DC
B reator DF
B iller Inherita ceR
n elator D C eck
B h er S tisticFilte
ta r StatisticC lc r
a ulato
-quantity

Determine LoD
-status
-taxS tatus SA A
A T.B T
+calcTax()
+calcS ubTotal()
+calcTotal()
+calcW eight() +calcTotalW eight()
Paym ent 1

-am ount 1

for CD and SD -num ber
-type
-expDate
Credit Cash
-cashTendered
Check
-nam e
-bankID
+authorized()
+ authoriz ed()


Relation between Level of Detail
and Defect Density

= 1 class

Level of Detail for Sequence Diagrams is significantly
(negatively) correlated with defect density.
More detailed model => fewer defects


Quality Assurance for
Software Design / UML Model

- How to define Quality for design/model?
- How to assess quality?
- How to check conformance in code?


Quality Models
Existing models

Boehm McCall
ISO 9126 Rombach

…

- Decomposition of
characteristics
- Bottom level: metrics
- Differences in
- Relations between characteristics
- Vocabulary Boehm’s Quality Model


How to Define UML Model Quality?

Framework for quality of UML models (Lange & Chaudron, 2005)
The quality attributes of a UML model is determined
by its purpose...


How to measure goodness of UML model?
Design Heuristics
- Coupling, Cohesion, absence of anti-patterns

Completeness
- Interaction of classes must be described in SD
- Methods of CD must be called in SD
Consistency
- In many casesSD can not be automatically determineddiagram
Messages in it must correspond to method in class
whether a flaw is an incompleteness or an inconsistency
Correspondence
- Dependencies in the implementation are allowed only if they
occur in the design

Automatically Checking Quality of Design
Quality = absence of ‘bad things’
such as omissions, spaghetti
UML model

UML Analysis Tool

Quality Metrics/Rules
• Completeness Visualization of
• Consistency model + metrics

Screenshot of the MetricView tool

Quality Metrics
• Completeness
• Consistency
• … extensible

= violation
= ok

MetricView Tool MetaView
to visualize the (hidden)
inter-diagram relations
Example:
Which classes contribute
to a certain use case?

http://www.youtube.com/watch?v=G3HJ_QR9EG4
Supporting task-oriented modeling using interactive UML views, C.F.J. Lange, M. A.M. Wijns, and M. R.V. Chaudron,
Journal of Visual Languages and Computing

Christian Lange 42

MetricView Tool MetricView
The values of metrics are
visualized on class
diagrams using colors
Example:
Coupling-Between-
Objects (CBO)

We will analyse your UML model : mail to Chaudron@liacs.nl

Christian Lange. Leiden University. The university to discover. 43

3D visualization of design measurements
Your cases welcome…


Christian Lange. Leiden University. The university to discover. 44

Findings from State of the Practice
UML is used in different ways for different purposes
- Incompleteness
- Modelers focus on complex and critical parts
- Disproportion
- Different parts of the system are modeled different in level
of quality
- Inconsistency
- Conflicting information in different views of one model

Quality Assurance for modeling should address these

Industrial Experience with MetricView
as Quality Assurance Tool
- Based on 15+ industrial projects

- If there are weak spots in the design, then these are
indicated as ‘suspects’ by MetricView

- MetricView indicates many weak spots that do not
require improvement according to project architects
(false positives?)
 prioritization is needed

- The later MetricView is applied, the fewer ‘weak spots’
are removed from the design  process issue


Understanding Effectiveness of
Modeling through Eyetracking


Hypothetical Benefits of Modeling
UML Modeling

Developer Process Product
Benefits Benefits Benefits

Better Improved Improved
requirements Fewer Quality
Understanding
Defects
Problem Domain

Improved Design Reduced Improved
Compliance rework Productivity

Better
Improved Reduced
Understanding More efficient
Design Quality testing
Solution Space Testing
effort

Reduced
More efficient
Improved Shared maintenance
Maintenance
Communication System Model effort

More Project Management
Better Risk
Accurate Benefits
management
Estimating


Concluding Remarks
- Modeling is becoming common in software development.
There are payoffs for modeling in:
- improved code quality
- improved communication

- Empirical studies enhance alignment of research and
practice – your input is needed

- Practical Quality Assurance for UML should use low
hanging fruit:
i) Naming and layout-conventions, ii) Reviews & Inspections
iii) Versioning, iv) Metrics


Future Directions
- Automated Reasoning about Designs
- Diagnosis & Prioritization
- Severity Assessment of design flaws /
technical debt

- Next: Empirical studies
- by Hafeez & Arjan
- by Dave

Hafeez Dave


Comparing “MDD with code generation”
& “Modeling + Programming”
MDD with code generation Modeling with Programming
Requires: Off the shelves tooling
- stable domain
- very good understanding of Standard language
domain (staff hiring & turnover)

- Multiple projects Divergence of model and code

- Don’t introduce in critical
projects

Needs dedicated techniques for
Quality Assurance, estimation

How often do design issues lead to problems?
70
Never
Sometimes
60 Often
Very Often

50

40

30

20

10

0
Inconsistency Understandability Inaccuracy Incompleteness

Incompleteness of models was considered most
often to lead to implementation problems

How do developers apply detail to UML models?

70.0
Disagree
Somewhat Disagree
61%
60.0
Neutral
Somewhat agree
52%

50.0 Agree

40.0
37%

30.0

20.0

10.0

0.0
Equal details Details proportional More details for Programmers
for all parts to complexity critical parts determine details

Developers apply more detail on critical
and complex parts of a system

Which factor most often drives developers to

non-correspondence?

60

Never
50
Sometimes
Often
Very Often 48%
40

30
31%
29%
20

10

0

Meeting Impractical Incomplete Model doesn’t satisfy
deadline model model requirements

Among other factors, developers considered incomplete model
more often drives them to non-correspondence


Does strictness of implementing a model differ

for different modeling constructs?
50.0
Loosely 36%

Somewhat loosely
45.0
Neutral
40.0
Somewhat strict
Strictly
46%
35.0

30.0
31%
25.0

20.0

15.0

10.0

5.0

0.0

Package Dependency Inheritance Class and The order of
structure relations relations method names method calls

Developers favor to implement dependency and
inheritance relations more strictly than other constructs

Which classes are modeled?

Complexity and Coupling is higher for classes
that are modeled than not modeled

Maintenance paradox
- Maintainers of software would like better
documentation

- Even if UML models are created during design

Too detailed models become outdated

Better abstraction in reverse- / round trip-
engineering is needed in the setting of
incremental & iterative development


Practical QA for UML modeling
Naming and layout-conventions
The Elements of UML2.0 Style,
Scott Ambler, Cambridge Univ Press, 2005 Low Hanging Fruit
(c) Erin Witzel, 2010

Reviews & Inspections
Guidelines by e.g. Shull et.al., Biffl,

Version Management
Many tools around (e.g. CVS, SVN, …)

Metrics
SDMetrics http://www.sdmetrics.com/


Investigations into effectiveness of software modeling

MetricView contributors
 Johan Muskens
 Christian Lange
 Martijn Wijns
 Dennis van Opzeeland
 Ariadi Nugroho
 Werner Heijstek
 Maurice Vermeer
 Alex Telea
 Robin van den Broek

MRV Chaudron
Sheet 59

Publications
Investigations into effectiveness of software modeling

 Journal Publications
 Supporting task-oriented modeling using interactive UML views. PDF
C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Journal of Visual Languages and Computing (Elsevier), Volume 18, Issue 7, August
2007.
 In Practice: UML Software Architecture and Design Description.
C.F.J. Lange, M.R.V. Chaudron, J. Muskens. IEEE Software, Volume 23, Issue 2, March 2006.
 Conference Publications
 A Visualization Framework for Task-Oriented Modeling using UML.
C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 40th Annual Hawaii International Conference on System Sciences
(HICSS'07). January 2007.
 An Experimental Investigation of UML Modeling Conventions.
C.F.J. Lange, Bart DuBois, M.R.V. Chaudron, Serge Demeyer. Proceedings of the ACM/IEEE International Conference on Model-
driven Engineering Languages and Systems (MoDELS 2006). October 2006.
 Effects of Defects in UML Models - An Experimental Investigation.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006.
(acceptance rate: 9%)
 Improving the Quality of UML Models in Practice.
C.F.J. Lange. Proceedings of the International Conference on Software Engineering (ICSE 2006). May 2006.
 Visual Exploration of Combined Architectural and Metric Information.
M. Termeer, C.F.J. Lange, A. Telea, M.R.V. Chaudron. Proceedings of VISSOFT 2005, IEEE CS Press. September 2005.
 Combining Metrics Data and the Structure of UML Models using GIS Visualisation Approaches.
C.F.J. Lange, M.R.V. Chaudron. IEEE International Conference on Information Technology 2005. Las Vegas, April 2005.
 An Exploratory Study on the Industrial Use of UML: Improving Control over Design Quality.
C.F.J. Lange, M.R.V. Chaudron. JACQUARD Conference. February 2005.
 Investigations in Applying Metrics to Multi-View Architecture Models.
J. Muskens, M.R.V. Chaudron, C.F.J. Lange. In Proceedings of EUROMICRO’04, Rennes, France, September 2004.
 An Empirical Assessment of Completeness in UML Design.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of the 8th Conference on Empirical Assessment in Software Engineering (EASE04). May
2004.
 Konsistenz und Vollständigkeit industrieller UML Modelle.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of “Modellierung 2004“, Marburg, Germany, 24-26 March 2004.
 Workshop Publications
 Towards Task-Oriented Modeling using UML.
C.F.J. Lange, M.A.M. Wijns, M.R.V. Chaudron. Proceedings of the 10th QAOOSE workshop, co-located with ECOOP2006. July 2006.
 Managing Model Quality in UML-based Software development.
C.F.J. Lange, M.R.V. Chaudron. Proceedings of IEEE Conference on Software Technology and Engineering Practice 2005 (STEP), in
the "Workshop on Software Process Improvement, Quality Assurance and Measurement. September 2005.
 Michel Chaudron, Christian Lange, Martijn for the Assessment of Correspondence between UML Designs and Implementations.
Quantitative Techniques Wijns
60
Dennis J.A. van Opzeeland, Christian F.J. Lange, Michel R.V. Chaudron. Proceedings of the 9th QAOOSE workshop, co-located with
ECOOP2005. July 2005.

Devnology Back to School: Empirical Evidence on Modeling in Software Development

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Devnology Back to School: Empirical Evidence on Modeling in Software Development

Similar a Devnology Back to School: Empirical Evidence on Modeling in Software Development (20)

Más de Devnology

Más de Devnology (20)

Último

Último (20)

Devnology Back to School: Empirical Evidence on Modeling in Software Development