Presentation from the IWSM Mensura 2015 conference held October 5-7 in Cracow, Poland

1. A FUNCTIONAL SOFTWARE
MEASUREMENT APPROACH
BRIDGING THE GAP BETWEEN
PROBLEM AND SOLUTION DOMAINS
by Dr.Erdir Ungan, Prof.Dr. Onur Demirörs

2. Problem
•
Granularity
•
Issues
in
Parametric
Es3ma3on
Methods
•
Issues
in
Benchmarking
•
Reliability
Issues
in
FSM
Measurements
•
Issues
Regarding
Measurement
Procedure

3. Anything
you
try
to
quan.fy
can
be
divided
into
any
number
of
"anythings,"
or
become
the
thing
-­‐
the
unit
-­‐
itself.
And
what
is
any
number,
itself,
but
just
another
unit
of
measurement?
What
is
a
'six'
but
two
'threes',
or
three
'twos'...half
a
'twelve',
or
just
six
'ones'
-­‐
which
are
what?
-­‐
F.L.
Vanderson

4. Issues in Parametric
EsJmaJon Methods
•
Most
es3ma3on
methods
suggest
◦ predic3ng
soEware
size
◦ Measure
PD
size
and
convert
it
to
SD
size
based
on
some
historical
data.
◦ ….Huge
varience.
• The
process
of
developing
a
solu3on
to
a
problem
is
ambiguous
and
soE.
Therefore,
es3ma3on
methods
in
PD
need
subjec3ve
input
◦ This
is
one
of
the
points
where
the
reliability
and
repeatability
of
these
methods
are
ques3oned.

5. Issues in Benchmarking
•
Problems
in
measurement
results
and
granularity
also
affects
the
quality
of
data
in
benchmarking
data
sets.
•
Özcan
Top
and
Yilmaz
concluded
that,
those
sets
lack
structural
informa3on
about
the
projects
•
We
cannot
deduct
informa3on
about
the
abstrac3on
level
of
measurements
in
those
data
sets
•
Comparing
data
from
varying
abstrac3on
levels
will
result
in
erroneous
benchmarking

6. SeparaJon of Problem and SoluJon
Domains
The difference between theory and prac2ce equals
your inep2tude...
Anonymous

7. What vs. How
◦ How,
is
the
process
of
breaking
down
a
larger
What
to
smaller
What’s.
◦ What1
◦ What
1.1
◦ What
1.2
◦ What
1.2.1
◦ What
1.2.2
◦ What
1.2.3
◦ What
1.2.3.1
◦ .
◦ .
◦ One
of
these
transi3ons
in
the
W-­‐H
chain
will
pose
as
a
boundary
between
Problem
and
Solu3on
domains
based
on
the
defini3on
of
the
system
at
hand.

8. What vs. How
•
Increase
Profit
•
…
•
Decrease
Costs
•
Increase
Sales
◦ ...
◦ Increase
Adver3sement
◦ Establish
an
online
store
◦ …
◦ Get
a
Server
◦ Build
an
e-­‐commerce
site
◦ Design
Web
Site
◦ Test
Web
site
◦ Develop
Web
Site
◦ Develop
Interface
◦ Develop
Client
Side
SoEware
◦ Develop
Server
Side
SoEware
•
Develop
Database
•
Develop
Data
Abstrac3on
Layer
•
Develop
Business
Layer
◦ Develop
Admin
Module
◦ Develop
Stock
Module
◦ Develop
Customer
Module
◦ Develop
membership
func3ons
◦ Develop
CRM
func3ons
◦ Develop
Customer
Class
◦ Develop
Purchases
Class
◦ Develop
add
purchase
method
◦ Develop
get
purchasename
method

9. PD & SD in SoQware Engineering

10. Mutual Independence of PD & SD
• Although
common
sense
and
experience
dictates
that
PD
and
SD
are
two
separate
domains.
We
needed
to
demonstrate
this
phenomenon.
• We
conducted
two
case
studies
◦ First
Study:
We
inves3gated
the
correla3on
of
problem
and
solu3on
domain
sizes
with
the
problem
and
solu3on
domain
effort
◦ Second
Study:
We
conducted
a
case
study
where
a
single
set
of
requirements
were
to
be
developed
using
two
different
implementa3on
approaches
• Both
studies
supported
our
sugges3ons
about
problem
and
solu3on
domain
separa3on

11. Mutual Independence of PD & SD
Bytes
/
CFP
Std.
Dev.
R2
Comfort
&
Convenience
Type
Components
7.6
0.99
Display
&
Indica3on
Type
Components
21.6
0.992
For
single
organiza3on’s
data
set,
varia3on
and
correla3on
in
Physical
size
(Bytes)
to
COSMIC
size
is
given
in
(Gencel,
Heldal,
and
Lind
2009)
SLOC
/
CFP
Std.
Dev.
R2
Comfort
&
Convenience
Type
Components
8.2
0.4855
15.4
0.417
For
single
organiza3on’s
data
set,
varia3on
and
correla3on
in
SLOC
sizes
to
COSMIC

12. Context

13. Context
◦ We
define
Gaps
1-­‐2-­‐3-­‐5
and
6
as
imperfec3ons
and
therefore
evitable.
◦ However,
we
believe,
ideally
there
is:
◦ One
correct
and
exact
defini3on
of
the
real
life
problem.
◦ One
correct
set
of
requirements
that
describe
the
problem.
Conforming
to
a
requirements
standard.
◦ One
correct
set
of
specifica3ons
that
breaks
down
the
requirements
to
lower
level
specifica3ons.
◦ One
best
implementa3on
of
the
design
based
on
the
technology
and
implementa3on
method
used.
◦ One
best
and
unique
soEware
build
for
the
implementa3on.
Through
an
ideal
compiler.

14. Context
•
However,
Gap
4,
the
gap
between
specifica3ons
and
design
is
different
in
nature
from
these
gaps.
“There
is
no
one
correct
solu3on
to
a
problem.”
•
Engineering
solu3ons
are
guided
by
many
principles,
design
palerns
and
rules.
Nevertheless,
there
is
always
a
subjec3ve,
crea3ve
element
in
crea3ng
solu3ons
to
problems.
Therefore
this
gap
is
essen3ally
inevitable.
•
The
transforma3on
between
specifica3ons
and
design
is
actually
where
we
define
the
border
between
the
problem
domain
and
solu3on
domain
is.

15. Context

16. SoluJon
Approach

17. SoluJon Approach
•
U3lize
a
Common
Concept
For
PD
&
SD
•
Es3ma3on
Accuracy
is
Based
on
Level
of
Detail
•
Incorpora3ng
Decomposi3on
Level
(Granularity)
Into
Measurement
•
Defining
an
Absolute
Reference
Point
•
Automated
Measurement
•
Minimizing
Interpreta3ons
and
Assump3ons

18. Proposed Method
• We
propose
a
two
dimensional
measurement:
◦ Data
movement
◦ Decomposi3on
Level
Therefore,
we
define
each
aspect
of
a
measurement
method
in
these
two
dimensions.

19. Conceptual Modeling
Increasing
Decomposi3on
Decreasing
Tier
#

20. Tier
•
Each
decomposi3on
of
the
system
will
result
in
a
new
Tier
of
system
defini3on.
Each
increase
in
the
3er
number
represents
one
higher
level
of
decomposi3on
traversed
in
the
descrip3on
of
the
system.
•
Each
3er
consists
of
structural
en33es
communica3ng
with
same
3er
level
of
en33es.
Note:
One
set
of
objects
in
a
3er
can
be
present
in
a
higher
level
if
the
entry
and
exit
point
of
data
movements
between
them
and
other
objects
does
not
change
on
their
end.
That
is,
certain
object
may
belong
to
more
than
one
3er
at
the
same
3me.

21. Measurement Method
Etalon
for
Func=onal
Size
Abran
states
that,
• “The
key
concept
of
func3onality
at
the
highest
level
of
commonality
that
is
present
in
all
soEware
was
iden3fied
as
the
‘data
movement’.
This
data
movement
concept
was
then
assigned
to
the
metrology
concept
of
a
size
unit.”
• The
base
unit
of
decomposi3on
level
is
defined
as
the
level
of
a
single
method
within
a
single
class.
• The
base
unit
of
decomposi3on
level
is
defined
as
Tier
0.
Tier
0
is
the
level
of
decomposi3on
in
which
further
level
of
decomposi3on
will
be
irrelevant
of
the
measurement
objec3ves
of
DMP
method.
(eg.
Effort
correla3on)

22. Conceptual Modeling
•
We
chose
sequence
diagram
as
the
main
soEware
model
for
the
empirical
world
to
measure
func3onal
size
alribute.
As:
◦ Possible
and
automated
round
trip
engineering.
Code
–
Seq.Diagam
&
Seq.Diagram
–
Code.
Many
CASE
tools
and
IDEs.
◦ Can
be
defined
in
each
composi3on
level.
◦ Are
being
developed
for
business
level
behavior
and
design
level
behavior.
That
is,
they
can
be
u3lized
in
both
problem
and
solu3on
domains.
◦ Incorporate
more
informa3on
about
a
system
than
other
UML
diagrams.
informa3on
about
the
structures
in
a
system
as
well
as
their
rela3ons
and
system
behavior.

23. Measurement Method
Opera=onal
View
for
Func=onal
Size
• In
DMP
measurement
method,
sequence
diagram
elements
which
represent
the
data
movements
are
taken
into
account
and
other
elements
are
used
for
suppor3ng
informa3on.
• Sequence
Diagram
elements
that
represent
data
movement
are:
◦ Synchronous
message
◦ Asynchronous
message
◦ Crea3on
message
◦ Destruc3on
message
◦ Self
message
◦ Found
message

24. Mapping Phase
Iden=fying
Scenarios
Based
on
the
decomposi3on
level
(Tier
Value)
for
the
measurement,
defini3on
of
the
func3onal
processes
and
their
triggering
entries
change.
1. The
triggering
entry
of
the
func3onal
process
must
be
visible
in
the
system
model
defined
at
this
level.
2. The
structural
en3ty
receiving
the
triggering
entry
must
be
visible
in
the
system
model
defined
at
this
level.
3. The
output
of
the
process
(Exit
or
Write)
must
be
visible
in
the
system
model
defined
at
this
level.

25. Measurement Tool
•
Developed
as
a
master
project
with
Yalın
Meriç
supervised
by
Onur
Demirörs,
Erdir
Ungan
(2013)
The
automated
XMI
genera3on
process
was
as
below:
1. Select
programming
language.
2. Select
folder
where
source
code
files
are
present.
3. Automa3cally
generate
all
sequence
diagrams
for
imported
source
codes.
4. Create
and
save
XMI
file.
5. Count
data
movements
for
each
method.
6. Assign
objects
to
3er
level(s)
7. Enter
3er
level
8. Consolidate
DMs
in
that
level
and
measure.

26. 11
Data
Movements

27. A1
A3
A2
B1 C1
C2
D1
D2

28. A1
A3
A2
B1 C1
C2
D1
D2

29. A
B
C
D
7 Data Movements

30. 2 Data Movements
System

31. ValidaJon
•
Theore3cal
Valida3on
•
Empirical
Valida3on

32. ValidaJon
•
Theore3cal
Valida3on
◦ Rice
Cooker
Case
◦ Lavazza
et
al.
and
Levesque
et
al.
separately
measured
the
system
with
sequence
diagrams.
◦ They
reached
different
measurement
conclusions.
◦ They
modelled
the
func3onality
differently.

33. No. Functional
Process
Triggering Event Data Movement
Description
Data Group DM CF
P
1 Control
Cooking
Lamp
Tick (elapsed) Tick (elapsed) TimedEvents E 4
Cooking Mode Cooking Mode R
Cooking Status CookingLamp X
Cooking Status WarningLamp X
2 5 sec. signal
management
(control heater)
Every 5s signal Signal5 TimedEvents E 4
GetTargetTemp CookingState R
ReadTemp TemperatureSens
or
E
HeaterOn or Heater command X
3 set target
temperature
Signal30 Signal30 TimedEvents E 3
GetMode CookingMode R
SetTargetTemp CookingState W
4 Select Cooking
Mode
Selected Cooking
Mode
Selected Cooking
Mode
E 2
Selected Cooking
Mode
W
TOTAL 13
Rice Cooker COSMIC Measurement Results According to Levesque et. al.

34. No. Functional Process Triggering Event Data Movement
Description
Data Group DM CFP
1 Control
Cooking
Lamp
Tick (elapsed) Tick (elapsed) TimedEvents E 2
On CookingLamp X
2 5 sec. signal
management (control
heater)
Signal5 Signal5 TimedEvents E 4
GetTargetTemp CookingState R
ReadTemp TemperatureSensor E
HeaterOn or Heater command X
3 30 sec. signal
management (set
target temperature)
Signal30 Signal30 TimedEvents E 5
GetMode CookingMode R
Tick(elapsed) TimedEvents E
GetCookingTemp CookingSpecs R
SetTargetTemp CookingState W
TOTAL 11
Rice Cooker COSMIC Measurement Results According to Lavazza

35. • Levesque
Iden3fied
a
func3onal
process
as
“Select
cooking
mode”
• It
complies
with
the
defini3on
of
func3onal
process.
• However,
it
is
one
decomposi3on
level
lower
than
the
others.
• It
should
have
been
a
sub
process
of
“set
target
temperature”.
• That
is
:
◦ The
size
of
the
rice
cooker
is
◦ 11
DMP
in
Tier
1
◦ 13
DMP
in
Tier
0

36. Emprical ValidaJon -­‐ Findings
DMP
had
beOer
correla=on
with
effort
than
COSMIC
but
worse
than
LOC
DMP
vs.
Effort
R2
=
0,89
COSMIC
vs.
Effort
R2
=
0,62
LOC
vs.
Effort
R2
=
0,92

37. Example
•
Upon
receiving
X
from
A,
the
system
shall
read
X
from
B
and
compare
their
values.
If
X
is
greater
than
the
Balance,
system
shall
send
Z
to
C.
•
COSMIC
Size:
E
R
X
:
3

38. • Upon
receiving
the
Amount_to_be_Withdrawn
from
the
Customer,
the
system
shall
read
Current_Balance
from
Customer's
Account
and
compare
their
values.
• If
the
Amount_to_be_Withdrawn
is
greater,
system
shall
send
Warning
SMS
#213
to
Customer's
Cell
Phone.
• COSMIC
Size:
3
• Tier:
7
Example

39. • Upon
receiving
WebService_Withdrawal
from
System_ATM,
the
system
shall
read
Current
Balance
from
Customer's
Account
and
compare
their
values.
• If
the
Amount_to_be_Withdrawn
is
greater,
system
shall
send
WebService_BalanceNotEnough
to
CRM.Module.
• COSMIC
Size:
3
• Tier:
5

40. • Upon
receiving
WebService_Withdrawal
from
System_ATM,
the
system
shall
WebService.CustomerAccount.
Upon
response,
system
will
read
Current
Balance
from
the
response
and
compare
with
The
Amount_to_be_Withdrawn.
If
The
Amount_to_be_Withdrawn
is
greater,
system
shall
send
WebService_BalanceNotEnough
to
CRM.Module.
• COSMIC
Size:
5
• Tier:
4
Example

41. • Upon
receiving
CurrentHeat
from
Thermometer,
the
system
shall
read
RequiredHeat
from
ReqHeatRegister
and
compare
their
values.
• If
CurrentHeat
is
greater
than
RequiredHeat,
system
shall
send
TurnoffHeater
Signal
to
the
Heater.
• COSMIC
Size:
3
Example

42. • If
we
are
measuring
the
embedded
soEware
on
a
microcontroller
level
of
measurement
can
well
be
Tier
:
0
• If
the
system
is
comprised
of
a
higher
level
soEware
running
on
a
computer
and
communicate
with
intelligent
sensors
and
heater
through
certain
interfaces:
◦ If
the
boundary
is
only
the
soEware
running
on
the
computer,
and
see
sensors
as
black
boxes:
COSMIC
Size:
3,
Tier:0
◦ If
the
boundary
includes
the
sensors
and
interfaces:
COSMIC
Size:
3,
Tier:
3+
Example

43. ContribuJons
Streamlining
the
Measurement
Concepts
• We
defined
a
measurement
approach
based
on
a
concept
that
is
common
in
both
problem
and
solu3on
domains;
data
movement.
• This
streamlines
measurement
of
both
project
and
product
alributes
in
two
domains.
• Improves
conversion
of
units,
es3ma3ons,
approxima3ons
and
normaliza3on
of
several
size
defini3ons
and
values.

44. Improvements in Reliability of
Measurement Results
• Backfiring
concepts
of
FSM
from
the
source
code
elimina3ng
the
reliability
issues
in
Tier
0.
(product
measurement)
• We
traverse
through
higher
levels
u3lizing
metadata
for
the
lowest
level.
Only
point
of
human
interpreta3on
is
in
the
genera3on
of
this
metadata
which
relates
lower
level
concepts
to
higher
level
ones.
• This
mi3gates
measurement
errors
as
there
is
less
room
leE
for
interpreta3on
and
renders
errors
recoverable
by
fixing
the
metadata.

45. BeWer Benchmarking
• Having
decomposi3on
level
incorporated
into
measurement
results
makes
the
scope
and
abstrac3on
of
the
measured
soEware
product
visible.
• Comparing
measurement
methods
on
the
same
level
of
decomposi3on
will
greatly
improve
the
accuracy
of
benchmarking
studies.

46. BeWer EsJmaJons
•
Instead
of
using
conversion
factors
between
two
different
size
measurements
that
are
essen3ally
unrelated,
we
define
an
es3ma3on
method
which
rely
on
the
same
concepts
that
the
measurement
method
does.
•
Es3ma3ons
become
less
prone
to
gaps
between
domains
and
project
phases.
•
By
measuring
in
lower
levels
of
decomposi3on,
DMP
method
measures
data
manipula=ons
which
would
otherwise
be
leE
out
in
higher
levels
into
measurement.
This
also
increases
the
accuracy
of
es3ma3ons.

47. EsJmaJon Approach
• N
–
Op3mal
Decomposi3on
Level
(Tier
#)
• Expected
Accuracy

48. BeWer Measurement of
SoQware Changes
• By
DMP
method,
it
is
possible
to
measure
specifica3ons
defined
in
levels
lower
than
func3onal
user
requirements.
• This
makes
measuring
lower
level
soware
changes
more
accurate
than
exis3ng
FSM
methods.
• Iden3fying
the
3er
level
of
change
requests
also
improves
the
change
management
processes
as
the
scope
and
impact
of
the
change
can
be
beler
analyzed.

49. Further Studies
• The
same
approach
can
be
applied
to
system
specifica3on
level
or
business
process
defini3ons.
• Another
PhD
thesis
is
being
conducted
in
our
research
group
on
sizing
business
process
models
based
on
func3onality.
◦ It
is
possible
to
extend
the
problem-­‐solu3on
flow
to
the
level
of
business
processes.
◦ This
would
be
a
great
opportunity
to
build
a
streamlined
measurement
and
es3ma3on
framework
spanning
the
whole
soEware
engineering
process.