On the Quality Properties of Model Transformations: Performance and Correctness. PhD Thesis Presentation. Lola Burgueño. April 28th, 2016

1. On the Quality
Properties of Model
Transformations:
Performance and
Correctness
LOLI BURGUEÑO
SUPERVISED BY:
ANTONIO VALLECILLO AND MANUEL WIMMER
APRIL 28, 2016

2. Outline
 Introduction
 Background
 Parallelization of Model Transformations
 Testing of Model Transformations
 Conclusions and Future Work
2
2

3. Motivation
 The more present software is
in our life, the more we
demand from it
 Problems are increasingly
complex
3

4.  MDE as an attempt to
solve problems
Manipulates the
complexity of
large software
systems by
considering only
the aspects of
interest
4

5.  Model Transformations
MDE
 Large models
 Non-trivial
transformation logic
Motivation
5

6. In practice we need to:
 Store and handle models with
millions of instances
 Transform these models in
reasonable amount of time
 Make better use of our current
IT infrastructure
 Networks of distributed computers
 Multi-core machines
 Prove correctness
6

7. Challenges
 RQ1. Is it possible to provide a
concurrent approach and the
appropriate mechanisms to support
the parallel and distributed
executions of MTs?
 RQ2. Is it possible to build
traceability mechanisms between
implementations and specifications
of MTs?
7

8. Contributions
 A model transformation engine called LinTra
that transforms models─that can be
distributed over a set of machines─in parallel
 A light-weight testing approach that can be
used as a first step towards identifying bugs
in MT implementations at an early stage,
quick and cost effectively
8

9. Parallelization of Model
Transformations
PERFORMANCE
9

10. Background. Linda
 Definition:
 Coordination model that uses a shared memory space as the only
means of communication among parallel processes
Blackboard
(“Bye”, “World”)
read(?, “World”)
write(“Bye”, “World”)
write(25, “dollars”)
(25, “dollars”)
10

11. Background. Linda
Every tuple space can be distributed over sets of machines, in a user-
transparent way
The Linda primitives are conceived to be integrated in a host language
Mature implementations and tools already exist
11

12. LinTra. Architecture
 Mature Linda implementations available
 Key-value databases
 For different general purpose languages
 GPL are too low level languages to write MTs
 We will place another language (MTL) on top
that compiles to GPLs
HMTL
(high-level MTL)
LMTL
(low-level MTL: Java)
LinTra Blackboard
12

13. Blackboard implementation
14

14.  MMs are represented by means of Java
classes
 Models are composed of class instances
(objects)
package classMM;
public class Attribute extends NamedElt
implements IdentifiableElement, Serializable{
String id;
Boolean multivalued;
String typeID; Boolean typeIsComposed = false;
String ownerID; Boolean ownerIsComposed = false;
public Attribute (String id, String name,
Boolean multiValued, String ownerId,
String typeId){
super(name);
this.id = id;
this.multivalued = multiValued;
this.ownerID = ownerId;
this.typeID = typeId;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public Boolean getMultivalued() {
return multivalued;
}
public void setMultivalued(Boolean multivalued) {
this.multivalued = multivalued;
}
Models and Metamodels
15

15. Traceability
 Does not store traces explicitly
 Implicitly implemented
 Bidirectional function
 Based on identifiers
16

16. Master-Slave configuration
 Out-place MT
 Blackboard
partitioned in
different areas
 Master/Slave +
Shared Memory
 No dependencies
18

17. Model Transformation Example
public class BibTeXML2DocBook implements ITransformation {
public Collection<IdentifiableElement> transform(Collection<IdentifiableElement> elems) {
List<IdentifiableElement> out = new LinkedList<IdentifiableElement>();
for (IdentifiableElement e : elems) {
if (e instanceof Article) {
Article a = (Article) e; out.add(article2Section(a));
} else if (e instanceof Author){
Author a = (Author) e; out.add(author2Paragraph(a));
}
}
return out;
}
private Section article2Section(Article a){
return new Section(
TraceFuntion.create(a.getId(),Rules.Art2Sec), a.getTitle(),
TraceFunction.resolveAll(a.getAuthorsIds(),RuleNames.Auth2Par));
}
private Paragraph author2Paragraph(Author a) {
return new Paragraph(
TraceFunction.create(a.getId(),Rules.Auth2Par),a.getAuthor());
}
}
 MT specification
 Article  Section
↑ ↑
 Author  Paragraph
19

18. Model Transformation Chains
 Extension of our approach with
Master
load
Slaves
Serialization
TODO SRC
TRG
20

19. Model Transformation Chains
 Extension of our approach with
 synchronization mechanisms between the transformations
 pairs of identifiers
Master (T1)
load
Slaves (T1)
Serialization
TODO
SRC TRG
Intermediate
Master (T2)
TODO
Slaves (T2)
21

20.  Research questions
 How does jLinTra perform compared to existing sequential execution engines?
 How does jLinTra perform compared to other emerging parallel execution engines?
 How does jLinTra model transformation chains perform?
 How is the performance of jLinTra affected when models do not reside in memory?
22

21. DBLP—Model queries
• Example of query: Find all journals where people who are actively publishing at the IST journal are also
publishing
IMDb Movie Database—Model copy and traversal
• Proposed in TTC 2014
• Finds all pairs of people who played together in at least in 3 movies
Java Refactoring—Model modification
• Proposed in TTC 2015
• All the @Singleton annotations are removed from Java programs and their implicit behavior is replaced with
the actual Java code they represent.
Java to Prefuse Graph
• Taken from the model visualization domain
Java to Prefuse Graph and then to Reduced Graph—MT chains
Five case studies
23

22. LinTra. Evaluation
Machine:
64-bit Linux
11.7 Gb of RAM memory
16 cores of 2.67GHz
24

23. IMDB - Identity IMDB – Find Couples
25

24. Java Refactoring
26

25. Java to Graph
27

26. Speed-ups (Slow-downs)
28

27. Penalty: 3,7 times slower
(1.5 times faster using SSD)
 RAM + Hard disk
 Synthetic models for the IMDb case study according to the procedure described in the TTC
case
 Identity transformation
 Models with 8 million of elements
29

28.  Answering the research questions
 How does jLinTra perform compared to existing sequential execution engines?
 Faster than any sequential engine
 How does jLinTra perform compared to other emerging parallel execution engines?
 Navigations are the weakness of jLinTra
 In any case, in many cases performs faster than pATL
 How does jLinTra model transformation chains perform?
 In the same way as there were executed sequentially
 No price is paid when data in the output model is needed as soon as possible
 How is the performance of jLinTra affected when models do not reside in memory?
 Transformations work with only a small delay 30

29.  Many situations in which we need to
evolve models, instead of creating them
anew
 Migrating, optimizing and modernizing software
 Atomic transformation actions:
 Deletions, creations, updates
 Recursive vs. non-recursive matching
LinTra. In-place MTs
RHS
LHS
Circle2CircleStar
Non-Recursive
Recursive
RHS
LHS
CircleStar2Triangle
31

30. Case study: Public to Private
- Remove all the comments
- Change the attributes from
to private
- Create the getters and setters
 Lines of code
 Out-place LinTra: 3,302
 In-place LinTra: 194
32

31. Testing of Model
Transformations
CORRECTNESS
33

32. Background. Tracts
 In general it is difficult and expensive (time and
computational complexity-wise) to validate in full
the correctness of a model transformation
 Tracts certify that a MT works for a selected set of
test input models
 Identify bugs in a cost-effective, light-weight manner
34

33. MT
Specification
MT
Implementation
fulfils
describes
Specification: A document that
specifies, in a complete, precise,
verifiable manner, the
requirements, design, behavior,
or other characteristics of a
system or component…
[IEEE Standard
Computer Dictionary]
Implementation:
(1) The process of translating a
design into hardware components,
software components, or both.
(2) The result of the process in (1)
[IEEE Standard Computer Dictionary]
(What?) (How?)
35

34. a set of constraints on the
source and target metamodels
a set of source-target
constraints
a tract test suite
(a collection of source
models satisfying the source
constraints)
A Tract defines
36

35. -- C1: SRC_oneDaughterOneSon
Family.allInstances->forAll(f|f.daughters->size=1 and f.sons->size=1)
-- C2: SRC_TRG_Mother2Female
Family.allInstances->forAll(fam|Female.allInstances->exists(f |
fam.mother.firstName.concat(' ').concat(fam.lastName)=f.fullName)) 37

36. ATL for the implementation of MTs
 Hybrid MT language
 Mainly composed by a set of rules
 Input pattern
 (Filter)
 Output pattern
module Families2Persons;
create OUT: Persons from IN: Families;
helper context Families!Member def:isFemale:Boolean=
if not self.familyMother.oclIsUndefined() then
true
else
if not self.familyDaughter.oclIsUndefined() then
true
else
false
endif
endif;
rule Member2Female {
from
s: Families!Member (s.isFemale)
to
t: Persons!Female(fullName<-s.firstName+' '+s.familyName)
}
…
38

37. Testing Transformations with Tracts
• Check source constraints
for the each model of
the test suite
Step 1
• Run the transformation
implementation for each
model of the test suite
Step 2
• Check the target
constraints for the
generated models
Step 3
• Check the source-target
constraints for each pair
of input-output models
Step 4
39

38. Matching Tables
 Tracts allow the specification and testing of M2M transformations but they do
not track where the faults in the implementation are
 Establish the alignments between specification and
implementation
 Extend Tracts to deal with T2M and M2T
transformations
40

39. Establishing relations between implementations and
specifications
Misalignment between specification and
implementation
 RQ1. Which transformation rule(s) implement(s) which constraint(s)?
 RQ2. Are all constraints covered by transformation rules?
 RQ3. Are all transformation rules covered by constraints?
41

40.  Use the common part that
implementation and specification
share: types and features
OCL Constraints
ATL
Rules
Source
Metamodel(s)
Target
Metamodel(s)
OCL
Metamodel
ATL Metamodel
«c2» «c2»
«u»
«u»
«u»
«u»
42

41. OCL constraints
(as text)
Parser
Footprint
Extraction
Matching tables
Matching
function
OCL constraints
(as model)
Footprints of OCL
constraints
ATL Rules
(as text)
ATL Rules
(as model)
Footprints of ATL
rules
Footprint extraction
From Constraints
• OCL expressions are
dependent on their contexts

• consider only last elements
of OCL expressions
• consider only the types and
features inside the deepest
iterator (forAll, select…)
• discard primitive types and
constants
From Rules
• Idem as constraints plus:
• types from the in-pattern
• types from the out-pattern
• types from the imperative
parts
• only the type helpers and lazy
rules return
43

42. OCL constraints
(as text)
Parser
Footprint
Extraction
Matching tables
Matching
function
OCL constraints
(as model)
Footprints of OCL
constraints
ATL Rules
(as text)
ATL Rules
(as model)
Footprints of ATL
rules
Constraint Coverage Rule Coverage Relateness
 The matching function for the cell [i,j] (where Ci is the
set of footprints for the constratint i and Rj for the rule j) is
given by:
CCi,j =
Ci ∩ Rj
|Ci|
RCi,j =
|Ci ∩ Rj|
|Rj|
RCRi,j =
|Ci ∩ Rj|
|Ci ∪ Rj|
44

43. For the Families to Persons Case Study [1]…
[1] Burgueño L., Troya J., Wimmer M., Vallecillo A.: Static Fault Localization in Model Transformations. In IEEE Transactions on
Software Engineering, 2015.
45

44.  Research questions
 Are the detected alignments correct?
 Are the detected alignments complete?
 When more than one alignment is
reported, are the correctly identified
alignments outperforming the incorrectly
identified alignments?
46

45. UML to ER
CPL to SPL
• From the ATL zoo
• Call Processing Language to Session Processing Language
BibTeX to DocBook
• From the ATL Zoo
Ecore to Maude
• Used by the tool Maude
Four case studies
47

46. Measures to assess accuracy
 Precision
 Fraction of correctly detected alignments among the set of all detected alignments
 Recall
 Fraction of correctly detected alignments among the set of all actually occurring alignments
 f-measure
 Combination of Precision and Recall
TP = True positive
FP = False positive
FN = False negative
48

47. Results
TP = True positive
FP = False positive
FN = False negative
49

48. Faulty Transformations
 We have used mutation analysis to introduce bugs into MTs
 7 faults injected to the CPL2SPL transformation
 Our approach was able to detect all of them except one (addition of an out-
pattern to a rule) 50

49. A-priori applicability test: Similarity Matrices
 Gives us an indication of how rules are related with each other
 i.e., the factor of common types/features they.
 If they share many types and features, our approach is less likely to succeed.
 They have rules in both columns and rows
 Similarity matrix for the UML to ER case study:
 To calculate the fitness for the transformation,
 extract average and standard deviation
 The lower both values are  the fewer footprints rules have in common  the higher the chance for a
successful application of our approach is
 Recommend to apply our approach when the mean and standard deviation are below 0.15
 Average = 0.033; Deviation = 0.052 for the BT2DB case study
 Half of the MTs in the ATL zoo have values below 0.15 51

50. Extending Tracts for M2T and T2M
transformations
 Reduce the problem to an M2M transformation testing problem
M2T
T2M
public class CalculateArrayAverageExample {
public static void main(String[] args) {
//define an array
int[] numbers = new
int[]{10,20,15,25,16,60,100};
//calculate sum of all array elements
int sum = 0;
for(int i=0; i < numbers.length ; i++)
sum = sum + numbers[i];
//calculate average value
double average = sum /
numbers.length;
}
}
inject
 Inject complete repositories of text
artifacts into models conforming to a
generic metamodel
File.allInstances->forAll(f|
f.lines->exists(l|
l.matchesRE(‘class ’+f.name)))
 Libraries and operations to deal with
Strings and regular expressions in OCL
 Example of a target constraint for the
UML-to-Java case study
52

51.  Check the code generated by both commercial and open source UML tools
Evaluation
C1: Nested packages are transformed into nested
folders.
C6: Visibility of attributes mapped to Java.
C8: No Java keywords are allowed as names in
UML models.
C10: Generic classes mapped to Java.
53

52.  None of the tools performs well even with basic UML-to-Java code generators
 Several tools produce incorrect Java code (not compilable)
 Check the code generated by both commercial and open source UML tools
Evaluation
54

53. Conclusions and
Future Work
55

54. LinTra
 seems to be able to handle and scale larger models
 and significantly outperforms current model transformation engines, both sequential and parallel
Conclusions
Matching Tables
 Allow the engineer to trace the faulty rules that can be the cause of broken constraints
 Our evaluation shows that the approach is expected to be accurate for a large set of model transformations.
 By using the similarity matrixes, an automated and instant fitness test is available to check a-priori whether
the approach is likely to be helpful for a given transformation
Tracts for M2T and T2M transformations
 Show to be very useful for further improving code generators and reverse-engineering tools
 Real need for the discipline to develop and test M2T and T2M transformations
56

55. LinTra
 Introduce optimizations (caching local copies, etc.)
 Compiler from high-level MT languages to jLinTra
 Distribution of models and their transformation
 Extend the transformation of linear distribution chains to transformation networks
 In-place model that supports recursive matching
Future Work
Matching Tables
 Provide support to other type of constraints (PaMoMo, etc)
 Provide support to other languages that do not use OCL
Tracts for M2T and T2M transformations
 Study how dynamic approaches can be combined with ours
 Improve our tool support
57

56. LinTra
 Loli Burgueño, Manuel Wimmer, Antonio Vallecillo. A Linda-based Platform for the Parallel Execution of Out-
place Model Transformations. Information and Software Technology. Under review.
Main Publications
Matching Tables
 Loli Burgueño, Javier Troya, Manuel Wimmer, Antonio Vallecillo. Static Fault Localization in Model
Transformations. IEEE Transactions on Software Engineering 41(5):490-506, 2015.
Tracts for M2T and T2M transformations
 Loli Burgueño, Manuel Wimmer. Testing M2T/T2M Transformations. In Proc. of the ACM/IEEE 16th
International Conference on Model Driven Engineering Languages and Systems 2013 (MoDELS 2013).
Miami, FL, USA, September-October 2013.
58

57. Thanks!
59

58. On the Quality
Properties of Model
Transformations:
Performance and
Correctness
LOLI BURGUEÑO
SUPERVISED BY:
ANTONIO VALLECILLO AND MANUEL WIMMER
APRIL 28, 2016