ICT role in 21st century education and it's challenges.
Virtual Separation of Concerns
1. Photo (cc) Horia Varlan
Virtual Separation of Concerns
Toward preprocessors 2.0
Christian Kästner
2. Christian Kästner: Virtual Separation of Concerns 2
Software Product Lines & Features
• Set of related software systems (variants)
in a domain
• Generated from common code base
• Variants distinguished in terms of features
3. Christian Kästner: Virtual Separation of Concerns 3
Conditional Compilation with
Preprocessors
● Annotated code static int _rep_queue_filedone(...
DB_ENV *dbenv;
fragments REP *rep;
__rep_fileinfo_args *rfp; {
● Variants with and #ifndef HAVE_QUEUE
COMPQUIET(rep, NULL);
without features COMPQUIET(rfp, NULL);
return (__db_no_queue_am(dbenv
#else
db_pgno_t first, last;
● Common for product- u_int32_t flags;
int empty, ret, t_ret;
line implementation #ifdef DIAGNOSTIC
DB_MSGBUF mb;
in industry #endif
...
Excerpt from Oracle’s Berkeley DB
Preprocessors in C, C++, Java ME, Pascal, Erlang, C#, Visual Basic, …
GPP, GNU M4, SPLET, Frames/XVCL, Gears, pure::variants
4. Christian Kästner: Virtual Separation of Concerns 4
Objections / Criticism
Designed in the 70th and hardly evolved since
“#ifdef considered harmful”
“#ifdef hell”
“maintenance becomes a ‘hit or miss’ process”
“is difficult to determine if the code being
viewed is actually compiled into the system”
“incomprehensible source texts”
“programming errors are easy “CPP makes maintenance difficult”
to make and difficult to detect”
“source code rapidly
“preprocessor diagnostics are poor” becomes a maze”
6. Christian Kästner: Virtual Separation of Concerns 6
Agenda
• Problems and Advantages of Preprocessors
• 4 Improvements
• Views
• Visual Representation
• Disciplined Annotations
• Product-Line-Aware Type System
• Summary and Perspective
7. Christian Kästner: Virtual Separation of Concerns 7
Problem 1: Lack of Separation of Concerns
• Scattered and tangled source code
• Global search to understand a feature
• Deleting obsolete feature as challenge
• Difficult distributed development
• Difficult reuse
Beispiel Berkeley DB:
Feature Sperren mit 1835 LOC
in 28 von 300 Dateien
an 155 Stellen
8. Christian Kästner: Virtual Separation of Concerns 8
Problem 2: Obfuscation
• Mixture of two class Stack {
class Stack {
void push(Object o
languages void push(Object o
#ifdef SYNC
#ifdef SYNC
, Transaction txn
(C und #ifdef, …) , Transaction txn
#endif
#endif
) {
• Control flow hard to ) {
if (o==null
if (o==null
#ifdef SYNC
#ifdef SYNC
follow || txn==null
|| txn==null
#endif
#endif
• Additional line breaks ) return;
) return;
#ifdef SYNC
#ifdef SYNC
destroy code layout Lock l=txn.lock(o);
Lock l=txn.lock(o);
#endif
#endif
• Long annotations elementData[size++] = o;
elementData[size++] = o;
#ifdef SYNC
#ifdef SYNC
difficult to recognize l.unlock();
l.unlock();
#endif
#endif
fireStackChanged();
fireStackChanged();
}
}
}
}
10. Christian Kästner: Virtual Separation of Concerns 10
Problem 3: Error-Prone
• Syntax Errors • Type Errors
static int _rep_queue_filedone(...) #ifdef TABLES
DB_ENV *dbenv; class Table {
REP *rep; void insert(Object data,
__rep_fileinfo_args *rfp; { Txn txn) {
#ifndef HAVE_QUEUE storage.set(data,
COMPQUIET(rep, NULL); txn.getLock());
COMPQUIET(rfp, NULL); }
return (__db_no_queue_am(dbenv)); }
#else #endif
db_pgno_t first, last; class Storage {
u_int32_t flags; #ifdef WRITE
int empty, ret, t_ret; boolean set(…) { ... }
#ifdef DIAGNOSTIC #endif
DB_MSGBUF mb; }
#endif
// over 100 lines of additional code
}
#endif
11. Christian Kästner: Virtual Separation of Concerns 11
Problem 3: Error-Prone
• Syntax Errors • Type Errors
static int _rep_queue_filedone(...) #ifdef TABLES
DB_ENV *dbenv; class Table {
REP *rep; void insert(Object data,
__rep_fileinfo_args *rfp; { Txn txn) {
#ifndef HAVE_QUEUE storage.set(data,
COMPQUIET(rep, NULL); txn.getLock());
COMPQUIET(rfp, NULL); }
return (__db_no_queue_am(dbenv)); }
#else #endif
db_pgno_t first, last; class Storage {
u_int32_t flags; #ifdef WRITE
int empty, ret, t_ret; boolean set(…) { ... }
#ifdef DIAGNOSTIC #endif
DB_MSGBUF mb; }
#endif
// over 100 lines of additional code
}
#endif
12. Christian Kästner: Virtual Separation of Concerns 12
Problem 3: Error-Prone
• Syntax Errors • Type Errors
static int _rep_queue_filedone(...) #ifdef TABLES
DB_ENV *dbenv; class Table {
REP *rep; void insert(Object data,
__rep_fileinfo_args *rfp; { Txn txn) {
#ifndef HAVE_QUEUE storage.set(data,
COMPQUIET(rep, NULL); txn.getLock());
COMPQUIET(rfp, NULL); }
return (__db_no_queue_am(dbenv)); }
#else #endif
db_pgno_t first, last; class Storage {
u_int32_t flags; #ifdef WRITE
int empty, ret, t_ret; boolean set(…) { ... }
#ifdef DIAGNOSTIC #endif
DB_MSGBUF mb; }
#endif
// over 100 lines of additional code
}
#endif
13. Christian Kästner: Virtual Separation of Concerns 13
Advantages
• Easy to use
• “annotate and remove”
• Already part of many languages / simple tools
• Most developers already familiar
• Flexible / Expressive
• Language-independent / Uniform
• Low Adoption Barrier
• esp. with legacy code
14. Christian Kästner: Virtual Separation of Concerns 14
Agenda
• Problems and Advantages of Preprocessors
• 4 Improvements
• Views
• Visual Representation
• Disciplined Annotations
• Product-Line-Aware Type System
• Summary and Perspective
17. Christian Kästner: Virtual Separation of Concerns 17
View on Feature Eval
• Shows all files containing Eval code
• Shows context information (kursiv)
• Hides code without annotations
18. Christian Kästner: Virtual Separation of Concerns 18
View on the Variant “Add and Eval”
• Entirely hides file Power
• Hides method print in Add ([…])
• Code without annotation remains visible
19. Christian Kästner: Virtual Separation of Concerns 19
Agenda
• Problems and Advantages of Preprocessors
• 4 Improvements
• Views
• Visual Representation
• Disciplined Annotations
• Product-Line-Aware Type System
• Summary and Perspective
21. Christian Kästner: Virtual Separation of Concerns 21
Alternative Visual Representations
● NetBeans ● Spotlight
22. Christian Kästner: Virtual Separation of Concerns 22
Scaling Visual Representations
• Focus on few features at a time
• Repeating colors / manual assignment
sufficient
• Analysis of 4 Java ME and 40 C programs:
• 96 % pages of source code with ≤ 3 colors
• 99 % pages of source code with ≤ 7 colors
23. Christian Kästner: Virtual Separation of Concerns 23
Program Comprehension: An Experiment
• #ifdef vs. colors
• 43 subjects in 2
groups
• S1-2: search tasks
faster with colors
(43% & 23%)
• M1-3: maintenance
tasks same perform.
• M4: maintenance
task with red back-
ground color -37%
• No influence on correctness
• Subjects prefer colors
24. Christian Kästner: Virtual Separation of Concerns 24
Agenda
• Problems and Advantages of Preprocessors
• 4 Improvements
• Views
• Visual Representation
• Disciplined Annotations
• Product-Line-Aware Type System
• Summary and Perspective
25. Christian Kästner: Virtual Separation of Concerns 25
[TOOLS‘09]
Disciplined Annotations
class Foo { }
• Syntax errors caused by
annotations on plain text class Foo { }
• Solution: Use underlying artifact structure
• Enforce disciplined annotations on entire classes,
methods, or statements
• Annotations of isolated brackets or ClassDeclaration
Name=C
keywords regarded as undisciplined MethodDeclaration
Name=m
ReturnType Parameter
class C { Type-void Name=p
Block
void m(int p) {
s1();
MethodInvk MethodInvk
s2(p, true); Name=s1 Name=s2
}
} Parameter Parameter
Name=p Name=true
26. Christian Kästner: Virtual Separation of Concerns 26
Variant Generation by AST Transformation
ClassDeclaration ClassDeclaration
Name=C Name=C
MethodDeclaration
MethodDeclaration
Name=m
Name=m
ReturnType Parameter
Block
Type-void Name=p ReturnType Parameter
Block
Type-void Name=p
MethodInvk MethodInvk
Name=s1 Name=s2
variant MethodInvk
Name=s1
Parameter Parameter generation
Name=p Name=true
print /
parsing unparse
class C { class C {
void m(int p) { void m(int p) {
s1(); s1();
s2(p, true); }
} }
}
Variant generation cannot cause syntax errors.
27. Christian Kästner: Virtual Separation of Concerns 27
Expressiveness of Disciplined Annotations
• Not all annotations are possible
• Sometimes workarounds required
• Experience: not a significant restriction in
practice
• ~90% of all annotations in 40 C programs is
disciplined already
high flexibility no flexibility
unrestricted disciplined annotations classes no
annotations (underlying structure) only annotations
on any character
28. Christian Kästner: Virtual Separation of Concerns 28
Agenda
• Problems and Advantages of Preprocessors
• 4 Improvements
• Views
• Visual Representation
• Disciplined Annotations
• Product-Line-Aware Type System
• Summary and Perspective
29. Christian Kästner: Virtual Separation of Concerns 29
[ASE’08]
Product-Line-Aware Type System
class Database {
Storage storage;
• Base: Type correct without void insert(Object data
Txn txn) {
annotations (for tool support) storage.set(data,
txn.getLock())
• Detecting all pairs: }
}
• Method invocation and class Storage {
#ifdef WRITE
method declaration boolean set(…) { ... }
• Reference and declaration #endif
}
of class, variable, etc.
• … Related Work:
• Comparison of annotations for • SafeGen
• fmp2rsm
each pair • Safe Composition
• FFJPL
Declaration annotated + Error in some • Conditional
Reference not annotated variants Methods
30. Christian Kästner: Virtual Separation of Concerns 30
Type Checking in the Context
of a Feature Model
¬Read
• Handling dependencies class Database {
between features void insert(…) {
storage.set(…);
• Can all variants with
}
}
class Storage {
reference reach a boolean set(…) { ... }
corresp. declaration? }
Write
FM = API ∧ ( API ⇒
• Implementation: Propositional (read ∨ write))
logic and SAT solvers:
In all variants in which code fragment A is
included also code fragment B is included:
FM ⇒ ( AT (a ) ⇒ AT (b))
31. Christian Kästner: Virtual Separation of Concerns 31
[ASE’08]
Formalization: CFJ
● On top of Featherweight Java
● Theorem: Generation preserves typing
● Formal proof with Coq
33. Christian Kästner: Virtual Separation of Concerns 33
[GPCE’09]
Automated Refactorings
• Feature Module ● Annotationen
class Stack {
class Stack { class Stack {
class Stack {
int[] data;
int[] data; int[] data;
int[] data;
Base
void push(int o) { … }
void push(int o) { … } #ifdef UNDO
#ifdef UNDO
int pop() { /*...*/ }
int pop() { /*...*/ } int backup;
int backup;
}
} void undo() { /*...*/ }
void undo() { /*...*/ }
#endif
#endif
refines class Stack {
refines class Stack { #ifdef TOP
#ifdef TOP
Automated Transformationtop() { /*...*/ }
Top
int top() { /*...*/ }
int top() { /*...*/ } int top() { /*...*/ }
int
}
} #endif
#endif
void push(int o) {
void push(int o) {
refines class Stack {
refines class Stack { #ifdef UNDO
#ifdef UNDO
int backup;
int backup; backup = top();
backup = top();
void undo() { /*...*/ }
void undo() { /*...*/ } #endif
#endif
Undo
void push(int o) {
void push(int o) { /*...*/
/*...*/
backup = top();
backup = top(); }
}
original(v);
original(v); int pop() { /*...*/ }
int pop() { /*...*/ }
}
} }
}
34. Christian Kästner: Virtual Separation of Concerns 34
Agenda
• Problems and Advantages of Preprocessors
• 4 Improvements
• Views
• Visual Representation
• Disciplined Annotations
• Product-Line-Aware Type System
• Summary and Perspective
35. Christian Kästner: Virtual Separation of Concerns 35
Summary
• Problems: • Improvements:
• Separation of Concerns • Views
• Obfuscation • Visual representation
• Error-proneness • Disciplined annotations
• Product-line-aware type
system
• Retained advantages:
• Easy to use
• Flexible
• Language-independent
• Low adoption barrier
“Virtual Separation of Concerns”
36. Christian Kästner: Virtual Separation of Concerns 36
Summary
• Problems: • Improvements:
• Separation of Concerns • Views
• Obfuscation • Visual representation
• Error-proneness • Disciplined annotations
• Product-line-aware type
system
• Retained advantages:
• Easy to use
• Flexible
• Language-independent
• Low adoption barrier
“Virtual Separation of Concerns”
37. Christian Kästner: Virtual Separation of Concerns 37
Summary
• Problems: • Improvements:
• Separation of Concerns • Views
• Obfuscation • Visual representation
• Error-proneness • Disciplined annotations
• Product-line-aware type
system
• Retained advantages:
• Easy to use • Remaining Problems:
• Flexible • Separate Compilation
• Language-independent • Black-box Reuse
• Low adoption barrier
“Virtual Separation of Concerns”
38. Christian Kästner: Virtual Separation of Concerns 38
Perspective
• Preprocessors common in practice,
despite strong criticism
• Neglected by researchers
• Tool support can address most problems
• Interim solution (with migration path) or
serious alternative?
• Give preprocessors a second chance!