Slides of the PhD thesis defense

1. Valerio Cosentino
Advisor: Jordi Cabot
Co-advisor: Patrick Albert
PhD thesis defense
Nantes, 18 December 2013
A Model-Based Approach for
Extracting Business Rules out of
Legacy Information Systems
1

2. Outline
 Introduction
–Context & problem description
–Goal
–State of the art
 Business Rule Extraction framework
–Adaptations
– COBOL, Java, Relational databases
 Conclusion & further research
2

3. Context
 Our economic infrastructure relies on legacy
information systems
3
Information System

4. Context
 Legacy systems
 Embed business logic/policies to preserve
 Difficult/expensive to maintain and evolve
 Migration process
 High risk:
lack of a clear insight of the logic
enforced
 High cost:
ad-hoc solutions
4

5. Context
 IBM funded this thesis
 Why?
 Many of its customers rely on legacy
systems
 Extracting business logic from legacy
systems is economically valuable
5

6. Problem description
 Business rule[1]
:
 Statement that defines or constrains some
aspects of the business
 Applied on business terms
 Independent from programming languages
6
[1] Defining Business Rules. "What Are They Really." The Business Rules Group, 2000, 1-77
Organization
System
Business logic/policies = set of business rules
Business rule implementations

7. Problem description
 Business rules spread in the source code
 Poor/outdated system documentation
 The original developers may have left the
organization
7
misalignment
Organization
System
Business logic/policies = set of business rules
Business rule implementations

8. Goal
8
 Identify and extract the business rules in
legacy code
 Present the identified rules to different
typologies of users (technical/business users)
 Link the business rules identified to the source
code

9. State of the art
9
Business Rule Extraction
Software Comprehension
[2] Sneed, H.M., Erdös, K.: Extracting Business Rules from Source Code. In: Fourth Workshop
on Program Comprehension. (1996) 240–247
[3] Rugaber, S.: Program comprehension. Encyclopedia of Computer Science and Technology
35(20) (1995) 341–368
 Business Rule Extraction (BReX):
 Understanding of the business policies
enforced in a system[2]
 Part of the sofware comprehension[3]

10. State of the art
10
Behavioral part
Business Rule Extraction
Identify the business
variable in the code
Retrieve the pieces of
code related to these
variables
Represent/store the
identified pieces of
code
Structural part
Represent the data
definitions in a
conceptual model
Identify the database
constraints
Represent/store the
identified constraints

11. Outline
 Introduction
–Context & problem description
–Goal
–State of the art
 Business Rule Extraction framework
–Adaptations
– COBOL, Java, Relational databases
 Conclusion & further research
11

12. Business Rule Extraction framework
12
Behavioral part
Business Rule Extraction
Identify the business
variable in the code
Retrieve the pieces of
code related to these
variables
Represent/store the
identified pieces of
code
Structural part
Represent the data
definitions in a
conceptual model
Identify the database
constraints
Represent/store the
identified constraints
Business Term
Identification
Business Rule
Identification
Business Rule
Representation

13. Business Rule Extraction framework
13
Source code Model Representation
grammarware modelware
Model
Discovery
Business Term
Identification
Business Rule
Identification
Business Rule
Representation
 Generic and modular framework
 Traceability from the code to the rules
 Internal & external representations of rules

14. Outline
 Introduction
–Context & problem description
–Goal
–State of the art
 Business Rule Extraction framework
–Adaptations
– COBOL, Java, Relational databases
 Conclusion & further research
14

15. BReX framework for COBOL
 Motivation
 Still largely used
 200 billion lines of code in existance
 70%-75% of business/transaction systems
 90% of global financial transactions
 Shortage of COBOL developers
 Study on 357 IT professionals
 46% says they already notice a shortage
 50% says the average age is > 45
 22% says the average age is > 55
15

16. BReX framework for COBOL
 Procedural language
 Global data structures
 Group item (level number 01 - 49)
 Data item (level number 77)
 Divisions: identification, data, environment,
procedure
 Procedure
 Section, paragraph, sentence, statement
 The sequential control flow is modified
 PERFORM, GO TO, NEXT SENTENCE
16

17. BReX framework for COBOL
17
Bag
Money
Is open
Need Quantity
Price
 Business rules in the program
 If the shop is open, the customer can buy products
 If a product P is needed, the customer buys P
 If the customer has money/room, he can buy products
 If a product is available, it can be bought
 If a product is bought, its quantity is decreased by one

18. BReX framework for COBOL
18
 Program structure Data structures

19. BReX framework for COBOL
19
 The rule for PR-MEAT is spread in the code

20. BReX framework for COBOL
20
COBOL Model
Discovery
Business Term
Identification
Business Rule
Identification
Business Rule
Representation
COBOL code
Text
Graph

21. BReX framework for COBOL
21
COBOL Model
Discovery
Business Term
Identification
Business Rule
Identification
Business Rule
Representation
COBOL code
Text
Graph
COBOL Application
Model
Identify variables
carrying a business
meaning
Identify the code
related to the
business variable(s)
Represent the rules
for different users

22. BReX framework for COBOL
Model Discovery
22

23. BReX framework for COBOL
Business Term Identification
 Business relevant variables
 Conditions of conditional statements
 If-then, If-then-else, If-then-goto, etc.
 Mathematical statements
 Compute, add, subtract, etc.
 Initialisation statements
 MOVE « literals » TO var
 In/out statements
 Accept, display, etc.
23

24. BReX framework for COBOL
Business Term Identification
24

25. BReX framework for COBOL
Business Rule Identification
25
Control Flow
Analysis
Data Flow
Analysis
Rule
Discovery
COBOL model CFG
Business var(s)
Enriched CFG
Optional step to find
relations among variables

26. COBOL
Control flow analysis
26
 Rules to calculate the next statement

27. BReX framework for COBOL
Control Flow Analysis
27

28. BReX framework for COBOL
Rule Discovery
28
Business variable

29. BReX framework for COBOL
Rule Discovery
29
Rule

30. BReX framework for COBOL
Rule Discovery
30
Rule

31. BReX framework for COBOL
Rule Discovery
31
Rule Context

32. BReX framework for COBOL
Data Flow Analysis
32
Related variables

33. COBOL
Business Rule Identification
33

34. BReX framework for COBOL
Business Rule Representation
34
Vocabulary
Extraction
Application Vocabulary
Visualization
Enriched CFG
COBOL model
Code view
Voc view
Graph
text
text
Orchestration
Vars
Optional step

35. BReX framework for COBOL
Vocabulary extraction
35

36. BReX framework for COBOL
Textual output
36

37. BReX framework for COBOL
Graph output
37
PR-BREAD
PR-MEAT
MONEY

38. BReX framework for COBOL
Evaluation
 Analysis of an IBM COBOL application[*]
 14 programs, 130 variables, 6500 LOC
 4 internal IBM COBOL experts
 They knew the application
 They analysed the rules extracted by our
framework
 The framework is able to identify complete
business rules
– The verbalization can be improved
38
* Cosentino V., Bauquel P., Perronnet J., Albert P. & Cabot J. "Extracting Business Rules from
COBOL: A Model-Based Framework." Working Conference on Reverse Engineering. 2013.

39. BReX framework for Java
 Motivation
 Largely used
 Billions of devices rely on Java
 Many Java ISs are legacy
 Poor/out-of-date documentation
 Lack of a clear insight of the business logic
enforced
39

40. BReX framework for Java
40
Java Model
Discovery
Business Term
Identification
Business Rule
Identification
Business Rule
Representation
Java code
Text
Graph
MoDisco

41. BReX framework for Java
Business Term Identification
 Business relevant variables
 Conditions of conditional/loop statements
 If, for, while, etc.
 Mathematical statements
 Infix, prefix and postfix expressions
 Initialisation statements
 Assignments, variable declarations
 In/out statements
 BufferedReader, System.out, etc.
41

42. BReX framework for Java
Business Rule Identification
42
Rule
Discovery
Java model
Business var(s)
Rule-annotated
Java model
Business rule
model
Business Rule
Model Extraction
CF & DF info

43. BReX framework for Java
Business Rule Representation
43
Vocabulary
Extraction
Application Vocabulary
Visualization
Code view
Voc view
Graph
text
text
Orchestration
Vars
Class
Method
Business rule model
Java model

44. Related work
System behavioral part (selection)
44
[2] Sneed et al. "Extracting Business Rules from Source Code." Fourth Workshop on Program
Comprehension. 1996.
[6] Huang et al. "Business rule extraction from legacy code." Computer Software and Applications
Conference, 1996.
[7] Lösch, Felix. Instrumentation of Java program code for control flow analysis. Diss. 2005.
[8] Putrycz et al. "Recovering business rules from legacy source code for system modernization."
Advances in Rule Interchange and Applications, 2007
[9] Fu et al. "A framework for business rule presentation." Database and Expert Systems
Applications, 2001.
intrusive
Internal-external
representation
Traceability
Vocabulary-based
representation
Orchestration
[2] [6] [7] [8] [9] BReX

45. BReX framework for Relational DB
 Motivation
 Key role in most organizations
 Discover inconsistencies with the business
logic implemented in the behavioral part of
the system
45

46. Business rule extraction framework
Relational databases
DB Model
Discovery
Business Term
Identification
Business Rule
Identification
Business Rule
Representation
DB implementation
46
OCL
UML+OCL
Schema to UML
class diagrams
Constraints to OCL
 UML is a visual language to create models of object-oriented systems
 OCL is a constraint language used to define constraints that cannot be
expressed in UML

47. BReX framework for Relational DB
Business Rule Identification
 Declarative constraints
 PRIMARY KEY
 CHECK
 UNIQUE
 NOT NULL
 Operational constraints
 Triggers
 Stored procedures
 Mapping from SQL to OCL
47

48. BReX framework for Relational DB
Declarative constraints to OCL
48
 Mappings

49. BReX framework for Relational DB
Triggers to OCL
49
 All triggers that embed a PL/SQL statement
raising an user-defined exception are classified
as constraint rules
PL/SQL block
Exception
handlers
 PL/SQL extends SQL with procedural language elements (conditions,
loops, variables, procedures, functions, types, ...)

50. BReX framework for Relational DB
Triggers to OCL
50
 Mapping

51. BReX framework for Relational DB
Triggers to OCL
51
 Example

52. Related work
System structural part
52
[5] Chaparro, O., Aponte, J., Ortega, F., Marcus, A.: Towards the
Automatic Extraction of Structural Business Rules from Legacy Databases.
In: Working Conference on Reverse Engineering. (2012) 479–488
[4] Cabot, J., Gómez, C., Planas, E., Rodríguez, M.E.: Reverse Engineering
of OO constructs in Object-Relational Database Schemas. Jornadas de
Ingenieria del Software y Bases de Datos (2008)
 DB constraints:
 to OCL[4]
 to SBVR[5]
 Focus only on simple declarative
constraints that do not contain SQL
statements

53. Outline
 Introduction
–Context & problem description
–Goal
–State of the art
 Business Rule Extraction framework
–Implementations
– COBOL, Java, Relational databases
 Conclusion & further research
53

54. Conclusion
 Business rule extraction framework
 Model-based approach
 Generic and modular
 Traceability from the code to the rules
 Internal & external representations of rules
 System behavioral part
 COBOL
 Java
 System structural part
 Relational databases
54

55. Publications
55
 COBOL
 Cosentino V., Bauquel P., Perronnet J., Albert P. & Cabot J. "Extracting Business
Rules from COBOL: A Model-Based Framework." Working Conference on Reverse
Engineering. 2013. International conference
 Cosentino V., Bauquel P., Perronnet J., Albert P. & Cabot J. "Extracting Business
Rules from COBOL: A Model-Based Tool." Working Conference on Reverse
Engineering. 2013. Demo tool in international conference
 Cosentino V., Bauquel P., Perronnet J., Albert P. & Cabot J. "Un Framework dirigé par
les modèles pour l'extraction de règles métier à partir d'applications COBOL."
Conférence en IngénieriE du Logiciel. 2013. National conference
 Java
 Cosentino V., Bauquel P., Perronnet J., Albert P. & Cabot J. "A model driven reverse
engineering framework for extracting business rules out of a Java application." Rules
on the Web: Research and Applications. 2012. International conference
 Relational databases
 Cosentino V. & Martinez S. "Extracting UML/OCL Integrity Constraints and Derived
Types from Relational Databases." 13th International Workshop on OCL, Model
Constraint and Query Languages. 2013. International workshop

56. Further research
 Dead code identification
 Automatic validation of business rules
 Code verbalization
 Business rule extraction process for the
presentation part of the system
 Identification/extraction of business rules that
cross different languages used in the system
 Rule-driven system re-documentation
56

57. Q & A
57
Source code Model Representation
grammarware modelware
Model
Discovery
Business Term
Identification
Business Rule
Identification
Business Rule
Representation

58. Backup
58

59. COBOL
Rule Discovery
59
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9

60. COBOL
Rule Discovery
60
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9

61. COBOL
Rule Discovery
61
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s4 s5 s7 s8 s9
0 0 x x 0

62. COBOL
Rule Discovery
62
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0

63. COBOL
Rule Discovery
63
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0

64. COBOL
Rule Discovery
64
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x

65. COBOL
Rule Discovery
65
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0

66. COBOL
Rule Discovery
66
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}

67. COBOL
Rule Discovery
67
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}

68. COBOL
Rule Discovery
68
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}
P7: {s4, s8}

69. COBOL
Rule Discovery
69
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}
P7: {s4, s8}
P8: {s4, s8, s9}

70. COBOL
Rule Discovery
70
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}
P7: {s4, s8}
P8: {s4, s8, s9}

71. COBOL
Rule Discovery
71
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}
P7: {s4, s8}
P8: {s4, s8, s9}

72. COBOL
Rule Discovery
72
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}
P7: {s4, s8}
P8: {s4, s8, s9}

73. COBOL
Rule Discovery
73
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}
P7: {s4, s8}
P8: {s4, s8, s9}

74. COBOL
Rule Discovery
74
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
s4
s5
s7
s8
s9
s4 s5 s7 s8 s9
0 0 x x 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 x
0 0 0 0 0
P1: {s4}
P2: {s5}
P3: {s7}
P4: {s8}
P5: {s9}
P6: {s4, s7}
P7: {s4, s8}
P8: {s4, s8, s9}

75. COBOL
Rule Discovery
75
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
P1: {s4, s7}
P2: {s4, s8, s9}
P3: {s5}

76. COBOL
Rule Discovery
76
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
P1: {s4, s7}
P2: {s4, s8, s9}
P3: {s5}

77. COBOL
Rule Discovery
77
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
Rule-1: {{s2, s4}, {s6, s7}}
P1: {s4, s7}
P2: {s4, s8, s9}
P3: {s5}

78. COBOL
Rule Discovery
78
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
Rule-1: {{s2, s4}, {s6, s7}}
Rule-2: {{s2, s4}, {!s6, s8}, s9}
P1: {s4, s7}
P2: {s4, s8, s9}
P3: {s5}

79. COBOL
Rule Discovery
79
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
Rule-1: {{s2, s4}, {s6, s7}}
Rule-2: {{s2, s4}, {!s6, s8}, s9}
Rule-3: {s5}
P1: {s4, s7}
P2: {s4, s8, s9}
P3: {s5}

80. COBOL
Rule Discovery
80
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
Rule-1: {{s2, s4}, {s6, s7}}
Rule-2: {{s2, s4}, {!s6, s8}, s9}
Rule-3: {s5}

81. Business rule extraction framework
Rule Discovery
81
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
Context-1: {s0, {Rule-1}}
Rule-1: {{s2, s4}, {s6, s7}}
Rule-2: {{s2, s4}, {!s6, s8}, s9}
Rule-3: {s5}

82. Business rule extraction framework
Rule Discovery
82
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
Context-1: {s0, {Rule-1}}
Context-2: {s0, {Rule-2}}
Rule-1: {{s2, s4}, {s6, s7}}
Rule-2: {{s2, s4}, {!s6, s8}, s9}
Rule-3: {s5}

83. Business rule extraction framework
Rule Discovery
83
s0
s1
s2
s3
s5
s4
s6
s7 s8
s9
Branches: s0, s2, s6
Context-1: {s0, {Rule-1}}
Context-2: {s0, {Rule-2}}
Context-3: {s0, s2, {Rule-3}}
Rule-1: {{s2, s4}, {s6, s7}}
Rule-2: {{s2, s4}, {!s6, s8}, s9}
Rule-3: {s5}

84. Business rule extraction framework
Rule Discovery
84
boolean alive
method-A setDead() ... method-n
uses
method-C incrementAge()
invokes
... invokes
...
granularity-0
granularity-1
granularity-nmain(...)

85. Business rule extraction framework
Rule Discovery
85
granularity-0
granularity-1
@related: {setDead, incrementAge, ..., main}
@sliced-variable: alive
boolean alive
method-A setDead() ... method-n
uses
method-C incrementAge()
invokes
... invokes
...
main(...) granularity-n

86. Business rule extraction framework
Rule Discovery
86
incrementAge()
{
setAge(getAge() + 1);
if (getAge() > maxAge)
setDead();
}
maxAge @related-variable
getAge() @reachable
@rule
@context

87. Business rule extraction framework
Rule Discovery
87
getAge()
method-X method-Y ... method-n
contained invocations
method-W
contained invocations
...
reachability-0
reachability-1

88. Business rule extraction framework
Rule Discovery
88

89. Business rule extraction framework
Rule Discovery
89

90. Business rule extraction framework
Rule Discovery
90