An overview of software engineering project (MSc Thesis) to objectively evaluate the benefits and drawbacks of using Model-Driven Engineering tools on Eclipse Modelling Framework (EMF) such as Epsilon and Xtext to implement a compiler for a sufficiently complex intermediate language. The results seem promising. For more imnformation, please see this blog post: http://modeling-languages.com/re-implementing-apache-thrift-with-mde/

1. by Sina Madani
September 2016

2.  A software framework which enables cross-platform
(asynchronous) communication via Remote Procedure Call
 Does this through an Interface Definition Language (IDL)
 Define services and the functions provided by these services
 Supports complex data types (struct, list, map etc.)

3. MyCalculator.thrift
i32 add(i32 num1, i32 num2);
i32 subtract(i32 num1, i32 num2);
i64 multiply(i32 num1, i32 num2);
double divide(i32 num1, i32 num2);
Compiler
thrift –-gen [language]
“MyCalculator.thrift”
MyCalculator.java
Client code
Server code
MyCalculator.py
Client code
Server code
MyCalculator.cpp
Client code
Server code

4.  Re-implement (a subset of) Thrift’s compiler using MDE tools
– more specifically, the parser + 2 output languages
 Evaluate the model-driven approach and current MDE
technologies for implementing intermediate languages
 Compare to existing implementation to observe benefits and
drawbacks of the MDE approach

5.  No reports on replicating existing compilers using MDE
 Wide variance in productivity gains
 MDE adoption – good tools are high priority, but not the only
concern – “necessary but insufficient”
 Benefits: improved communication and transparency of design

6. .thrift file
flex
(Lexer)
Bison
(Parser)
Semantic
Validation
Code
generation
Output files
 C++ based (a primarily systems programming language)
 Low-level parsing tools (manually build AST and types)
 Validation performed at various stages
 Code generation difficult to read (lots of functions)

7.  Java-based tools
 Eclipse Modelling Framework (EMF)
 Xtext for parsing
 Epsilon for validation and code generation
 Solution centred around the IDL’s (meta)model

8.  Define the grammar using Xtext
 Self-documenting EBNF notation
 Automatically generates Ecore metamodel
 Lexing and parsing handled automatically
 Editor support with syntactic validation and highlighting

9.  Context-free grammar cannot capture language semantics
 Need to ensure semantics and scoping are valid
 For example, this is syntactically valid, but does it make sense?
const list<i32> myNumberz = “a string!”;
 Use Epsilon Validation Language to express these constraints

10.  Epsilon Generation Language (EGL) – template-based M2T
 Static sections output verbatim
 Dynamic sections expressed in Epsilon Object Language (EOL)
 Can use operations, imports, extended properties etc.
 EGX invokes EGL templates with parameters

11.  Took a TDD approach to implementing code generators
 Various complex Thrift IDL files available on website
 Auto-generated editor validates syntax
 Eclipse file comparison editor to highlight output differences
 Automated tests using JUnit (compare as strings or file size)

12. Stage/Implementation C++ (existing) EMF (model-driven)
Language definition
(parsing & validation)
3419 (105 KB) 447 (14 KB)
Language-agnostic code 712 (22KB) 1036 (26 KB)
Java generator 5129 (187 KB) 2224 (73 KB)
Ruby generator 1231 (40 KB) 422 (14 KB)
Overall total >10491 (~395 KB) 4149 (~128 KB)

13.  More concise – less code to do (more or less) the same job
 Reduced coupling and higher cohesion
 Higher level of abstraction – focus on what actually matters
 Easier to understand what’s happening at each stage
 Arguably more modifiable and easier to maintain as a result

14.  Greater dependence on tools and libraries (EMF, Xtext, Epsilon)
 Performance is significantly worse due to the overhead of these
tools compared to the “close-to-metal” C++ solution
 Troubleshooting and long-term support (skills, complexity of
underlying MDE technologies...)

15.  First complete re-implementation using MDE technologies of
such a complex system
 MDE solution is significantly more concise overall
 Main benefits come in the form of readability, cohesion and,
arguably, a more intuitive workflow
 Need to formally evaluate subjective aspects - e.g. readability,
modifiability, understandability, maintainability...
 Would the results be similar for other IDL frameworks?

16.  Thank you for listening!

17. .thrift IDL file
“Interpreter”
Grammar
definition file
Lexer/Parser
has a
Metamodel
makes a
conforms to
Validator
Model
elements
Constraints
uses
Code generator
Templates
Additional
files
Model
Model
uses
Output files in
target language
if no
constraints
are violated

18. Headers / imports
Regular Expressions Action to take
(code block)
Additional code
Generated parser
Output
directly
Output
directly
Thrift C++ compiler
flex and Bison