Groovy DSLs - S2GForum London 2011 - Guillaume Laforge

Designing your own
Domain-Specific Language
with Groovy

Guillaume Laforge
Groovy Project Manager
SpringSource / VMware

Guillaume Laforge

• Groovy Project Manager
• JSR-241 Spec Lead
• Head of Groovy Development
at SpringSource / VMWare
• Initiator of the Grails framework
• Creator of the Gaelyk toolkit
• Co-author of Groovy in Action

• Speaker: JavaOne, QCon, JavaZone, Sun TechDays,
Devoxx, The Spring Experience, SpringOne, JAX, Dynamic
Language World, IJTC, and more...

2

Domain-Specific Languages
are everywhere!

Technical dialects
Notations
Real-life Groovy DSLs

1. e4 e5
2. Nf3 Nc6
3. Bb5 a6

L2 U F -1 B L2 F B -1 U L2

Real-life Groovy examples

• Anti-malaria drug resistance simulation
• Human Resources employee skills representation
• Insurance policies risk calculation engine
• Loan acceptance rules engine for a financial platform
• Mathematica-like lingua for nuclear safety simulations
• Market data feeds evolution scenarios

• and many more...

10

Subject Matter Expers,
Business Analysts...

Developer producing LOLCODE

HAI
CAN HAS STDIO?
I HAS A VAR
IM IN YR LOOP
UP VAR!!1
VISIBLE VAR
IZ VAR BIGGER THAN 10?
KTHXBYE
IM OUTTA YR LOOP

And in the end...
nobody understands
each other

Expressing Requirements...

17

DSL: a potential solution?

• Use a more expressive language
than a general purpose one
• Share a common metaphore of understanding
between developers and subject matter experts
• Have domain experts help with the design
of the business logic of an application
• Avoid cluttering business code with
too much boilerplate technical code
• Cleanly separate business logic from application code
• Let business rules have their own lifecycle

18

Towards more readibility

19


20%

19


20


80%

20

Three levels of techniques

Flexible &
malleable syntax Meta- Hooking into
programming the compiler
• scripts
• optional typing • POGO • AST traversal
• native syntax • categories • local & global
constructs • builders transformations
• custom MetaClass • hooks into Antlr
• parens / semi • ExpandoMetaClass
ommission
• command chains
• named arguments
• BigDecimal
• operator overloading
• closures

21

Level #1:
A flexible and malleable syntax

Scripts vs classes

• Hide all the boilerplate technical code
– an end-user doesn’t need to know about classes

23

Scripts vs classes


public class Rule {
    public static void main(String[] args) {
        System.out.println("Hello");
    }
}

23

Scripts vs classes


public class Rule {
    public static void main(String[] args) {
        System.out.println("Hello");
    }
}

println "Hello"

23

Optional typing

• No need to bother with types or even generics
– unless you want to!
– but strongly typed if you so desire

24

Optional typing


// given this API method
public Rate<LoanType, Duration, BigDecimal>[] lookupTable() { ... }

24

Optional typing



// verbose Java notation
Rate<LoanType, Duration, BigDecimal>[] = lookupTable();

24

Optional typing



// verbose Java notation
Rate<LoanType, Duration, BigDecimal>[] = lookupTable();

// leaner Groovy variant
def table = lookupTable()

24

Native syntax constructs

25


// Lists

25


// Lists
def days = [Monday, Tuesday, Wednesday]

25


// Lists

// Maps

25


// Lists

// Maps
def states = [CA: 'California', TX: 'Texas']

25


// Lists

// Maps

// Ranges (you can create your own)

25


// Lists

// Maps

def bizDays = Monday..Friday

25


// Lists

// Maps

def bizDays = Monday..Friday
def allowedAge = 18..65

25

Optional parens & semis

• Make statements and expressions
look more like natural languages

26



move(left);

26



move(left);

move left

26

Named arguments

• In Groovy you can mix named and unnamed arguments for
method parameters
– named params are actually put in a map parameter
– plus optional parens & semis

  take 1.pill, 
   of: Chloroquinine, 
after: 6.hours

// Calls a method signature like:
def take(Map m, MedicineQuantity mq)

27

Command chains expressions

28


• A grammar improvement allowing you to
drop dots & parens when chaining method calls
– an extended version of top-level statements like println

28



• Less dots, less parens allow you to
– write more readable business rules
– in almost plain English sentences
• (or any language, of course)

28



• Less dots, less parens allow you to
– write more readable business rules
– in almost plain English sentences
• (or any language, of course)

• Let’s have a look at some examples

28


 turn left  then right 

29


Alternation of
method names


29


Alternation of
method names


and parameters
(even named ones)

29


Equivalent to:

     (    ).    (     )

29

LookM a!
N op are ns,
no do ts!


take 2.pills  of  chloroquinine  after  6.hours

31


    (       ).   (             ).      (       )
take 2.pills  of  chloroquinine  after  6.hours

31


// environment initialization
Integer.metaClass.getPills { ‐> delegate }
Integer.metaClass.getHours { ‐> delegate }
 
// variable injection
def chloroquinine = /*...*/
 

{ }
// implementing the DSL logic
def take(n) {
    [of: { drug ‐>
        [after: { time ‐> /*...*/ }]
    }]
}

// ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
take 2.pills of chloroquinine after 6.hours

32


take 2 .pills of  chloroquinine after  6.hours

33


take 2 .pills of  chloroquinine after  6.hours

... some dots remain ...

33


take 2  pills of  chloroquinine after  6 hours

34


    ( ).     (  ).             (     ). (     )
take 2  pills of  chloroquinine after  6 hours

34


// variable injection
def (of, after, hours) = /*...*/
 
// implementing the DSL logic

{ }
def take(n) {
    [pills: { of ‐>
        [chloroquinine: { after ‐>
            ['6': { time ‐> }]
        }]
    }]
}

// ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
take 2 pills of chloroquinine after 6 hours

35


36


// methods with multiple arguments (commas)

36


take coffee  with sugar, milk  and liquor

36



// leverage named‐args as punctuation

36



check that: margarita  tastes good

36




// closure parameters for new control structures

36




given {}  when {}  then {}

36





// zero‐arg methods require parens

36





select all  unique() from names

36






// possible with an odd number of terms

36






take 3  cookies

36


    (      ).    (           ).   (      )





36


    (      ).    (           ).   (      )

     (               ).      (    )




36


    (      ).    (           ).   (      )

     (               ).      (    )

     (  ).    (  ).    (  )



36


    (      ).    (           ).   (      )

     (               ).      (    )

     (  ).    (  ).    (  )

      (   ).        .    (     )


36


    (      ).    (           ).   (      )

     (               ).      (    )

     (  ).    (  ).    (  )

      (   ).        .    (     )

    ( ).

36

BigDecimal by default

• Main reason why financial institutions often decide
to use Groovy for their business rules!
– Although these days rounding issues are overrated!

37



BigDecimal uMinusv = c.subtract(a); 
BigDecimal vMinusl = b.subtract(c); 
BigDecimal uMinusl = a.subtract(b); 
return e.multiply(uMinusv)
  .add(d.multiply(vMinusl))
  .divide(uMinusl;

37



BigDecimal uMinusv = c.subtract(a); 
BigDecimal vMinusl = b.subtract(c); 
BigDecimal uMinusl = a.subtract(b); 
return e.multiply(uMinusv)
  .add(d.multiply(vMinusl))
  .divide(uMinusl;

(d * (b ‐ c) + e * (c ‐ a)) / (a ‐ b)

37

Custom control structures w/ closures

• When closures are last, they can be put “out” of the
parentheses surrounding parameter

unless (account.balance > 100.euros, { 
    account.debit 100.euros 
})

38



unless (account.balance > 100.euros, { 
    account.debit 100.euros 
})

// calling the following method:
def unless(boolean b, Closure c)

38



unless (account.balance > 100.euros) {
    account.debit 100.euros
}

// calling the following method:
def unless(boolean b, Closure c)

38

Operator overloading

a + b  // a.plus(b) • Currency amounts
a ‐ b  // a.minus(b)
a * b  // a.multiply(b) – 15.euros + 10.dollars
a / b  // a.divide(b)
a % b  // a.modulo(b) • Distance handling
a ** b // a.power(b) – 10.kilometers - 10.meters
a | b  // a.or(b)
a & b  // a.and(b) • Workflow, concurrency
a ^ b  // a.xor(b)
a[b]   // a.getAt(b) – taskA | taskB & taskC
a << b // a.leftShift(b)
a >> b // a.rightShift(b) • Credit an account
+a     // a.unaryPlus() – account << 10.dollars
‐a     // a.unaryMinus() account += 10.dollars
~a     // a.bitwiseNegate() account.credit 10.dollars

39

Level #2:
Dynamic metaprogramming

The MOP
(Meta-Object Protocol)

Groovy’s MOP

• All the accesses to methods, properties, constructors,
operators, etc. can be intercepted thanks to the MOP

• While Java’s behavior is hard-wired
at compile-time in the class

• Groovy’s runtime behavior is adaptable
at runtime through the metaclass

• Different hooks for changing the runtime behavior
– GroovyObject, custom MetaClass implementation,
categories, ExpandoMetaClass

42

Hooks

• GroovyObject(Support)
– invokeMethod(), get/setProperty(), get/setMetaClass()
– methodMissing(), propertyMissing()

• Custom MetaClasses add
– invokeConstructor(), invokeMissingMethod(),
invokeStaticMethod()
– get/setAttribute()
– respondsTo(), hasProperty()

• Categories
– thread-scope & lexical-scope addition of methods

43

Hooks: ExpandoMetaClass

• A DSL for MetaClasses!

MoneyAmount.metaClass.constructor = { ... }
Number.metaClass.getDollars = { ... }
Distance.metaClass.toMeters = { ... }
Distance.metaClass.static.create = { ... }

• To avoid repetition of Type.metaClass, you can pass a
closure to metaClass { ... }

• The delegate variable in closure represents the current
instance, and it the default parameter

44

Adding properties to numbers

• Three possible approaches

– create a Category
• a category is a kind of decorator for default MCs

– create a custom MetaClass
• a full-blown MC class to implement
and to set on the POGO instance

– use ExpandoMetaClass
• friendlier DSL approach but with a catch

45

With a Category

class DistanceCategory {
    static Distance getMeters(Integer self) {
        new Distance(self, Unit.METERS)
    }
}
 
use(DistanceCategory) {
    100.meters
}  

• Interesting scope: thread-bound & lexical
• Have to surround with “use”
– but there are ways to hide it

46

With an ExpandoMetaClass

Number.metaClass.getMeters = {‐> 
    new Distance(delegate, Unit.METERS) 
}
 
100.meters

• But the catch is it’s really a global change,
so beware EMC enhancements collisions

47

A builder for HR

softskills {
    ideas {
        capture 2
        formulate 3 
    }
    ...
}
knowhow {
    languages {
        java 4
        groovy 5
    }
    ...
}  

49

A builder for HR

          (
softskills {
         (
    ideas {
               ( )
        capture 2
                 ( )
        formulate 3 
     )
    }
    ...
 )
}
       (
knowhow {
             (
    languages {
            ( )
        java 4
              ( )
        groovy 5
     )
    }
    
    ...
 )
}  

49

Level #3:
Hooking into the compiler

Compile-time metaprogramming

• Groovy 1.6 introduced AST Transformations
• Compile-time == No runtime performance penalty!

Transformation

51

Compile-time metaprogramming

• With metaprogramming, Groovy’s able to modify the
behaviour of programs... at runtime

• Groovy 1.6 introduced AST Transformations
– AST: Abstract Syntax Tree
– Ability to change what’s being compiled at compile-time!
• No runtime impact!
• Lets you change the semantics of your programs!
• Nice way of implementing patterns
and removing boiler-plate technical code
• Better interoperability with Java
– Jave code can call the methods / fields / etc injected in Groovy classes

52

AST Transformations

• Two kinds of transformations

– Global transformations
• applicable to all compilation units

– Local transformations
• applicable to marked program elements
• using specific marker annotations

53

AST Transformations

• Several (local) transformations available
– @ToString
– @Log
– @Delegate
– @Immutable
– and many more

54

@Immutable

• To properly implement immutable classes
– No mutators (state musn’t change)
– Private final fields
– Defensive copying of mutable components
– Proper equals() / hashCode() / toString() for
comparisons, or for keys in maps, etc.

@Immutable class Coordinates {
    Double lat, lng
}

def c1 = new Coordinates(lat: 48.8, lng: 2.5)
def c2 = new Coordinates(48.8, 2.5)
assert c1 == c2

55

Global transformations

• Implement ASTTransformation
• Annotate the transfo specifying a compilation phase

@GroovyASTTransformation(phase = CompilePhase.CONVERSION)
class MyTransformation implements ASTTransformation {
    void visit(ASTNode[] nodes, SourceUnit unit) 
    { ... }
}

• For discovery, create the file META-INF/services/
org.codehaus.groovy.transform.ASTTransformation
• Add the fully qualified name of the class in that file

56

Local transformations

• Same approach as Globale transformations
• But you don’t need the META-INF file
• Instead create an annotation to specify on which element
the transformation should apply

@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass(["com.foo.MyTransformation"])
@interface CoolTransform {...}

57

Example: the Spock framework

• Changing the semantics of the original code
• But keeping a valid Groovy syntax
@Speck
class HelloSpock {
    def "can you figure out what I'm up to?"() {
        expect:
        name.size() == size

        where:
        name     | size
        "Kirk"   |  4
        "Spock"  |  5
        "Scotty" |  6
    }
}

• Check out http://www.spockframework.org
58

Various integration mechanisms

• Java 6’s javax.script.* APIs (aka JSR-223)
• Spring’s language namespace
• Groovy’s own mechanisms

• But a key idea is to externalize those DSL programs
– DSL programs can have their own lifecycle
– no need to redeploy an application because of a rule change
– business people won’t see the technical code

60

Java 6’s javax.script.* API

• Groovy provides an implementation of the javax.script.* API

ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine engine = mgr.getEngineByName("Groovy");
String result = (String)engine.eval("2+3");

61

Spring’s lang namespace

• POGOs (Plain Old Groovy Objects) can be pre-compiled as
any POJO and used interchangeably with POJOs in a
Spring application
• But Groovy scripts & classes can be loaded at runtime
through the <lang:groovy/> namespace and tag
• Reloadable on change
• Customizable through a custom MetaClass

• <lang:groovy id="events"
script-source="classpath:dsl/bizRules.groovy"
customizer-ref="rulesCustomizerMetaClass" />

• More to come in the next version of Spring

62

Groovy’s own mechanisms

• GroovyShell, Eval, GroovyScriptEngine
– for more complex scripts and DSLs

• GroovyClassLoader or CompilationUnit
– the most powerful mechanisms

63

GroovyShell

• A Binding provides a context of execution
– can implement lazy evaluation if needed

• A base script class can be specified
– providing «global» methods and fields

def binding = new Binding()
binding.mass = 22.3
binding.velocity = 10.6

def shell = new GroovyShell(binding)
shell.evaluate("mass * velocity ** 2 / 2")

64

Imports customizer

• Inject imports in your scripts
– so that users don’t have to add imports manually

def configuration = new CompilerConfiguration()
 
def custo = new ImportCustomizer()
custo.addStaticStar(Math.name)
configuration.addCompilationCustomizers(custo) 
 
def result = new GroovyShell(configuration)
    .evaluate(" cos PI/3 ")

65

Applying an AST transformation

• Apply local AST transformations transparently
def configuration = new CompilerConfiguration()

configuration.addCompilationCustomizers(
    new ASTTransformationCustomizer(Log)) 
 
new GroovyShell(configuration).evaluate("""
    class Car {        
        Car() {            
            log.info 'Car constructed'        
        }    
    }     
    log.info 'Constructing a car'    
    def c = new Car()
""")

66

Externalize business rules

• Although Groovy DSLs can be embedded in normal Groovy
classes, you should externalize them

• Store them elsewhere
– in a database, an XML file, etc.

• Benefits
– Business rules are not entangled
in technical application code
– Business rules can have their own lifecycle, without requiring
application redeployments

67

Domain-Specific Language Descriptors

• DSLs are nice, but what about support in my IDE?

• DSLD to the rescue
– Domain-Specific Language Descriptors
– for Eclipse STS (but GDLS concent available in IntelliJ IDEA too)

• Idea: a DSL for describing DSLs, in order to provide
– code-completion
– code navigation
– documentation hovers

68


69


currentType( subType( Number ) ).accept {
    [m: "meter", yd: "yard", cm: "centimeter", mi: "mile", km: "kilometer"].each {
        property name:it.key, type:"Distance",
            doc: """A <code>${it.value}</code> from <a href="$url">$url</a>"""
    }
}

69

A few considerations

Growing a language
Security
Testability

Start small, with key concepts
Beware over-engineering!

Grow your language
progressively

Get your hands dirty
Play with the end-users

Let your DSL ﬂy,
it’s not yours, it’s theirs!

Tight feedback loop
Iterative process

Stay humble,
You can’t get it right the 1st time.

Don’t design alone at your desk
Involve the end users from the start

Example use case

• Biggest European travel services company

• Their customers
– travel agencies, airlines, booking websites...

• Customers want to customize the user experience
– use personalized forms and templates
– configure and tweak the underlying reservation features

• The company runs customers’ Groovy code
on their own platform and mutualised servers!
– How to secure that?

78

Sandboxing approaches

• Groovy supports the usual
Java Security Managers
– avoid System.exit(0)

• Use metaprogramming tricks to prevent calling or
instanciating certain classes
– think overriding constructors to throw exceptions

• Groovy 1.8 to the rescue
– compiler customizers
• SecureASTCustomizer
• ASTTransformCustomizer

79

Secure AST customizer

Idea: Implement an «arithmetic shell»
Being able control what a user script is
allowed to do: only arithmetic expressions

80



• Let’s setup our environment
– some imports
– an import customizer to import java.lang.Math.*
– prepare a secure AST customizer

80



• Let’s setup our environment
– some imports
– an import customizer to import java.lang.Math.*
– prepare a secure AST customizer
import org.codehaus.groovy.control.customizers.*
import org.codehaus.groovy.control.*
import static org.codehaus.groovy.syntax.Types.*
 
def imports = new ImportCustomizer().addStaticStars('java.lang.Math')
def secure = new SecureASTCustomizer()

80


...
secure.with {
// disallow closure creation
closuresAllowed = false 
// disallow method definitions
methodDefinitionAllowed = false 
 
// empty white list => forbid imports
importsWhitelist = [] 
staticImportsWhitelist = []
// only allow the java.lang.Math.* static import
staticStarImportsWhitelist = ['java.lang.Math'
...

81

...
// language tokens allowed
tokensWhitelist = [
PLUS, MINUS, MULTIPLY, DIVIDE, MOD, POWER, PLUS_PLUS, 
MINUS_MINUS, COMPARE_EQUAL, COMPARE_NOT_EQUAL, 
COMPARE_LESS_THAN, COMPARE_LESS_THAN_EQUAL, 
COMPARE_GREATER_THAN, COMPARE_GREATER_THAN_EQUAL
]
 
// types allowed to be used (including primitive types)
constantTypesClassesWhiteList = [
Integer, Float, Long, Double, BigDecimal, 
Integer.TYPE, Long.TYPE, Float.TYPE, Double.TYPE
]
 
// classes who are allowed to be receivers of method calls
receiversClassesWhiteList = [ 
Math, Integer, Float, Double, Long, BigDecimal ]
}
...

82


• Ready to evaluate our arithmetic expressions!
...
def config = new CompilerConfiguration()
config.addCompilationCustomizers(imports, secure)

def shell = new GroovyShell(config)
 
shell.evaluate 'cos(PI/3)'

• But the following would have failed:

shell.evaluate 'System.exit(0)'

83

Test, test, test!

• Don’t just test for nominal cases
– Explicitely test for errors!

• Ensure end-users get meaninful error messages

84

Summary

• Groovy DSLs in the wild manage millions,
and even billions of Euros (one of my customers)
– so I guess we can say Groovy can be trusted :-)

• Groovy 1.8 adds some nice capabilities
for writing nicer plain-English-like business rules

• Tooling is improving greatly with DSL descriptors
– offering nice IDE integration for authoring business rules

86

Thank you!

e
aforg pment
ume L Develo
Guilla Groovy om
He ad of e@ gmail.c
glaforg rge
Email: @glafo
:
Twitter

87

Groovy DSLs - S2GForum London 2011 - Guillaume Laforge

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (6)

Destacado

Destacado (6)

Similar a Groovy DSLs - S2GForum London 2011 - Guillaume Laforge

Similar a Groovy DSLs - S2GForum London 2011 - Guillaume Laforge (20)

Más de Guillaume Laforge

Más de Guillaume Laforge (20)

Último

Último (20)

Groovy DSLs - S2GForum London 2011 - Guillaume Laforge

Notas del editor