Groovy's AST Transformation is a compile time meta-programming technique and allows developer to hook into compilation process and add new fields or methods, or modify existing methods.

1. Meta-Programming with AST
Transformations in Groovy
Gagan Agrawal
Xebia

2. Agenda
Meta-Programming with AST Transformation
What ?
How ?
Why ?

3. Metaprogramming

4. What is Metaprogramming
Is the writing of computer programs that write or
manipulate other programs
Minimizes the number of lines of code to express
a solution
Reduces development time
Could be Run-time or Compile-time
Groovy
Run-Time : MetaClass
Compile-Time : AST Transformations

5. Concrete Syntax Tree

6. Concrete Syntax Tree
Is a representation of grammar in a tree-like form.
Is a one-to-one mapping from the grammar to a
tree-form.

7. Concrete Syntax Tree
Example in C
return a+2;

8. Abstract Syntax Tree

9. Abstract Syntax Tree
Is a tree representation of the abstract syntactic
structure of source code
Each node of the tree denotes a construct in
source code
'Abstract' – Does not represent every detail that
appears in the real syntax
E.g.
Parentheses are implicit
Semicolon ';' is discarded

10. Abstract Syntax Tree
return a+2;

11. AST Transformation

12. AST Transformations
Compile-time metaprogramming
Allows developers to hook into the compilation
process
To modify Abstract Syntax Tree before bytecode
generation
Performance is better than Run-time
metaprogramming

13. AST Transformation
Compiler Phases
Initialization : Source files are opened and
environment configured
Parsing : The grammar is used to produce tree of
tokens representing the source code
Conversion : An Abstract Syntax Tree(AST) is created
from token trees
Semantic Analysis : Performs consistency and validity
checks that the grammar can't check for and resolves
classes

14. AST Transformation
Compiler Phases :
Canonicalization: Complete the AST
Instruction Selection : instruction set is choosen, e.g.
Java5 or pre Java5
Class Generation : creates binary output in memory
Output : Write the binary output to the file system
Finalization : Perform any last cleanup

15. Expressions, Statements and Blocks

16. Expressions
An expression is a construct made up of variables,
operators and method invocations
Evaluates to a single value
E.g. 2+3 evaluates to 5
A variable is an expression because it denotes a
value in memory
int var = 0;

17. Expression
Compound Expression
1*2*3
x+y/100
(x+y)/100
Other Types
Cast Expression - (int c = (int)d)
Constant Expression – (null, true, false)
Field Expression – (this.foo)
Method Call Expression – (object.method())
Boolean Expression - (value1==value2)

18. Statements
Roughly equivalent to sentences in natural
languages.
Forms a complete unit of execution.
Types
Expression Statements
Declaration Statements
Control Flow Statements

19. Expression Statements
Created by terminating expression with
semicolon (;)
Assignment Statement
aValue = 8933.234;
Increment Statement
aValue++;
Method Invocation Statement
System.out.println("Hello World!");
Object Creation Statement
Bicycle myBike = new Bicycle();

20. Declaration and Control flow Statements
Declaration Statements
double aValue = 8933.234;
Control flow Statements
If-then and if-then-else Statements
Switch Statements
While and do-while Statements
For Statements

21. Block Statements
A block is a group of zero or more statements
between balanced braces
Can be used anywhere a single statement is
allowed.

22. Block Statements
class BlockDemo {
public static void main(String[] args) {
boolean condition = true;
if (condition) { // begin block 1
System.out.println("Condition is true.");
} // end block one
else { // begin block 2
System.out.println("Condition is false.");
} // end block 2
}
}

23. AST Building Blocks

24. AST Building Blocks
AST can be transformed in one of the two ways
Core AST APIs
ASTBuilder

25. Core AST APIs

26. Core AST APIs
A S T No d e
A n n o ta te d No d e G e n e r ic s T y p e M o d u le N o d e Sta te m e nt
E x p r e s s io n

27. Core AST APIs
ASTNode
Base class for any AST node.
This class supports basic information used in all nodes of
the AST
Line and column number information.
Meta data of a node
A phase operation or transform can use this to transport arbitrary
information to another phase operation or transform provided
transform runs after the part storing the information.

28. Core AST APIs - Expression
Expression
BooleanExpression
CastExpression
ConstantExpression
ConstructorCallExpression
FieldExpression
MethodCallExpression
StaticMethodCallExpression
VariableExpression

29. Core AST APIs - Expression
this.println(message) can be represented as-
new MethodCallExpression(
new VariableExpression("this"),
new ConstantExpression("println"),
new ArgumentListExpression(
new ConstantExpression(message)
)
)

30. Core AST API - Statement
Statement
BlockStatement
SwitchStatement
CaseStatement
BreakStatement
ExpressionStatement
IfStatement
TryCatchStatement
WhileStatement
ReturnStatement

31. Core AST API - Statement
this.println(message) can be represented as-
return new ExpressionStatement(
new MethodCallExpression(
new VariableExpression("this"),
new ConstantExpression("println"),
new ArgumentListExpression(
new ConstantExpression(message)
)
)
)

32. AST Builder

33. AST Builder
Three ways to build AST via ASTBuilder
Build From Strings
Build From Code
Build From a DSL-like Specification

34. Building AST From String
def builder = new AstBuilder()
List<ASTNode> nodes =
builder.buildFromString(
CompilePhase.SEMANTIC_ANALYSIS,
true,
"""
this.println('Hello World')
"""
)

35. Building AST From String
Phase parameter tells AST from which phase to
return AST.
"statementsOnly" boolean parameter tells the
builder to discard the generated top level Script
ClassNode.
String paramter is the input
The builder returns List<ASTNode>

36. Building AST From String
Produces following AST
BlockStatement
-> ExpressionStatement
-> MethodCallExpression
-> VariableExpression
-> ConstantExpression
-> ArgumentListExpression
-> ConstantExpression

37. Building AST From String
Advantages
Does not require author to understand ASTNode subtypes
Allows author to target a CompilePhase
Communicates source code being generated
Robust - Should need no changes even if AST is updated in
release

38. Building AST From String
Disadvantages
IDE can not check syntax or grammar
IDE can not refactor across String
Some entities cannot be created, like the AST for a field
declaration

39. Building AST From Code
def builder = new AstBuilder()
List<ASTNode> nodes = builder.buildFromCode(
CompilePhase.SEMANTIC_ANALYSIS,
true){
println "Hello World"
}

40. Building AST From Code
Produces following AST
BlockStatement
-> ExpressionStatement
-> MethodCallExpression
-> VariableExpression
-> ConstantExpression
-> ArgumentListExpression
-> ConstantExpression

41. Building AST From Code
Advantages
Clearly communicates source being generated
Does not require author to understand ASTNode subtypes
Allows author to target a CompilePhase
Robust - Should need no changes even if AST is updated in
a release
IDE supports syntax checking and refactoring in Closure

42. Building AST From Code
Disadvantages
Some entities cannot be created, like the AST for a field
declaration

43. Building AST From DSL-like Specification
List<ASTNode> nodes = new AstBuilder().buildFromSpec{
block{
expression {
methodCall {
variable 'this'
constant 'println'
argumentList {
if(locale == 'US') constant 'Hello'
if(locale == 'FR') constant 'Bonjour'
else constant "Ni hao"
}
}
}
}
}

44. Building AST From DSL-like Specification
Advantages
Allows conditionals (or any Groovy code) to be executed
during the AST building process.
Allows any ASTNode subtype to be created
Fully documented with lengthy examples in TestCase

45. Building AST From DSL-like Specification
Disadvantages
It can be difficult to determine what AST you need to
write
Verbose - does not always communicate the source being
created
Fragile - AST may need to change between major releases
IDE does not yet provide code tips

46. AST Transformation
Two types of AST Transformations
Local Transformations
Global Transformations

47. Local AST Transformations
Can be applied on a per class basis.
Can only be applied at semantic analysis or later
phases
Annotation Driven
@Retention - Retention Policy should be SOURCE
@Target – Can be Method, Field, Constructor ..//need to
verify
@GroovyASTTransformationClass

48. Local AST Transformations
Logging Example
@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass("LoggingASTTrans
formation")
@interface WithLogging {
}

49. Local AST Transformations
@GroovyASTTransformation(phase=CompilePhase.
SEMANTIC_ANALYSIS)
class LoggingASTTransformation implements
ASTTransformation{
@Override
public void visit(ASTNode[] nodes,
SourceUnit
source) {
.....
}
}

50. Local AST Transformations
Usage
class TestAst {
@WithLogging
void testMethod(){
println "This is test Method"
}
}

51. Global AST Transformations
Global transformations are applied to by the
compiler on the code being compiled, wherever
the transformation apply.
A JAR should be added to the classpath of the
compiler
It should contain a service locator file at META-
INF/services/org.codehaus.groovy.transform.ASTTr
ansformation
File should have a line with the name of the
transformation class

52. Global AST Transformations
The transformation class must have a no-args
constructor and implement the
org.codehaus.groovy.transform.ASTTransformatio
n interface
It will be run against every source in the
compilation

53. Global AST Transformation
Example

54. Global AST Transformation
Summary
Write an ASTTransformation subclass
Create a Jar metadata file containing the name of your
ASTTransformation
Create a Jar containing the class and metadata
Invoke groovyc with that Jar on your classpath.

55. Built-in Annotations

56. @Delegate
With @Delegate annotation, a class field becomes
an object to which method calls are delegated.
Example
class Event{
@Delegate Date when
String title, url
}

57. @Delegate
Groovy compiler adds all of Date's methods to the
Event class.
Behind the scene methods simply delegate the
call to Date field.

58. @Singleton
class T {
private static volatile T instance
private T() {}
static T getInstance () {
if (instance) {
instance
} else {
synchronized(T) {
if (instance) {
instance
} else {
instance = new T ()
}
}
}
}
}

59. @Singleton
@Singleton class T {
//properties...
//methods..
}

60. @Immutable
No mutators (methods that modify internal state)
Class must be final
Fields must be private and final
equals, hashCode and toString must be
implemented in terms of the fields if you want to
compare your objects or use them as keys in e.g.
maps

61. @Immutable
Java code requires lot of boiler plate code.
Groovy requires only annotation @Immuatable on
class
@Immutable final class Person {
String first
String last
}

62. @PackageScope
class PackageScopeExample {
@PackageScope
String name
}

63. @Lazy
class LazyExample {
@Lazy pets = ['Cat', 'Dog', 'Bird']
static main(args){
def example = new LazyExample()
println "Pets intialized :
"+example.dump().contains('Cat')
println "List size : "+example.pets.size()
println "Pets intialized :
"+example.dump().contains('Cat')
}
}

64. @Newify
Ruby Approach
@Newify([Integer])
void test(){
println Integer.new(23)
}

65. @Newify
Python Approach
class Tree {
def elements
Tree(Object... elements) { this.elements =
elements as List }
}
class Leaf {
def value
Leaf(value) { this.value = value }
}

66. @Newify
def buildTree() {
new Tree(new Tree(new Leaf(1), new
Leaf(2)), new Leaf(3))
}
@Newify([Tree, Leaf])def buildTree() {
Tree(Tree(Leaf(1), Leaf(2)), Leaf(3))
}

67. @Category
interface Vehicle{
String getName()
}
@Category(Vehicle)
class FlyingAbility{
def fly(){
"I am ${name} and I fly"
}
}

68. @Mixin
@Category(Vehicle)
class DrivingAbility{
def drive(){
"I am ${name} and I drive"
}
}
@Mixin([DrivingAbility, FlyingAbility])
class JamesBondVehicle implements Vehicle{
public String getName(){
"James Bond Vehicle"
}
}

69. Where AST Transformations are used?
Grails 2.0
GormTransformer
ControllerActionTransformer
ControllerTransformer
GormToJpaTransformer
Spock framework
Grails Plugins
Audit-Trail Plugin
Rich Domain Classes Plugin
Redis Plugin
Neo4j Plugin

70. Thank You