Barcelona JUG (Java Users Group) slides about walkmod, an open source tool to apply coding conventions. 6th March 2014.

1. walkmod
a tool to apply coding conventions
@walkmod
http://walkmod.com

2. hello!
@raquelpau
raquelpau@gmail.com
@acoroleu
acoroleu@gmail.com

3. introduction
how it works
transformations
query engine
merge engine
plugins
roadmap

4. Introduction
motivation
•
Don’t repeat yourself (DRY) principle. Multiple layer architectures
contains lots of wrappers, which might be generated. E.g. Tests, DAOs,
Facades..
•
The application domain relevance. The business rules / domain logic
can’t be generated (really?). We need to focus on this!
•
Code review of ALL contributors. Who fixes the code?

5. Introduction

6. Introduction
What are coding conventions?
“coding conventions are a set of guidelines for a specific programming
language that recommend programming style, practices and methods for
each aspect of a piece program written in this language.” wikipedia

7. Introduction
examples
•
•
•
•
Naming conventions
Code formatting
Code licenses
Code documentation
•
•
•
•
Code refactoring
Apply programming practices
Architectural best practices
More..

8. Introduction
examples

9. Introduction
walkmod
•
automatizes the practice of code conventions in development teams
(i.e license usage).
•
automatizes the resolution of problems detected by code quality tools
(i.e PMD, , sonar, findbugs).
•
automatizes the development and repetitive tasks i.e: creating basic
CRUD services for the entire model.
•
automatizes global code changes: i.e refactorings.

10. Introduction
walkmod is open!
•
•
•
•
•
open source: feel free to suggest changes!
free for everybody: tell it to your boss!
extensible by plugins do it yourself and share them! DIY+S
editor/IDE independent
community support

11. introduction
how it works
transformations
query engine
merge engine
plugins
roadmap

12. how it works
• All conventions are applied
with blocks of
transformations for each
source file.
• Transformations may
update the same sources or
creating/updating another
ones.
workflow

13. how it works
overview
•
reader: reads the sources. i.e retrieves all files from a directory
recursively.
•
•
walker: executes a chain of transformations for each source.
•
writer: writes the sources. i.e using the eclipse formatter.
transformation: updates or creates sources for the following
transformation. Transformations are connected like a pipe through a
shared context.

14. how it works
reader
•
•
•
•
Reads the sources. By default, reads the folder src/main/java.
Works with multiple include/exclude rules.
Creates a resource object, whose content is iterated from the walker.
Works for sources of any programming language. The resource object
could be a source folder, a parsed json file, etc..

15. how it works
walker
•
Executes a chain of transformations for each object allocated in a
resource. i.e all java source files of an specific folder.
•
merges the output produced by transformations with existent
resources.
•
•
invokes the writer with the final (and merged) output.
analyzes and reports which changes have been produced by the chain
of transformations in each object.

16. how it works
transformations
•
•
modifies or creates objects that will be written.
There are three ways to design a transformation. Using:
templates,
scripts,
or visitors.

17. how it works
writer
•
writes each object allocated in a resource. i.e all java source files of a
specific folder.
•
•
Has include/exclude rules.
There are useful writer implementations, such as the storage of the
contents of a toString() object method or the eclipse formatter.

18. how it works
remember

19. introduction
how it works
transformations
query engine
merge engine
plugins
roadmap

20. transformations
why templates?
•
•
•
•
Templates are used to avoid manipulating the AST directly.
Generates dynamic content querying the AST.
DRY compliance.
groovy is the default template engine, but can be customized.

21. transformations
template configuration
<!DOCTYPE walkmod PUBLIC "-//WALKMOD//DTD" "http://www.walkmod.com/dtd/walkmod-1.0.dtd">
<walkmod>
...
<transformation type="walkmod:commons:template" >
<param name="templates">
["src/main/walkmod/templates/jpa-entities.groovy"]
</param>
</transformation>
...
</walkmod>

22. transformations
why scripts?
•
•
•
Scripts allow the design of inline transformations.
Scripts should be used to apply simple modifications in source files.
Support for multiple languages. Those which implement the standard
Java scripting interface. i.e. groovy, javascript, python..

23. transformations
script configuration
...
<transformation type="walkmod:commons:scripting" >
<param name="language">
groovy
</param>
<param name="location">
src/main/walkmod/scripts/fields.groovy
</param>
</transformation>
...

24. transformations
why visitors?
•
•
Visitors are developed and compiled in java.
•
Visitors should be used to apply complex modifications in source files.
To include transformations as plugins to be shared inside the
community.

25. transformations
visitor transformations
public class HelloVisitor extends VoidVisitor<VisitorContext>{
...
@Overwrite
public void visit(MethodDeclaration md, VisitorContext ctx){
//TODO
}
@Overwrite
public void visit(FieldDeclaration fd, VisitorContext ctx){
//TODO
}
...
}

26. transformations
visitor configuration
<!DOCTYPE walkmod PUBLIC "-//WALKMOD//DTD" "http://www.walkmod.com/dtd/walkmod-1.0.dtd">
<walkmod>
<plugins>
<plugin groupId="mygroupId" artifactId="myartifactId" version="versionNumber">
</plugin>
</plugins>
...
<transformation type="mygroupId:myartifactId:my-visitor" >
<param name="param1">
value1
</param>
<param name="paramN">
valueN
</param>
</transformation>
...
</walkmod>

27. introduction
how it works
transformations
query engine
merge engine
plugins
roadmap

28. query engine
query engine
•
•
Write less to do the same!
•
The default query language is gPath (groovy), but you can change it for
your favorite language.
•
Common used large query expressions can be referenced from Alias.
“TypeDeclaration.metaClass.getMethods = { -> delegate.members.findAll({it instanceof MethodDeclaration}); }”
All queries have an object context and a query expression. By default, the
context is the root element of a parsed source file.

29. query engine
MethodDeclaration method = null;
Collection members = type.getMembers();
Iterator it = members.iterator();
while (it.hasNext()){
BodyDeclaration member = (BodyDeclaration)it.next();
if (member instance of MethodDeclararion){
MethodDeclarion aux = (MethodDeclaration) member;
if(“execute”.equals(aux.getName()){
method = aux;
break;
}
}
}
type.methods.find({it.name.equals(“execute”)})

30. query engine
queries from templates
Using the query object: ${query.resolve(“expr”)}.
import org.apache.log4j.Logger;
public class ${query.resolve("type.name")}{
public static Logger log = Logger.getLogger(${query.resolve("type.name")}.class);
}
template to add Loggers

31. query engine
queries from scripts
accessing through a binding called query
..
for( type in node.types) {
def result = query.resolve(type, “methods”);
...
}
...
groovy script querying the type methods

32. query engine
queries from visitors
Implementing QueryEngineAware or extending VisitorSupport.
public class MyVisitor extends VisitorSupport{
@Overwrite
public void visit(TypeDeclaration td, VisitorContext ctx){
Object result = query(
td, //context
“methods.find({it.name.equals(“foo”)})” //expr
);
}
}
visitor code with a gpath query

33. introduction
how it works
transformations
query engine
merge engine
plugins
roadmap

34. merge engine
why a merge engine?
•
To avoid duplicate results by transformations (i.e duplicate a method)
in existing code.
•
Simplify transformations. Otherwise, transformations must check many
conditions to avoid repeating code or overwriting it.
•
Sometimes, developer changes (i.e. adding a new method) must be
respected by the engine.

35. walkmod merges outputs with existential sources

36. merge engine
semantic merge
•
Code is merged according to the meaning of its elements instead of
simply merging text.
•
Only elements with the same identifier are merged. Indentifiable
element types must implement the interface mergeable.

37. merge engine
previous concepts
•
•
local object is the current version of an element.
remote object is the modified version of an element generated by a
single transformation. It may be a fragment to add in a local object.

38. merge engine
merge policies
Configurable and extensible merge policies for each object type.
Object
merge
policy
Finds the equivalent local version of a remote (and thus,
generated) object and merge recursively their children.
These objects must be identifiable to be searched and found.
These policies should be applied for fields, , methods, , types …
Type select which merge action do for local and remote objects which are
merge not identifiable, although being instances of the same class. i.e policies
policy for the statements of an existent method.

39. merge engine
object merge policies
•
append policy only writes new values for null fields. Otherwise, these
are not modified.
•
overwrite policy modifies all field values of a local object for those
generated by a transformation.

40. merge engine
type merge policies
•
assign policy only writes the remote values. i.e replacing the list of
statements of a method for the generated list.
•
unmodify policy only writes the local values. i.e to respect the
developer changes in the statements of an old transformation.
•
addall policy that appends the remote values into the local values.

41. introduction
how it works
transformations
query engine
merge engine
plugins
roadmap

42. plugins
why and how to
extend walkmod?
•
You can extend walkmod with new components or override the
existing ones (visitors, readers, writers, walkers, merge policies…)
•
Creating new plugins (java libraries) and deploying them into a maven
repository (public or private). See repo.maven.apache.org
•
All walkmod extensions need a plugin descriptor of their components
in the META-INF/walkmod directory inside the jar library.

43. plugins
plugin descriptor
•
Plugin descriptor is a XML file with the name walkmod-xxx-plugin.xml
which follows the spring bean configuration.
•
New readers, writers, walkers or transformations must be specified
with a unique name “groupId:artifactId:name” as a spring bean in the
plugin descriptor.
<beans>
...
<bean id="walkmod:commons:import-cleaner" class="org.walkmod.visitors.ImportOrganizer" />
...
</beans>

44. plugins
plugin usage (I)
Plugins are declared into ${project}/walkmod.xml
<walkmod>
<plugins>
<plugins>
<plugin groupId="walkmod" artifactId="walkmod-commons-plugin" version="1.0">
</plugin>
</plugins>
<chain name="my-chain">
...
</chain>
<walkmod>

45. plugins
plugin usage (II)
Beans declared in a plugin descriptor can be referenced from walkmod.xml
using the type attribute.
<walkmod>
<plugins>
<plugin groupId="walkmod" artifactId="walkmod-commons-plugin" version="1.0">
</plugin>
</plugins>
<chain name="my-chain">
...
<transformation type="walkmod:commons:imports-cleaner" />
...
</chain>
</walkmod>

46. plugins
plugins backend
•
Walkmod embeds apache ivy to download plugins from maven
repositories in runtime.
•
Custom repositories are in ${walkmod-home}/conf/ivysettings.xml
•
All plugin jars and their dependencies are files loaded dynamically into
a new classloader.
•
All beans used and declared in plugin descriptors are resolved by the
spring source engine using the new classloader.

47. introduction
how it works
transformations
query engine
merge engine
plugins
roadmap

48. roadmap
next steps
•
modularization: split the project in modules in GitHub and deploy them in a
Maven public repository.
•
•
Java 8 support (lambda expressions)
•
- less configuration: reutilization by inheritance / include rules of the XML
elements.
•
saas: publish online services for running walkmod and all its plugins (also
private).
+ plugins: Create and publish new plugins (e.g. refactoring or support for other
programming languages).