Binary Studio Academy PRO: ANTLR course by Alexander Vasiltsov

1. ANTLR 4
ANother Tool for Language Recognition
by Alexander Vasiltsov

2. What ANTLR can do
● Can generate parser using formal language
description called grammar
● Grammar describes language in EBNF-like
way
● Automatically generates classes for walking
through syntax tree
● Contains powerful error recovery mechanism
● Can deal with left recursive rules

3. Terence Parr
Where to read
http://www.antlr.org/

4. Successful usages
● Twitter search engine
● Hadoop (Hive & Pig)
● Oracle (SQL Developer IDE, Migration
Tools)
● NetBeans IDE

5. How it works

6. Target Languages
ANTLR 4 Target languages:
● Java
● C#
● Python
ANTLR 3 also supports following languages: C,
C#, Java, JavaScript, ActionScript, Objective-C,
Perl, Python, Ruby and other.

7. Setup for Java
Java 1.6 or newer required
1) Download latest ANTLR4 package (antlr-4.4-
complete.jar) at
http://www.antlr.org/download.html
It’s done!

8. Setup for C#
Java 1.6 or newer required!
1) Add ANTLR reference to the project
PM> Install-Package Antlr4
2) Install ANTLR Language Support extension

9. ANTLRWorks
http://tunnelvisionlabs.com/products/demo/antlrworks

10. Lexing
Lexing (tokenizing) - is the process of grouping
of input chars stream into words (tokens).
Token contains at least 2 data fragments: its
type and matched text

11. Parsing
Parsing - is the process of matching of linear
sequence of tokens with language’s formal
grammar
Parse tree (syntax tree) is a result of parsing

12. Syntax tree
Syntax tree represents the structure of
recognized sentence where each node gives
an abstract name to its children nodes
Nodes represent grammar rules
Leafs represent tokens

13. Parsing process

14. Parser generation by ANTLR4

15. ArrayInitParser.java (.cs) Contains parser class definition according to grammar
named ArrayInit
ArrayInitLexer.java (.cs) Contains lexer class definition respectively
ArrayInit.tokens
ArrayInitLexer.tokens
Internal ANTLR’s files, contain token dictionary with
corresponding identifiers
ArrayInitListener.java (.cs) Listener’s interface - for walking through syntax tree
and its processing
ArrayInitBaseListener.java (.cs) Base listener class with empty methods
ArrayInitVisitor.java (.cs) Visitor’s interface - also for walking through syntax tree
using Visitor design pattern
ArrayInitBaseVisitor.java (.cs) Base visitor class with empty methods

16. Syntax tree structure

17. Walker

18. Listener

19. Visitor

20. “Visitor” design pattern

21. Parser’s generation step-by-step
● Java target language:
> java -jar antlr-4.4-complete.jar <grammar-file-name>
● C# target language: add grammar file to the
project and compile it. Generated classes
will be added to objDebug directory

22. Common grammar structure

23. Typical Grammar