Practical Research 1 Lesson 9 Scope and delimitation.pptx
Compiler Components and their Generators - Traditional Parsing Algorithms
1. Compiler Components & Generators
Traditional Parsing Algorithms
Guido Wachsmuth
Delft
Course IN4303
University of
Technology Compiler Construction
Challenge the future
2. Recap: Lexical Analysis
lessons learned
What are the formalisms to describe regular languages?
• regular grammars
• regular expressions
• finite state automata
Why are these formalisms equivalent?
• constructive proofs
How can we generate compiler tools from that?
• implement DFAs
• generate transition tables
Traditional Parsing Algorithms 2
9. The exam
topics
• introduction
• declarative language definition (theory)
• compiling imperative & object-oriented languages
• compiler components & generators
style
• theory in praxis
• understanding over facts
• open questions
• closed book
Traditional Parsing Algorithms 5
14. MiniJava Compiler
liveness analysis
.method public static main([Ljava/lang/String;)V
.limit locals 3
.limit stack 4
.var 1 is x I from startX to endX
.var 2 is y Z from startY to endY
...
.end method
Traditional Parsing Algorithms 10
15. MiniJava Compiler
submission
procedure
ensure compiler works
add 5 positive test cases
include your Net ID in project name
export project as zip file
ensure editor works when imported
submit zip file via Blackboard
submission due Jan 9, 23:59
delay costs you 25 points
plus 1 point per hour or part thereof
Traditional Parsing Algorithms 11
18. Recap: A Theory of Language
formal languages
Traditional Parsing Algorithms 14
19. Recap: A Theory of Language
formal languages
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
Traditional Parsing Algorithms 14
20. Recap: A Theory of Language
formal languages
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
string over Σ
finite sequence of elements chosen from Σ
word, sentence, utterance
Traditional Parsing Algorithms 14
21. Recap: A Theory of Language
formal languages
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
string over Σ
finite sequence of elements chosen from Σ
word, sentence, utterance
formal language λ
set of strings over a vocabulary Σ
λ ⊆ Σ*
Traditional Parsing Algorithms 14
22. Recap: A Theory of Language
formal grammars
Traditional Parsing Algorithms 15
23. Recap: A Theory of Language
formal grammars
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
Traditional Parsing Algorithms 15
24. Recap: A Theory of Language
formal grammars
formal grammar G = (N, Σ, P, S)
nonterminal symbols N nonterminal symbol
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
Traditional Parsing Algorithms 15
25. Recap: A Theory of Language
formal grammars
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
context
Traditional Parsing Algorithms 15
26. Recap: A Theory of Language
formal grammars
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
replacement
Traditional Parsing Algorithms 15
27. Recap: A Theory of Language
formal grammars
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)*
start symbol S∈N
grammar classes
type-0, unrestricted
type-1, context-sensitive: (a A c, a b c)
type-2, context-free: P ⊆ N × (N∪Σ)*
type-3, regular: (A, x) or (A, xB)
Traditional Parsing Algorithms 15
28. Recap: A Theory of Language
formal languages
Traditional Parsing Algorithms 16
29. Recap: A Theory of Language
formal languages
formal grammar G = (N, Σ, P, S)
Traditional Parsing Algorithms 16
30. Recap: A Theory of Language
formal languages
formal grammar G = (N, Σ, P, S)
derivation relation G ⊆ (N∪Σ)* × (N∪Σ)*
w G w’
∃(p, q)∈P: ∃u,v∈(N∪Σ)*:
w=u p v ∧ w’=u q v
Traditional Parsing Algorithms 16
31. Recap: A Theory of Language
formal languages
formal grammar G = (N, Σ, P, S)
derivation relation G ⊆ (N∪Σ)* × (N∪Σ)*
w G w’
∃(p, q)∈P: ∃u,v∈(N∪Σ)*:
w=u p v ∧ w’=u q v
formal language L(G) ⊆ Σ*
L(G) = {w∈Σ* | S G
*
w}
Traditional Parsing Algorithms 16
32. Recap: A Theory of Language
formal languages
formal grammar G = (N, Σ, P, S)
derivation relation G ⊆ (N∪Σ)* × (N∪Σ)*
w G w’
∃(p, q)∈P: ∃u,v∈(N∪Σ)*:
w=u p v ∧ w’=u q v
formal language L(G) ⊆ Σ*
L(G) = {w∈Σ* | S G
*
w}
classes of formal languages
Traditional Parsing Algorithms 16
45. Predictive parsing
encoding precedence
Exp → Num
Exp → “(” Exp “)”
Exp → Exp “*” Exp
Exp → Exp “+” Exp
Fact → Num
Fact → “(” Exp “)”
Term → Term “*” Fact
Term → Fact
Exp → Exp “+” Term
Exp → Term
Traditional Parsing Algorithms 27
46. Predictive parsing
eliminating left recursion
Term → Term “*” Fact
Term → Fact
Exp → Exp “+” Term
Exp → Term
Term’ → “*” Fact Term’
Term’ →
Term → Fact Term’
Exp’ → “+” Term Exp’
Exp’ →
Exp → Term Exp’
Traditional Parsing Algorithms 28
54. LR parsing
idea
problems with LL parsing
• predicting right rule
• left recursion
LR parsing
• see whole left-hand side of a rule
• look ahead
• shift or reduce
Traditional Parsing Algorithms 33
58. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
Traditional Parsing Algorithms 34
59. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
Traditional Parsing Algorithms 34
60. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
Traditional Parsing Algorithms 34
61. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
Traditional Parsing Algorithms 34
62. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
Traditional Parsing Algorithms 34
63. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
Traditional Parsing Algorithms 34
64. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
Traditional Parsing Algorithms 34
65. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
Traditional Parsing Algorithms 34
66. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
Traditional Parsing Algorithms 34
67. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
Traditional Parsing Algorithms 34
68. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
Traditional Parsing Algorithms 34
69. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
Traditional Parsing Algorithms 34
70. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
$ E $
Traditional Parsing Algorithms 34
71. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
$ E $
Traditional Parsing Algorithms 34
72. LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
$ E $
$ S
Traditional Parsing Algorithms 34
73. LR parsing
parse table
rows T1 ... N1 ...
• states of a DFA 1 s3
2 g5
columns
3 r1
• topmost stack symbol
4 r2
• Σ, N
5
entries
6 g1
• reduce, goto state 7 s1
• shift, goto state
8
• goto state
...
Traditional Parsing Algorithms 35
82. Summary
lessons learned
How can we parse context-free languages effectively?
• predictive parsing algorithms
• LR parsing algorithms
Traditional Parsing Algorithms 38
83. Summary
lessons learned
How can we parse context-free languages effectively?
• predictive parsing algorithms
• LR parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
• LR(k) grammars, LR(k) languages
Traditional Parsing Algorithms 38
84. Summary
lessons learned
How can we parse context-free languages effectively?
• predictive parsing algorithms
• LR parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
• LR(k) grammars, LR(k) languages
How can we generate compiler tools from that?
• implement automata
• generate parse tables
Traditional Parsing Algorithms 38
85. Summary
lessons learned
How can we parse context-free languages effectively?
• predictive parsing algorithms
• LR parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
• LR(k) grammars, LR(k) languages
How can we generate compiler tools from that?
• implement automata
• generate parse tables
Traditional Parsing Algorithms 38
87. Literature
learn more
formal languages
Noam Chomsky: Three models for the description of language. 1956
J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata
Theory, Languages, and Computation. 2006
Traditional Parsing Algorithms 39
88. Literature
learn more
formal languages
Noam Chomsky: Three models for the description of language. 1956
J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata
Theory, Languages, and Computation. 2006
syntactical analysis
Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in
Java, 2nd edition. 2002
Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers:
Principles, Techniques, and Tools, 2nd edition. 2006
Traditional Parsing Algorithms 39
89. Outlook
coming next
lectures
• Lecture 13: SDF inside
• Lecture 14: Static analysis
• Lecture 15: Beyond grammarware
Question & Answer Jan 5
• 10 questions, submit & vote
Lab Dec 3
• explore Java Bytecode
• explore Jasmin
• start generator
Traditional Parsing Algorithms 40
92. Pictures
copyrights
Slide 1: Book Scanner by Ben Woosley, some rights reserved
Slide 5: Report card by Carosaurus, some rights reserved
Slides 7, 12: Students, TU Delft, Media Solutions, all rights reserved
Slides 8, 10: Italian Java book cover by unknown artist, some rights reserved
Slide 11: Envelopes by benchilada, some rights reserved
Slides 14-16: Noam Chomsky by Fellowsisters, some rights reserved
Slide 17, 18, 27-29: Tiger by Bernard Landgraf, some rights reserved
Slide 31: West Cornwall Pasty Co. by Dominica Williamson, some rights reserved
Slide 40: Ostsee by Mario Thiel, some rights reserved
Slide 41: Questions by Oberazzi, some rights reserved
Slide 42: Too Much Credit by Andres Rueda, some rights reserved
Traditional Parsing Algorithms 43
Notas del editor
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
linguistic theory\n\nNoam Chomsky\n
linguistic theory\n\nNoam Chomsky\n
linguistic theory\n\nNoam Chomsky\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
\n
\n
\n
\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
\n
\n
\n
\n
\n
\n
\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
restrictions on production rules => grammar classes\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n