Declarative Syntax Definition - Grammars and Trees
•
7 recomendaciones•3,073 vistas
This lecture lays the theoretic foundations for declarative syntax formalisms and syntax-based language processors, which we will discuss later in the course. We introduce the notions of formal languages, formal grammars, and syntax trees, starting from Chomsky's work on formal grammars as generative devices. We start with a formal model of languages and investigate formal grammars and their derivation relations as finite models of infinite productivity. We further discuss several classes of formal grammars and their corresponding classes of formal languages. In a second step, we introduce the word problem, analyse its decidability and complexity for different classes of formal languages, and discuss consequences of this analysis on language processing. We conclude the lecture with a discussion about parse tree construction, abstract syntax trees, and ambiguities.
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (20)
Similar a Declarative Syntax Definition - Grammars and Trees
Similar a Declarative Syntax Definition - Grammars and Trees (20)
Más de Guido Wachsmuth
Más de Guido Wachsmuth (9)
Último
Último (20)
Declarative Syntax Definition - Grammars and Trees
- 1. IN4303 2015/16 Compiler Construction Language Specification formal grammars Guido Wachsmuth
- 2. Formal Grammars 2 3 * +7 21 3 * +7 21 Exp ExpExp Exp Exp parser
- 3. Formal Grammars 3 syntax definition errors parse generate check parse table generic parser
- 4. Formal Grammars Stratego NaBL TS SDF3 ESV editor SPT tests 4 syntax definition concrete syntax abstract syntax static semantics name binding type system dynamic semantics translation interpretation
- 5. Formal Grammars Stratego NaBL TS SDF3 ESV editor SPT tests 5 syntax definition concrete syntax abstract syntax static semantics name binding type system dynamic semantics translation interpretation
- 6. Formal Grammars Stratego NaBL TS SDF3 ESV editor SPT tests 6 syntax definition concrete syntax abstract syntax static semantics name binding type system dynamic semantics translation interpretation
- 7. Formal Grammars 7 P A R E N T A L ADVISORY THEORETICAL CONTENT
- 8. Formal Grammars 8 formal languages
- 9. Formal Grammars 9 infinite productivity
- 10. Formal Grammars 10 finite models
- 11. Formal Grammars 11 Philosophy Linguistics lexicology grammar morphology syntax phonology semantics Interdisciplinary Computer Science syntax semantics
- 12. Formal Grammars 12 Philosophy Linguistics lexicology grammar morphology syntax phonology semantics Interdisciplinary Computer Science syntax semantics
- 14. Formal Grammars 13 vocabulary Σ finite, nonempty set of elements (words, letters) alphabet
- 15. Formal Grammars 13 vocabulary Σ finite, nonempty set of elements (words, letters) alphabet string over Σ finite sequence of elements chosen from Σ word, sentence, utterance
- 16. Formal Grammars 13 vocabulary Σ finite, nonempty set of elements (words, letters) alphabet string over Σ finite sequence of elements chosen from Σ word, sentence, utterance formal language λ set of strings over a vocabulary Σ λ ⊆ Σ*
- 17. Formal Grammars 14 formal grammars
- 18. Formal Grammars 15 formal grammar G derivation relation G formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w}
- 19. Formal Grammars 16 G = (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology
- 20. Formal Grammars 17 G = (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols
- 21. Formal Grammars 18 G = (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols N: finite set of non-terminal symbols
- 22. Formal Grammars 19 G = (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol
- 23. Formal Grammars 20 G = (N, Σ, P, S) Num → Digit Num Num → Digit Digit → “0” Digit → “1” Digit → “2” Digit → “3” Digit → “4” Digit → “5” Digit → “6” Digit → “7” Digit → “8” Digit → “9” decimal numbers morphology Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol P⊆N×(N∪Σ)* : set of production rules
- 24. Formal Grammars 21 Num Digit Num 4 Num 4 Digit Num 4 3 Num 4 3 Digit Num 4 3 0 Num 4 3 0 Digit 4 3 0 3 Num → Digit Num Digit → “4” Num → Digit Num Digit → “3” Num → Digit Num Digit → “0” Num → Digit Digit → “3” decimal numbers production
- 25. Formal Grammars 21 Num Digit Num 4 Num 4 Digit Num 4 3 Num 4 3 Digit Num 4 3 0 Num 4 3 0 Digit 4 3 0 3 Num → Digit Num Digit → “4” Num → Digit Num Digit → “3” Num → Digit Num Digit → “0” Num → Digit Digit → “3” decimal numbers production leftmost derivation
- 26. Formal Grammars 22 Num Digit Num Digit Digit Num Digit Digit Digit Num Digit Digit Digit Digit Digit Digit Digit 3 Digit Digit 0 3 Digit 3 0 3 4 3 0 3 Num → Digit Num Num → Digit Num Num → Digit Num Num → Digit Digit → “3” Digit → “0” Digit → “3” Digit → “4” decimal numbers production
- 27. Formal Grammars 22 Num Digit Num Digit Digit Num Digit Digit Digit Num Digit Digit Digit Digit Digit Digit Digit 3 Digit Digit 0 3 Digit 3 0 3 4 3 0 3 Num → Digit Num Num → Digit Num Num → Digit Num Num → Digit Digit → “3” Digit → “0” Digit → “3” Digit → “4” decimal numbers production rightmost derivation
- 28. Formal Grammars 23 G = (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax
- 29. Formal Grammars 24 G = (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols
- 30. Formal Grammars 25 G = (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols N: finite set of non-terminal symbols
- 31. Formal Grammars 26 G = (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol
- 32. Formal Grammars 27 G = (N, Σ, P, S) Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” binary expressions syntax Σ: finite set of terminal symbols N: finite set of non-terminal symbols S∈N: start symbol P⊆N×(N∪Σ)* : set of production rules
- 33. Formal Grammars 28 Exp Exp + Exp Exp + Exp * Exp 3 + Exp * Exp 3 + 4 * Exp 3 + 4 * 5 Exp → Exp “+” Exp Exp → Exp “*” Exp Exp → Num Exp → Num Exp → Num binary expressions production
- 34. Formal Grammars 29 formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N
- 35. Formal Grammars 29 formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N nonterminal symbol
- 36. Formal Grammars 29 formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N context
- 37. Formal Grammars 29 formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N replacement
- 38. Formal Grammars 29 formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N
- 39. Formal Grammars 30 type-0, unrestricted: P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* type-1, context-sensitive: (a A c, a b c) type-2, context-free: P ⊆ N × (N∪Σ)* type-3, regular: (A, x) or (A, xB)
- 40. Formal Grammars 31 formal grammars context-sensitive context-free regular
- 41. Formal Grammars 32 formal grammar G derivation relation G formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w}
- 42. Formal Grammars 33 formal grammar G = (N, Σ, P, S) derivation relation G ⊆ (N∪Σ)* × (N∪Σ)* w G w’ ∃(p, q)∈P: ∃u,v∈(N∪Σ)* : w=u p v ∧ w’=u q v formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w}
- 43. Formal Grammars 34 formal languages context-sensitive context-free regular
- 44. Formal Grammars 35 3 * +7 21 3 * +7 21 Exp ExpExp Exp Exp parser Derivation is about productivity. But what about parsing?
- 45. Formal Grammars 36 word problem
- 46. Formal Grammars 37 word problem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else theoretical computer science decidability & complexity
- 47. Formal Grammars 37 word problem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else decidability type-0: semi-decidable type-1, type-2, type-3: decidable theoretical computer science decidability & complexity
- 48. Formal Grammars 37 word problem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else decidability type-0: semi-decidable type-1, type-2, type-3: decidable complexity type-1: PSPACE-complete type-2, type-3: P theoretical computer science decidability & complexity
- 49. Formal Grammars 37 word problem χL: Σ* → {0,1} w → 1, if w∈L w → 0, else decidability type-0: semi-decidable type-1, type-2, type-3: decidable complexity type-1: PSPACE-complete type-2, type-3: P theoretical computer science decidability & complexity PSPACE⊇NP⊇P
- 50. Formal Grammars 38 formal grammars context-sensitive context-free regular theoretical computer science decidability & complexity
- 51. Formal Grammars 38 formal grammars context-sensitive context-free regular theoretical computer science decidability & complexity
- 52. Formal Grammars 38 formal grammars context-sensitive context-free regular theoretical computer science decidability & complexity
- 53. Formal Grammars 39 Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” context-free grammars production vs. reduction rules
- 54. Formal Grammars 39 Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” context-free grammars production vs. reduction rules productive form
- 55. Formal Grammars 39 Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” Num → Exp Exp “+” Exp → Exp Exp “−” Exp → Exp Exp “*” Exp → Exp Exp “/” Exp → Exp “(” Exp “)” → Exp context-free grammars production vs. reduction rules productive form
- 56. Formal Grammars 39 Exp → Num Exp → Exp “+” Exp Exp → Exp “−” Exp Exp → Exp “*” Exp Exp → Exp “/” Exp Exp → “(” Exp “)” Num → Exp Exp “+” Exp → Exp Exp “−” Exp → Exp Exp “*” Exp → Exp Exp “/” Exp → Exp “(” Exp “)” → Exp context-free grammars production vs. reduction rules productive form reductive form
- 57. Formal Grammars 40 3 + 4 + 5 Exp + 4 + 5 Exp + Exp + 5 Exp + Exp * Exp Exp + Exp Exp Num → Exp Num → Exp Num → Exp Exp “*” Exp → Exp Exp “+” Exp → Exp binary expressions reduction
- 58. Formal Grammars 41 3 * +7 21 3 * +7 21 Exp ExpExp Exp Exp parser The word problem is about membership. But what about structure?
- 59. Formal Grammars 42 syntax trees
- 60. Formal Grammars 43 Exp Num Exp +Exp Exp Exp −Exp Exp Exp *Exp Exp Exp /Exp Exp Exp Exp( ) context-free grammars tree construction rules
- 61. Formal Grammars 44 binary expressions tree construction 3 + *4 5
- 62. Formal Grammars 44 binary expressions tree construction Exp 3 + *4 5
- 63. Formal Grammars 44 binary expressions tree construction Exp Exp 3 + *4 5
- 64. Formal Grammars 44 binary expressions tree construction Exp Exp 3 + *4 5 Exp
- 65. Formal Grammars 44 binary expressions tree construction Exp Exp Exp 3 + *4 5 Exp
- 66. Formal Grammars 44 binary expressions tree construction Exp Exp Exp 3 + *4 5 Exp Exp
- 67. Formal Grammars 45 binary expressions ambiguity Exp Exp Exp 3 + *4 5 Exp Exp Exp Exp Exp 3 + *4 5 Exp Exp
- 68. Formal Grammars 46 syntax trees different trees for same sentence derivations different leftmost derivations for same sentence different rightmost derivations for same sentence NOT just different derivations for same sentence context-free grammars ambiguity
- 69. Formal Grammars 47 parse trees parent node: nonterminal symbol child nodes: terminal symbols abstract syntax trees (ASTs) abstract over terminal symbols convey information at parent nodes abstract over injective production rules syntax trees parse trees & abstract syntax trees
- 70. Formal Grammars 48 binary expressions parse tree & abstract syntax tree Exp Exp Exp Exp Exp (3 + *4 5 ) Exp
- 71. Formal Grammars 48 binary expressions parse tree & abstract syntax tree Const Mul Const 3 4 5 Const Add Exp Exp Exp Exp Exp (3 + *4 5 ) Exp
- 72. Formal Grammars 49 Except where otherwise noted, this work is licensed under
- 73. Formal Grammars 50 attribution slide title author license 1 The Pine, Saint Tropez Paul Signac public domain 2, 3, 35, 41 PICOL icons Melih Bilgil CC BY 3.0 9 Writing Caitlin Regan CC BY 2.0 10 Latin Grammar Anthony Nelzin 13, 15, 29-34, 38 Noam Chomsky Fellowsisters CC BY-NC-SA 2.0
Notas del editor
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- expressions: binary expressions, method calls\n\nobject-orientation: classes, single inheritance, methods, no overloading\n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- property of language\n\nsay what no-one said before\n\nholds for software languages as well\n
- scientific challenge\n
- \n
- linguistic theory\n\nNoam Chomsky\n
- linguistic theory\n\nNoam Chomsky\n
- linguistic theory\n\nNoam Chomsky\n
- restrictions on production rules => grammar classes\n
- restrictions on production rules => grammar classes\n
- restrictions on production rules => grammar classes\n
- restrictions on production rules => grammar classes\n
- restrictions on production rules => grammar classes\n
- restrictions on production rules => grammar classes\n
- restrictions on production rules => grammar classes\n
- restrictions on production rules => grammar classes\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- computer science: lexical syntax\n\ncan write this as a regular grammar\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- generative device\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- other derivations possible\n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- X* for zero or more X\n\nempty word\n\nbut no construct for separated repetition\n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- All actual, detailed work on assignments must be individual work. \nYou are encouraged to discuss assignments, programming languages used to solve the assignments, their libraries, and general solution techniques in these languages with each other. \nBut if you do so, then you must acknowledge the people with whom you discussed at the top of your submission. \nYou should not look for assignment solutions elsewhere; but if material is taken from elsewhere, then you must acknowledge it. \n\nYou are not permitted to provide or receive any kind of solutions of assignments. \nThis includes partial, incomplete, or erroneous solutions. \nYou are also not permitted to provide or receive programming help from people other than the teaching assistants or the instructor of the course.\n\n
- any kind of solution: an algorithm, code, intermediate solutions\nplagiarism: copying from others, from the web, mimic another ones solution \n\nAny violation of these rules will be reported as a suspected case of fraud to the Board of Examiners and handled according to the EEMCS Faculty's fraud procedure. \nIf the case is proven, a penalty will be imposed: a minimum of exclusion from the course for the duration of one academic year up to a maximum of a one-year exclusion form all courses at TU Delft. \nFor details on the procedure, see Section 2.1.26 in the faculty's Study Guide for MSc Programmes.\n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- round-up on every lecture\n\nwhat to take with you\n\ncheck yourself, pre- and post-paration\n
- \n
- \n