SlideShare una empresa de Scribd logo
1 de 67
Of Rats And
                                Dragons
                               Achieving Parsing Sanity



                                     Sean Cribbs

                                   Web Consultant
                                Ruby and Erlang Hacker

Saturday, September 12, 2009
Quick Review
Saturday, September 12, 2009
context-free
                                grammars


Saturday, September 12, 2009
G = (V, Σ, R, S)



Saturday, September 12, 2009
V→w



Saturday, September 12, 2009
non-deterministic
                        pushdown
                         automata


Saturday, September 12, 2009
stack machine
                           w/ backtracking


Saturday, September 12, 2009
massage it



Saturday, September 12, 2009
ε productions



Saturday, September 12, 2009
produce nothing
                         never produced


Saturday, September 12, 2009
cycles



Saturday, September 12, 2009
ambiguities
                               “dangling else”


Saturday, September 12, 2009
if A then if B then C else D

                if A then if B then C else D

                if A then if B then C else D




Saturday, September 12, 2009
parsing expression
                     grammars


Saturday, September 12, 2009
top-down parsing
                       language (70’s)


Saturday, September 12, 2009
direct
                      representation of
                      parsing functions


Saturday, September 12, 2009
Brian Ford 2002



Saturday, September 12, 2009
focused on
                               recognizing


Saturday, September 12, 2009
computer
                               languages


Saturday, September 12, 2009
V←e



Saturday, September 12, 2009
e1 e2



Saturday, September 12, 2009
e1 / e2



Saturday, September 12, 2009
e+



Saturday, September 12, 2009
e*



Saturday, September 12, 2009
&e



Saturday, September 12, 2009
!e



Saturday, September 12, 2009
e?



Saturday, September 12, 2009
“string”
                                   .


Saturday, September 12, 2009
PEG > regexps



Saturday, September 12, 2009
combined
                               lex+parse


Saturday, September 12, 2009
no ambiguity



Saturday, September 12, 2009
choice is ordered



Saturday, September 12, 2009
dangling else
                                 obviated


Saturday, September 12, 2009
greedy repetition



Saturday, September 12, 2009
unlimited
                                 lookahead
                               with predicates


Saturday, September 12, 2009
no left-recursion!
                               (use *,+)




Saturday, September 12, 2009
Parsing
                               Techniques


Saturday, September 12, 2009
Tabular
                               test every rule


Saturday, September 12, 2009
Recursive-descent
                    call & consume


Saturday, September 12, 2009
Predictive
                               yacc/yecc


Saturday, September 12, 2009
Packrat
                               RD with memo


Saturday, September 12, 2009
sacrifice memory
                           for speed


Saturday, September 12, 2009
supports PEGs and
                   some CFGs


Saturday, September 12, 2009
Treetop
                                Pappy
                               neotoma


Saturday, September 12, 2009
neotoma
                 Behind the CodeTM




Saturday, September 12, 2009
can:has(cukes) ->
                          false.

Saturday, September 12, 2009
Cucumber uses
                                  Treetop


Saturday, September 12, 2009
PEG → leex/yecc
                              FAIL


Saturday, September 12, 2009
parsec → eParSec



Saturday, September 12, 2009
HOF protocol



Saturday, September 12, 2009
% Implements "?" PEG operator
                      optional(P) ->
                        fun(Input, Index) ->
                          case P(Input, Index) of
                            {fail, _} -> {[], Input, Index};
                            {_,_,_} = Success -> Success
                              % {Parsed, RemainingInput, NewIndex}
                          end
                        end.




Saturday, September 12, 2009
% PEG
                optional_space <- space?;


                % Erlang
                optional_space(Input,Index) ->
                  optional(fun space/2)(Input, Index).




Saturday, September 12, 2009
Yay! RD!
                               make it memo


Saturday, September 12, 2009
ets
                               Erlang Term
                                 Storage


Saturday, September 12, 2009
{key, value}




Saturday, September 12, 2009
key = Index



Saturday, September 12, 2009
value = dict



Saturday, September 12, 2009
% Memoization wrapper
     p(Inp, StartIndex, Name, ParseFun, TransformFun) ->
       % Grab the memo table from ets
       Memo = get_memo(StartIndex),
       % See if the current reduction is memoized
       case dict:find(Name, Memo) of
         % If it is, return the result
         {ok, Result} -> Result;
         % If not, attempt to parse
         _ ->
            case ParseFun(Inp, StartIndex) of
              % If it fails, memoize the failure
              {fail,_} = Failure ->
                memoize(StartIndex, dict:store(Name, Failure, Memo)),
                Failure;
              % If it passes, transform and memoize the result.
              {Result, InpRem, NewIndex} ->
                Transformed = TransformFun(Result, StartIndex),
                memoize(StartIndex, dict:store(Name, {Transformed,
     InpRem, NewIndex}, Memo)),
                {Transformed, InpRem, NewIndex}
            end
       end.


Saturday, September 12, 2009
parse_transform



Saturday, September 12, 2009
alternative(Input, Index) ->
                        peg:p(Input, Index, alternative, fun(I,P) ->
                            peg:choose([fun sequence/2, fun primary/2])(I,P)
                             end).

                      rule(alternative) ->
                        peg:choose([fun sequence/2, fun primary/2]);




Saturday, September 12, 2009
rules <- space? declaration_sequence space?;
                declaration_sequence <- head:declaration tail:(space declaration)*;
                declaration <- nonterminal space '<-' space parsing_expression space? ';';
                parsing_expression <- choice / sequence / primary;
                choice <- head:alternative tail:(space '/' space alternative)+;
                alternative <- sequence / primary;
                primary <- prefix atomic / atomic suffix / atomic;
                sequence <- head:labeled_sequence_primary tail:(space labeled_sequence_primary)+;
                labeled_sequence_primary <- label? primary;
                label <- alpha_char alphanumeric_char* ':';
                suffix <- repetition_suffix / optional_suffix;
                optional_suffix <- '?';
                repetition_suffix <- '+' / '*';
                prefix <- '&' / '!';
                atomic <- terminal / nonterminal / parenthesized_expression;
                parenthesized_expression <- '(' space? parsing_expression space? ')';
                nonterminal <- alpha_char alphanumeric_char*;
                terminal <- quoted_string / character_class / anything_symbol;
                quoted_string <- single_quoted_string / double_quoted_string;
                double_quoted_string <- '"' string:(!'"' ("" / '"' / .))* '"';
                single_quoted_string <- "'" string:(!"'" ("" / "'" / .))* "'";
                character_class <- '[' characters:(!']' ('' . / !'' .))+ ']
                anything_symbol <- '.';
                alpha_char <- [a-z_];
                alphanumeric_char <- alpha_char / [0-9];
                space <- (white / comment_to_eol)+;
                comment_to_eol <- '%' (!"n" .)*;
                white <- [ tnr];




Saturday, September 12, 2009
self-hosting



Saturday, September 12, 2009
Future directions



Saturday, September 12, 2009
self-contained
                                   parsers


Saturday, September 12, 2009
inline code in PEG



Saturday, September 12, 2009
Reia
                               retem
                               sedate


Saturday, September 12, 2009
questions?



Saturday, September 12, 2009

Más contenido relacionado

Destacado

French Revolution Comic
French Revolution ComicFrench Revolution Comic
French Revolution Comic
drloewen
 
2015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V22015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V2
Marina Micik
 
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test ActivityTechniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
IJTET Journal
 

Destacado (16)

Ftp
FtpFtp
Ftp
 
Hover Ige
Hover IgeHover Ige
Hover Ige
 
Income statement
Income statementIncome statement
Income statement
 
One direction
One directionOne direction
One direction
 
Utfordringer ved interhospital intensivtransport i Norge
Utfordringer ved interhospital intensivtransport i NorgeUtfordringer ved interhospital intensivtransport i Norge
Utfordringer ved interhospital intensivtransport i Norge
 
French Revolution Comic
French Revolution ComicFrench Revolution Comic
French Revolution Comic
 
Microsoft Learning Experiences Skills and Employability
Microsoft Learning Experiences Skills and Employability Microsoft Learning Experiences Skills and Employability
Microsoft Learning Experiences Skills and Employability
 
Smart Student Tracking System for Schools and Parents from CodeLand
Smart Student Tracking System for Schools and Parents from CodeLandSmart Student Tracking System for Schools and Parents from CodeLand
Smart Student Tracking System for Schools and Parents from CodeLand
 
DSM Based low oversampling using SDR transmitter
DSM Based low oversampling using SDR transmitterDSM Based low oversampling using SDR transmitter
DSM Based low oversampling using SDR transmitter
 
Computadora (tecnologia)
Computadora (tecnologia)Computadora (tecnologia)
Computadora (tecnologia)
 
Practica 20 mayo 2016
Practica 20 mayo 2016Practica 20 mayo 2016
Practica 20 mayo 2016
 
2015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V22015-CV-Marina-Micic-V2
2015-CV-Marina-Micic-V2
 
Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...
Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...
Recognition of Isolated Handwritten Arabic and Urdu Numerals along with their...
 
Colegio El Rosario 2016 -Partido seleccion y maestros
Colegio El Rosario 2016 -Partido seleccion y maestrosColegio El Rosario 2016 -Partido seleccion y maestros
Colegio El Rosario 2016 -Partido seleccion y maestros
 
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test ActivityTechniques for Minimizing Power Consumption in DFT during Scan Test Activity
Techniques for Minimizing Power Consumption in DFT during Scan Test Activity
 
Project management overview
Project management overviewProject management overview
Project management overview
 

Similar a Of Rats And Dragons

DelveUI Slides
DelveUI SlidesDelveUI Slides
DelveUI Slides
jkosoy
 
Spring in-the-cloud
Spring in-the-cloudSpring in-the-cloud
Spring in-the-cloud
Joshua Long
 
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 20092009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
Caue Guerra
 
Kostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-EntwicklungKostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-Entwicklung
xrb
 

Similar a Of Rats And Dragons (20)

DelveUI Slides
DelveUI SlidesDelveUI Slides
DelveUI Slides
 
Code Stinkers Anonymous
Code Stinkers AnonymousCode Stinkers Anonymous
Code Stinkers Anonymous
 
Google in Education (just a peek)
Google in Education (just a peek)Google in Education (just a peek)
Google in Education (just a peek)
 
Building A Framework On Rack
Building A Framework On RackBuilding A Framework On Rack
Building A Framework On Rack
 
Semcomp de São Carlos
Semcomp de São CarlosSemcomp de São Carlos
Semcomp de São Carlos
 
Software livre e padrões abertos no desenvolvimento Web
Software livre e padrões abertos no desenvolvimento WebSoftware livre e padrões abertos no desenvolvimento Web
Software livre e padrões abertos no desenvolvimento Web
 
Its all about collaboration
Its all about collaborationIts all about collaboration
Its all about collaboration
 
Localizing iOS Apps
Localizing iOS AppsLocalizing iOS Apps
Localizing iOS Apps
 
100% JS
100% JS100% JS
100% JS
 
Latinoware Rails 2009
Latinoware Rails 2009Latinoware Rails 2009
Latinoware Rails 2009
 
Expand to or work in China and Germany
Expand to or work in China and GermanyExpand to or work in China and Germany
Expand to or work in China and Germany
 
TRNK Presentation
TRNK PresentationTRNK Presentation
TRNK Presentation
 
Zeno rocha - HTML5 APIs para Mobile
Zeno rocha - HTML5 APIs para MobileZeno rocha - HTML5 APIs para Mobile
Zeno rocha - HTML5 APIs para Mobile
 
Mobile Approaches - LinguÁgil 2012
Mobile Approaches - LinguÁgil 2012Mobile Approaches - LinguÁgil 2012
Mobile Approaches - LinguÁgil 2012
 
Spring in-the-cloud
Spring in-the-cloudSpring in-the-cloud
Spring in-the-cloud
 
06 Data
06 Data06 Data
06 Data
 
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 20092009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
 
The Beauty of Bootstrapping - Doing it Anyway
The Beauty of Bootstrapping - Doing it AnywayThe Beauty of Bootstrapping - Doing it Anyway
The Beauty of Bootstrapping - Doing it Anyway
 
Kostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-EntwicklungKostentreiber bei der iOS-Entwicklung
Kostentreiber bei der iOS-Entwicklung
 
Kostentreiber bei der iOS Entwicklung
Kostentreiber bei der iOS EntwicklungKostentreiber bei der iOS Entwicklung
Kostentreiber bei der iOS Entwicklung
 

Más de Sean Cribbs

Riak (Øredev nosql day)
Riak (Øredev nosql day)Riak (Øredev nosql day)
Riak (Øredev nosql day)
Sean Cribbs
 
Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)
Sean Cribbs
 
Content Management That Won't Rot Your Brain
Content Management That Won't Rot Your BrainContent Management That Won't Rot Your Brain
Content Management That Won't Rot Your Brain
Sean Cribbs
 

Más de Sean Cribbs (19)

Eventually Consistent Data Structures (from strangeloop12)
Eventually Consistent Data Structures (from strangeloop12)Eventually Consistent Data Structures (from strangeloop12)
Eventually Consistent Data Structures (from strangeloop12)
 
Eventually-Consistent Data Structures
Eventually-Consistent Data StructuresEventually-Consistent Data Structures
Eventually-Consistent Data Structures
 
A Case of Accidental Concurrency
A Case of Accidental ConcurrencyA Case of Accidental Concurrency
A Case of Accidental Concurrency
 
Embrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleEmbrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with Ripple
 
Riak with node.js
Riak with node.jsRiak with node.js
Riak with node.js
 
Schema Design for Riak (Take 2)
Schema Design for Riak (Take 2)Schema Design for Riak (Take 2)
Schema Design for Riak (Take 2)
 
Riak (Øredev nosql day)
Riak (Øredev nosql day)Riak (Øredev nosql day)
Riak (Øredev nosql day)
 
Riak Tutorial (Øredev)
Riak Tutorial (Øredev)Riak Tutorial (Øredev)
Riak Tutorial (Øredev)
 
The Radiant Ethic
The Radiant EthicThe Radiant Ethic
The Radiant Ethic
 
Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)Introduction to Riak and Ripple (KC.rb)
Introduction to Riak and Ripple (KC.rb)
 
Riak with Rails
Riak with RailsRiak with Rails
Riak with Rails
 
Schema Design for Riak
Schema Design for RiakSchema Design for Riak
Schema Design for Riak
 
Introduction to Riak - Red Dirt Ruby Conf Training
Introduction to Riak - Red Dirt Ruby Conf TrainingIntroduction to Riak - Red Dirt Ruby Conf Training
Introduction to Riak - Red Dirt Ruby Conf Training
 
Introducing Riak and Ripple
Introducing Riak and RippleIntroducing Riak and Ripple
Introducing Riak and Ripple
 
Round PEG, Round Hole - Parsing Functionally
Round PEG, Round Hole - Parsing FunctionallyRound PEG, Round Hole - Parsing Functionally
Round PEG, Round Hole - Parsing Functionally
 
Story Driven Development With Cucumber
Story Driven Development With CucumberStory Driven Development With Cucumber
Story Driven Development With Cucumber
 
Achieving Parsing Sanity In Erlang
Achieving Parsing Sanity In ErlangAchieving Parsing Sanity In Erlang
Achieving Parsing Sanity In Erlang
 
Erlang/OTP for Rubyists
Erlang/OTP for RubyistsErlang/OTP for Rubyists
Erlang/OTP for Rubyists
 
Content Management That Won't Rot Your Brain
Content Management That Won't Rot Your BrainContent Management That Won't Rot Your Brain
Content Management That Won't Rot Your Brain
 

Último

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 

Of Rats And Dragons