SlideShare una empresa de Scribd logo
1 de 33
Descargar para leer sin conexión
Parsing
                           for Fun and Profit
                            (but mainly fun)


                                                  Ash Moran
                                   ash.moran@patchspace.co.uk
                                               PatchSpace Ltd
Saturday, 23 February 13
What?


Saturday, 23 February 13
Parsing


                           Adding structure and meaning to text




Saturday, 23 February 13
Parsing Human
                                  Languages
                           Jake stretched his legs
                           “Jake”, “stretched”, “his”, “legs”
                           “Jake”<noun>, “stretched”<verb, past>,
                           “his”<possessive pronoun>, “legs”<noun>
                           “Jake” <noun, subject>, “stretched”, (“his”,
                           “legs”)<noun phrase, object>


Saturday, 23 February 13
Parsing Computer
                                Languages
                           “foo = bar + 123”
                           “foo”, “=”, “bar”, “+”, “123”
                           “foo”<var>, “=”<assignment_op>,
                           “bar”<var>, “+”<op_plus>,
                           “123”<int_literal>



Saturday, 23 February 13
Why?


Saturday, 23 February 13
Not just
           compiling!
            Compilers breathe fire.




Saturday, 23 February 13
Pretty Printing
Saturday, 23 February 13
Pretty
                Printing
                           gofmt

                 http://gofmt.com/




Saturday, 23 February 13
Code Smell Detectors
                           https://rubygems.org/gems/reek

Saturday, 23 February 13
Code Smell Detectors
Saturday, 23 February 13
Other ideas
                           Code metrics
                           Bug detectors
                           Domain-specific languages
                           Language translators (e.g. Ruby -> PHP)
                           Code obfuscators
                           Alternative syntaxes (e.g. CoffeeScript)
                           Refactoring tools
Saturday, 23 February 13
How?


Saturday, 23 February 13
Step 1
          3 year computer science
                   degree




Saturday, 23 February 13
Lexing/Tokenising


                   if x > 100 then return “big” else return “small”
                   if x > 100 then return “big” else return “small”




Saturday, 23 February 13
Tree Building
                   if x > 100 then return “big” else return a + b
                                           if

                               >         then     else

                           x       100   return     return

                                         “big”           +
                                                   a         b


Saturday, 23 February 13
Parsing Expression
                                Grammars

                           Like regular expressions, but can handle
                           recursion, e.g. HTML
                           Not actually that much harder to use




Saturday, 23 February 13
Regexes and HTML




Saturday, 23 February 13
Treetop PEG grammar
Saturday, 23 February 13
Doing Sums


Saturday, 23 February 13
Switch to
                           Sublime Text, idiot
                               Code is now available:
         https://github.com/patchspace/parsing_for_fun_and_profit/




Saturday, 23 February 13
A Ruby Syntax
                             Highlighter


Saturday, 23 February 13
What


                           A tool to read in simple Ruby source and
                           output syntax highlighted HTML




Saturday, 23 February 13
Why

                           Because I thought it would be fun
                           It was
                           Because I thought it would be easy
                           …



Saturday, 23 February 13
Why



Saturday, 23 February 13
How
                           Build a parse tree of the Ruby source
                           Walk the tree and spit out a <span>
                           element for each bit of text
                           Oh yes, make sure each line goes in <div>
                           and <pre> tags
                           Wrap it in <html>
                           And for bonus points, do some fancy
                           method highlighting
Saturday, 23 February 13
Switch to Chrome,
                                 idiot


Saturday, 23 February 13
Switch to
        Sublime Text again, idiot
                            Code is now available:
           https://github.com/patchspace/parsing_for_fun_and_profit/




Saturday, 23 February 13
We’re doing this the
                                hard way

                           Ruby’s grammar is too complex and
                           undefined to easily implement as a PEG
                           Tools for parsing Ruby already exist




Saturday, 23 February 13
Ripper (Ruby 1.9.3)
Saturday, 23 February 13
Learn
                           more!
        Skip theoretical physics,
       start by playing with Lego




Saturday, 23 February 13
Do more
                           Ideas you might like to try:
                            CSV parser
                            JSON parser (return arrays & hashes)
                            XML parser
                            JSON highlighter
                            A simple JavaScript minifier (just kill
                            whitespace)

Saturday, 23 February 13
Thank you

                                              Ash Moran
                               ash.moran@patchspace.co.uk
                                           PatchSpace Ltd
Saturday, 23 February 13

Más contenido relacionado

Más de PatchSpace Ltd

Conflict in Complex Systems
Conflict in Complex SystemsConflict in Complex Systems
Conflict in Complex SystemsPatchSpace Ltd
 
Personal Kanban (lightning talk)
Personal Kanban (lightning talk)Personal Kanban (lightning talk)
Personal Kanban (lightning talk)PatchSpace Ltd
 
Why Won't My Car Start?
Why Won't My Car Start?Why Won't My Car Start?
Why Won't My Car Start?PatchSpace Ltd
 
ShRUG 5 - Scottish Ruby Conf edition
ShRUG 5  - Scottish Ruby Conf editionShRUG 5  - Scottish Ruby Conf edition
ShRUG 5 - Scottish Ruby Conf editionPatchSpace Ltd
 
Encouraging Agile Discipline
Encouraging Agile DisciplineEncouraging Agile Discipline
Encouraging Agile DisciplinePatchSpace Ltd
 
From Specification To Success
From Specification To SuccessFrom Specification To Success
From Specification To SuccessPatchSpace Ltd
 
Uses & Abuses of Mocks & Stubs
Uses & Abuses of Mocks & StubsUses & Abuses of Mocks & Stubs
Uses & Abuses of Mocks & StubsPatchSpace Ltd
 
NWRUG July 2009 - Darcs
NWRUG July 2009 - DarcsNWRUG July 2009 - Darcs
NWRUG July 2009 - DarcsPatchSpace Ltd
 
Elephants In The Meeting Room
Elephants In The Meeting RoomElephants In The Meeting Room
Elephants In The Meeting RoomPatchSpace Ltd
 

Más de PatchSpace Ltd (9)

Conflict in Complex Systems
Conflict in Complex SystemsConflict in Complex Systems
Conflict in Complex Systems
 
Personal Kanban (lightning talk)
Personal Kanban (lightning talk)Personal Kanban (lightning talk)
Personal Kanban (lightning talk)
 
Why Won't My Car Start?
Why Won't My Car Start?Why Won't My Car Start?
Why Won't My Car Start?
 
ShRUG 5 - Scottish Ruby Conf edition
ShRUG 5  - Scottish Ruby Conf editionShRUG 5  - Scottish Ruby Conf edition
ShRUG 5 - Scottish Ruby Conf edition
 
Encouraging Agile Discipline
Encouraging Agile DisciplineEncouraging Agile Discipline
Encouraging Agile Discipline
 
From Specification To Success
From Specification To SuccessFrom Specification To Success
From Specification To Success
 
Uses & Abuses of Mocks & Stubs
Uses & Abuses of Mocks & StubsUses & Abuses of Mocks & Stubs
Uses & Abuses of Mocks & Stubs
 
NWRUG July 2009 - Darcs
NWRUG July 2009 - DarcsNWRUG July 2009 - Darcs
NWRUG July 2009 - Darcs
 
Elephants In The Meeting Room
Elephants In The Meeting RoomElephants In The Meeting Room
Elephants In The Meeting Room
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelDeepika Singh
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusZilliz
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...apidays
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native ApplicationsWSO2
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Zilliz
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 

Último (20)

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 

Parsing for Fun and Profit

  • 1. Parsing for Fun and Profit (but mainly fun) Ash Moran ash.moran@patchspace.co.uk PatchSpace Ltd Saturday, 23 February 13
  • 3. Parsing Adding structure and meaning to text Saturday, 23 February 13
  • 4. Parsing Human Languages Jake stretched his legs “Jake”, “stretched”, “his”, “legs” “Jake”<noun>, “stretched”<verb, past>, “his”<possessive pronoun>, “legs”<noun> “Jake” <noun, subject>, “stretched”, (“his”, “legs”)<noun phrase, object> Saturday, 23 February 13
  • 5. Parsing Computer Languages “foo = bar + 123” “foo”, “=”, “bar”, “+”, “123” “foo”<var>, “=”<assignment_op>, “bar”<var>, “+”<op_plus>, “123”<int_literal> Saturday, 23 February 13
  • 7. Not just compiling! Compilers breathe fire. Saturday, 23 February 13
  • 9. Pretty Printing gofmt http://gofmt.com/ Saturday, 23 February 13
  • 10. Code Smell Detectors https://rubygems.org/gems/reek Saturday, 23 February 13
  • 12. Other ideas Code metrics Bug detectors Domain-specific languages Language translators (e.g. Ruby -> PHP) Code obfuscators Alternative syntaxes (e.g. CoffeeScript) Refactoring tools Saturday, 23 February 13
  • 14. Step 1 3 year computer science degree Saturday, 23 February 13
  • 15. Lexing/Tokenising if x > 100 then return “big” else return “small” if x > 100 then return “big” else return “small” Saturday, 23 February 13
  • 16. Tree Building if x > 100 then return “big” else return a + b if > then else x 100 return return “big” + a b Saturday, 23 February 13
  • 17. Parsing Expression Grammars Like regular expressions, but can handle recursion, e.g. HTML Not actually that much harder to use Saturday, 23 February 13
  • 18. Regexes and HTML Saturday, 23 February 13
  • 20. Doing Sums Saturday, 23 February 13
  • 21. Switch to Sublime Text, idiot Code is now available: https://github.com/patchspace/parsing_for_fun_and_profit/ Saturday, 23 February 13
  • 22. A Ruby Syntax Highlighter Saturday, 23 February 13
  • 23. What A tool to read in simple Ruby source and output syntax highlighted HTML Saturday, 23 February 13
  • 24. Why Because I thought it would be fun It was Because I thought it would be easy … Saturday, 23 February 13
  • 26. How Build a parse tree of the Ruby source Walk the tree and spit out a <span> element for each bit of text Oh yes, make sure each line goes in <div> and <pre> tags Wrap it in <html> And for bonus points, do some fancy method highlighting Saturday, 23 February 13
  • 27. Switch to Chrome, idiot Saturday, 23 February 13
  • 28. Switch to Sublime Text again, idiot Code is now available: https://github.com/patchspace/parsing_for_fun_and_profit/ Saturday, 23 February 13
  • 29. We’re doing this the hard way Ruby’s grammar is too complex and undefined to easily implement as a PEG Tools for parsing Ruby already exist Saturday, 23 February 13
  • 31. Learn more! Skip theoretical physics, start by playing with Lego Saturday, 23 February 13
  • 32. Do more Ideas you might like to try: CSV parser JSON parser (return arrays & hashes) XML parser JSON highlighter A simple JavaScript minifier (just kill whitespace) Saturday, 23 February 13
  • 33. Thank you Ash Moran ash.moran@patchspace.co.uk PatchSpace Ltd Saturday, 23 February 13