Web Scraping using Diazo!

•

5 recomendaciones•4,280 vistas

pythonchile

Web Scraping using Diazo! Talk given at the StarTechConf 2011 Santiago, Chile www.startechconf.com

Tecnología

Web Scraping
@alvaro_aguirre

Saturday, November 5, 2011

In search of our cosmic origins...

Saturday, November 5, 2011

Data Scraping
vs
Web Scraping

Saturday, November 5, 2011

Data Scraping
<html>
<header></header>
<body>
.....
</body>
</html>

Saturday, November 5, 2011

Web Scraping

Saturday, November 5, 2011

Deliverance
XDV
Diazo

Saturday, November 5, 2011

<replace css:content=”h1” css:theme=”#main” />

Saturday, November 5, 2011

<drop css:content=”h1” />

<drop css:theme=”breadcrumbs” />

Saturday, November 5, 2011

<replace css:theme=”#header” content=”#header-
element” if-content=”” />

Saturday, November 5, 2011

<drop css:theme="#info-box" if-path="/news"/>

Saturday, November 5, 2011

<theme/>
<notheme/>
<replace/>
<before/>
<after/>
<drop/>
<strip/>
<merge/>
<copy/>

Saturday, November 5, 2011

<replace css:theme="#details">
<dl id="details">
<xsl:for-each css:select="table#details > tr">
<dt><xsl:copy-of select="td[1]/text()" /></dt>
<dd><xsl:copy-of select="td[2]/node()"/></dd>
</xsl:for-each>
</dl>
</replace>/></dt>

<table id="details">
<dl id="details">
<tr>
<dt>One</dt>
<td>One</td>
<dd>1</dd>
<td>1</td>
<dt>Two</dt>
</tr>
<dd>2</dd>
<tr>
</dl>
<td>Two</td>
<td>2</td>
</tr>
</table>

Saturday, November 5, 2011

External Content

Saturday, November 5, 2011

• development of web & mobile interfaces
• legacy apps integrations
• prototypes
• low coupling

Saturday, November 5, 2011

from diazo.compiler import compile_theme
from lxml import etree
from diazo.compiler import compile_theme

absolute_prefix = "/static"

rules = "rules.xml"
theme = "theme.html"

compiled_theme = compile_theme(rules, theme,
absolute_prefix=absolute_prefix)

transform = etree.XSLT(compiled_theme)
content = etree.parse(some_content)
transformed = transform(content)

output = etree.tostring(transformed)

Saturday, November 5, 2011

github/aaguirre

Saturday, November 5, 2011

Más contenido relacionado

Similar a Web Scraping using Diazo!

Semantic chopPascal Rettig

What's new in HTML5, CSS3 and JavaScript, James PearceSencha

Moving to Dojo 1.7 and the path to 2.0James Thomas

international PHP2011_ilia alshanetsky_Hidden Features of PHPsmueller_sandsmedia

JS Tooling in Rails 3.1Duda Dornelles

Javascript - How to avoid the bad partsMikko Ohtamaa

잘 알려지지 않은 Php 코드 활용하기형우 안

Sequel @ madrid-rbFernando Blat

永不止步的“重构”Kejun Zhang

A Tour Through the Groovy EcosystemLeonard Axelsson

让开发也懂前端lifesinger

Parser combinatorslifecoder

Xmlbaabtra.com - No. 1 supplier of quality freshers

Clouds against the Floods (RubyConfBR2011) Leonardo Borges

Ext GWT 3.0 Advanced TemplatesSencha

HTML5 & CSS3 in Drupal (on the Bayou)Mediacurrent

Rails ORM De-mystifying Active Record has_manyBlazing Cloud

The Solar Framework for PHPConFoo

The Easy Way - Plone Conference 2011Mikko Ohtamaa

Xml presentationMiguel Angel Teheran Garcia

Similar a Web Scraping using Diazo! (20)

Semantic chop

What's new in HTML5, CSS3 and JavaScript, James Pearce

Moving to Dojo 1.7 and the path to 2.0

international PHP2011_ilia alshanetsky_Hidden Features of PHP

JS Tooling in Rails 3.1

Javascript - How to avoid the bad parts

잘 알려지지 않은 Php 코드 활용하기

Sequel @ madrid-rb

永不止步的“重构”

A Tour Through the Groovy Ecosystem

让开发也懂前端

Parser combinators

Xml

Clouds against the Floods (RubyConfBR2011)

Ext GWT 3.0 Advanced Templates

HTML5 & CSS3 in Drupal (on the Bayou)

Rails ORM De-mystifying Active Record has_many

The Solar Framework for PHP

The Easy Way - Plone Conference 2011

Xml presentation

Último

Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun

Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer

Presentation on how to chat with PDF using ChatGPT code interpreternaman860154

From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software

Histor y of HAM Radio presentation slidevu2urc

Artificial Intelligence: Facts and MythsJoaquim Jorge

08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls

GenCyber Cyber Security Day PresentationMichael W. Hawkins

08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls

Real Time Object Detection Using Open CVKhem

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo

Automating Google Workspace (GWS) & more with Apps Scriptwesley chun

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j

Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC

The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los

Scaling API-first – The story of a global engineering organizationRadu Cotescu

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science

Slack Application Development 101 Slidespraypatel2

Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2

Web Scraping using Diazo!

1. Web Scraping @alvaro_aguirre Saturday, November 5, 2011

2. In search of our cosmic origins... Saturday, November 5, 2011

3. Saturday, November 5, 2011

4. Saturday, November 5, 2011

5. Saturday, November 5, 2011

6. Saturday, November 5, 2011

7. Saturday, November 5, 2011

8. Saturday, November 5, 2011

9. Data Scraping vs Web Scraping Saturday, November 5, 2011

10. Data Scraping <html> <header></header> <body> ..... </body> </html> Saturday, November 5, 2011

11. Web Scraping Saturday, November 5, 2011

12. Saturday, November 5, 2011

13. Saturday, November 5, 2011

14. Deliverance XDV Diazo Saturday, November 5, 2011

15. Diazo Saturday, November 5, 2011

16. Saturday, November 5, 2011

17. <replace css:content=”h1” css:theme=”#main” /> Saturday, November 5, 2011

18. <drop css:content=”h1” /> <drop css:theme=”breadcrumbs” /> Saturday, November 5, 2011

19. <replace css:theme=”#header” content=”#header- element” if-content=”” /> Saturday, November 5, 2011

20. <drop css:theme="#info-box" if-path="/news"/> Saturday, November 5, 2011

21. <theme/> <notheme/> <replace/> <before/> <after/> <drop/> <strip/> <merge/> <copy/> Saturday, November 5, 2011

22. <replace css:theme="#details"> <dl id="details"> <xsl:for-each css:select="table#details > tr"> <dt><xsl:copy-of select="td[1]/text()" /></dt> <dd><xsl:copy-of select="td[2]/node()"/></dd> </xsl:for-each> </dl> </replace>/></dt> <table id="details"> <dl id="details"> <tr> <dt>One</dt> <td>One</td> <dd>1</dd> <td>1</td> <dt>Two</dt> </tr> <dd>2</dd> <tr> </dl> <td>Two</td> <td>2</td> </tr> </table> Saturday, November 5, 2011

23. Saturday, November 5, 2011

24. Saturday, November 5, 2011

25. Saturday, November 5, 2011

26. Tools Saturday, November 5, 2011

27. External Content Saturday, November 5, 2011

28. Saturday, November 5, 2011

29. • development of web & mobile interfaces • legacy apps integrations • prototypes • low coupling Saturday, November 5, 2011

30. from diazo.compiler import compile_theme from lxml import etree from diazo.compiler import compile_theme absolute_prefix = "/static" rules = "rules.xml" theme = "theme.html" compiled_theme = compile_theme(rules, theme, absolute_prefix=absolute_prefix) transform = etree.XSLT(compiled_theme) content = etree.parse(some_content) transformed = transform(content) output = etree.tostring(transformed) Saturday, November 5, 2011

31. github/aaguirre Saturday, November 5, 2011

32. diazo.org Saturday, November 5, 2011

33. plone.org Saturday, November 5, 2011

34. gracias! Saturday, November 5, 2011

Web Scraping using Diazo!

Recomendados

Recomendados

Más contenido relacionado

Similar a Web Scraping using Diazo!

Similar a Web Scraping using Diazo! (20)

Último

Último (20)

Web Scraping using Diazo!