SlideShare una empresa de Scribd logo
1 de 25
Descargar para leer sin conexión
Inside a modern RIA powered by Solr

                       Andrew Montalenti
                            Co-Founder &
                          Technology Lead
                      andrew@cogtree.com

                                            1
2
Mainstream
                      30,000
                                      Blogs
                                                 1

                900,000
1From   Technorati’s 2008 State of the Blogosphere
                                                     3
But,



What about your interests?   4
your interests,
your web



                  5
6
What is               ?
• Your unique interests
• … create a filtered, prioritized, and personalized news feed
• … built just for you!
• 120K+ news and blog sources tracked
• The most personally relevant items at the top


• Bottom line:
  You spend less time skimming headlines,
  and more time reading relevant content.
                                                                 7
Demo!
(if possible)
Let’s pop open the hood!



                           9
The RIA
       ExtJS + jQuery
       JS REST Binding
          REST API
       django-piston
Solr                 Postgres
                                10
11
Per-Doc
        Processing



Batch
 size


              IO-bound work




                              12
13
Solr in the Real World
•   Storage of "canonical data“
•   Relational vs. Search Index
•   Complexity of custom relevancy scoring
•   "Near-Real-Time" updates
•   Solr in a pipeline
•   Pushing bits and marshalling cost
•   Index size, corruption, and stability
•   Administrability
                                             14
Scaling Up Parse.ly
      • Custom scoring
      • Multicore
      • Distributed search
      • Celery / Disco
      • User-Article Binding Problem




                                       16
17
Basic        Almost There      Comprehensive


   solr.py
                 collective.solr      haystack
    pysolr


    solrpy
                    solango          python-solr
json/py output


                                                   18
19
Batching
                       Context Lib


 Caching
Memoization
                                                Multicore
                                                 Proxies
                  Comprehensive.
                    Pythonic.
                      Solr.


                                        Web
        Pagination
           Iterators                 Framework
                                       Django



                                                            20
powered by



             21
Didier   Sachin
Andrew




                           22
Quick Plug

Does your company or enterprise
need our services?




                                  23
24
24
Andrew Montalenti
                  andrew@cogtree.com



Twitter            Website                   Sign up now!
@amontalenti       http://parse.ly           It’s Free!

Product Twitter    Team Blog                 Promo Code
@parse_ly          http://blog.cogtree.com   SLIDES

                                                      25
                                                      25

Más contenido relacionado

Similar a Parse.ly: Inside a modern RIA built with Solr

高性能网站建设指南
高性能网站建设指南高性能网站建设指南
高性能网站建设指南Bob Huang
 
Hadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSi
Hadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSiHadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSi
Hadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSiCloudera, Inc.
 
[Workshop Summits] Microformats Workshop
[Workshop Summits] Microformats Workshop[Workshop Summits] Microformats Workshop
[Workshop Summits] Microformats WorkshopEmily Lewis
 
Introducing MongoDB into your Organization
Introducing MongoDB into your OrganizationIntroducing MongoDB into your Organization
Introducing MongoDB into your OrganizationMongoDB
 
Is Enterprise Search Ripe for Open Source Disruption?
Is Enterprise Search Ripe for Open Source Disruption?Is Enterprise Search Ripe for Open Source Disruption?
Is Enterprise Search Ripe for Open Source Disruption?Enterprise 2.0 Conference
 
Advertising with Linked Data in Web Content
Advertising with Linked Data in Web ContentAdvertising with Linked Data in Web Content
Advertising with Linked Data in Web ContentMartin Hepp
 
Morning with MongoDB Paris 2012 - Making Big Data Small
Morning with MongoDB Paris 2012 - Making Big Data SmallMorning with MongoDB Paris 2012 - Making Big Data Small
Morning with MongoDB Paris 2012 - Making Big Data SmallMongoDB
 
[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...
[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...
[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...Insight Technology, Inc.
 
microservice architecture and docker
microservice architecture and dockermicroservice architecture and docker
microservice architecture and dockeriitsasi
 
Refactoring a Solr based API application
Refactoring a Solr based API applicationRefactoring a Solr based API application
Refactoring a Solr based API applicationTorsten Köster
 
Lessons Learned: Refactoring a Solr-Based API App - Torsten Koester
Lessons Learned: Refactoring a Solr-Based API App - Torsten KoesterLessons Learned: Refactoring a Solr-Based API App - Torsten Koester
Lessons Learned: Refactoring a Solr-Based API App - Torsten Koesterlucenerevolution
 
01/2009 - Portral development with liferay
01/2009 - Portral development with liferay01/2009 - Portral development with liferay
01/2009 - Portral development with liferaydaveayan
 
Integrity St. Louis WordCamp 2012
Integrity St. Louis WordCamp 2012Integrity St. Louis WordCamp 2012
Integrity St. Louis WordCamp 2012Ed Morrissey
 
UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...
UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...
UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...Serge Huber
 
Drupal for Large scale project
Drupal for Large scale projectDrupal for Large scale project
Drupal for Large scale projectCyril Reinhard
 
Circuit 2015 Keynote - Carsten Ziegeler
Circuit 2015 Keynote -  Carsten ZiegelerCircuit 2015 Keynote -  Carsten Ziegeler
Circuit 2015 Keynote - Carsten ZiegelerICF CIRCUIT
 
Open Innovation means Open Source
Open Innovation means Open SourceOpen Innovation means Open Source
Open Innovation means Open SourceBertrand Delacretaz
 
BlogForever poster
BlogForever posterBlogForever poster
BlogForever posterBlogForever
 
Chirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterChirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterJohn Adams
 

Similar a Parse.ly: Inside a modern RIA built with Solr (20)

高性能网站建设指南
高性能网站建设指南高性能网站建设指南
高性能网站建设指南
 
Hadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSi
Hadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSiHadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSi
Hadoop World 2011: Hadoop in a Mission Critical Environment - Jim Haas - CBSi
 
[Workshop Summits] Microformats Workshop
[Workshop Summits] Microformats Workshop[Workshop Summits] Microformats Workshop
[Workshop Summits] Microformats Workshop
 
Introducing MongoDB into your Organization
Introducing MongoDB into your OrganizationIntroducing MongoDB into your Organization
Introducing MongoDB into your Organization
 
Is Enterprise Search Ripe for Open Source Disruption?
Is Enterprise Search Ripe for Open Source Disruption?Is Enterprise Search Ripe for Open Source Disruption?
Is Enterprise Search Ripe for Open Source Disruption?
 
Advertising with Linked Data in Web Content
Advertising with Linked Data in Web ContentAdvertising with Linked Data in Web Content
Advertising with Linked Data in Web Content
 
Morning with MongoDB Paris 2012 - Making Big Data Small
Morning with MongoDB Paris 2012 - Making Big Data SmallMorning with MongoDB Paris 2012 - Making Big Data Small
Morning with MongoDB Paris 2012 - Making Big Data Small
 
[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...
[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...
[db tech showcase Tokyo 2018] #dbts2018 #B23 『Python, Oracle Cloud, Blockchai...
 
microservice architecture and docker
microservice architecture and dockermicroservice architecture and docker
microservice architecture and docker
 
Refactoring a Solr based API application
Refactoring a Solr based API applicationRefactoring a Solr based API application
Refactoring a Solr based API application
 
Lessons Learned: Refactoring a Solr-Based API App - Torsten Koester
Lessons Learned: Refactoring a Solr-Based API App - Torsten KoesterLessons Learned: Refactoring a Solr-Based API App - Torsten Koester
Lessons Learned: Refactoring a Solr-Based API App - Torsten Koester
 
01/2009 - Portral development with liferay
01/2009 - Portral development with liferay01/2009 - Portral development with liferay
01/2009 - Portral development with liferay
 
Integrity St. Louis WordCamp 2012
Integrity St. Louis WordCamp 2012Integrity St. Louis WordCamp 2012
Integrity St. Louis WordCamp 2012
 
UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...
UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...
UXDev Summit Keynote : A real world story of Angular and Apache Unomi integra...
 
Drupal for Large scale project
Drupal for Large scale projectDrupal for Large scale project
Drupal for Large scale project
 
Circuit 2015 Keynote - Carsten Ziegeler
Circuit 2015 Keynote -  Carsten ZiegelerCircuit 2015 Keynote -  Carsten Ziegeler
Circuit 2015 Keynote - Carsten Ziegeler
 
W3 C Intro And Beyond - Eyal Sela
W3 C Intro And Beyond - Eyal SelaW3 C Intro And Beyond - Eyal Sela
W3 C Intro And Beyond - Eyal Sela
 
Open Innovation means Open Source
Open Innovation means Open SourceOpen Innovation means Open Source
Open Innovation means Open Source
 
BlogForever poster
BlogForever posterBlogForever poster
BlogForever poster
 
Chirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterChirp 2010: Scaling Twitter
Chirp 2010: Scaling Twitter
 

Último

Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Bhuvaneswari Subramani
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...apidays
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...apidays
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdfSandro Moreira
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesrafiqahmad00786416
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelDeepika Singh
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 

Último (20)

Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 

Parse.ly: Inside a modern RIA built with Solr

  • 1. Inside a modern RIA powered by Solr Andrew Montalenti Co-Founder & Technology Lead andrew@cogtree.com 1
  • 2. 2
  • 3. Mainstream 30,000 Blogs 1 900,000 1From Technorati’s 2008 State of the Blogosphere 3
  • 4. But, What about your interests? 4
  • 6. 6
  • 7. What is ? • Your unique interests • … create a filtered, prioritized, and personalized news feed • … built just for you! • 120K+ news and blog sources tracked • The most personally relevant items at the top • Bottom line: You spend less time skimming headlines, and more time reading relevant content. 7
  • 9. Let’s pop open the hood! 9
  • 10. The RIA ExtJS + jQuery JS REST Binding REST API django-piston Solr Postgres 10
  • 11. 11
  • 12. Per-Doc Processing Batch size IO-bound work 12
  • 13. 13
  • 14. Solr in the Real World • Storage of "canonical data“ • Relational vs. Search Index • Complexity of custom relevancy scoring • "Near-Real-Time" updates • Solr in a pipeline • Pushing bits and marshalling cost • Index size, corruption, and stability • Administrability 14
  • 15.
  • 16. Scaling Up Parse.ly • Custom scoring • Multicore • Distributed search • Celery / Disco • User-Article Binding Problem 16
  • 17. 17
  • 18. Basic Almost There Comprehensive solr.py collective.solr haystack pysolr solrpy solango python-solr json/py output 18
  • 19. 19
  • 20. Batching Context Lib Caching Memoization Multicore Proxies Comprehensive. Pythonic. Solr. Web Pagination Iterators Framework Django 20
  • 22. Didier Sachin Andrew 22
  • 23. Quick Plug Does your company or enterprise need our services? 23
  • 24. 24 24
  • 25. Andrew Montalenti andrew@cogtree.com Twitter Website Sign up now! @amontalenti http://parse.ly It’s Free! Product Twitter Team Blog Promo Code @parse_ly http://blog.cogtree.com SLIDES 25 25