SlideShare una empresa de Scribd logo
1 de 11
Descargar para leer sin conexión
Full-Text Search
on Heroku . . .
FOR FREE!
(by working around bogons in pg_search)


by Dave Aronson, T. Rex of Codosaurus, LLC
What's the Problem?

:-)   Heroku gives lots of stuff for free.
:-(   But not its full-text search add-ons.
:-)   But we still have the database.
:-(   But using LIKE for that, sucks.
:-)   But PostgreSQL has full text search.
:-(   But that feature is a royal pain to use.
:-)   But the pg_search gem makes it easy.
:-(   But its scopes have problems.
:-)   But I've figured out workarounds.
Example: Job Model

class JobsController < ApplicationController

  def index    # also serves as a search screen

      @jobs = Job.in_category(params[:category]).
                  desc_has(params[:desc]).
                  in_state(params[:state]).
                  title_has(params[:title])
                  # quick and dirty example,
                  # ignoring order and pagination

  end

end
Making pg_search_scopes
class Job < ActiveRecord::Base

  include PgSearch

  attr_accessible :category, :desc, :state, :title

  scope :in_category, lambda { |cat|
    where(category: cat) if cat.present? }

  pg_search_scope :desc_has, against: :desc

  scope :in_state, lambda { |st|
    where(state: st) if st.present? }

  pg_search_scope :title_has, against: :title

end
Bogon #1: Not Optional

- Fails SILENTLY (as far as user knows) --
  no results; only clue is in logs/screen:
Started GET "/jobs" [...]
Processing by JobsController#index as HTML

NOTICE: text-search query doesn't contain lexemes: ""
LINE 1: ...glish', coalesce("jobs"."desc"::text, ''))) @@
('')))

- "query doesn't contain lexemes" means:
 "I need something to search for".
Fix: Wrap it up, I'll take it

class Job < ActiveRecord::Base
  include PgSearch
  attr_accessible :category, :desc, :state, :title
  scope :in_category, lambda { |cat|
    where(category: cat) if cat.present? }
  scope :desc_has, lambda { |desc|
    _desc_has(desc) if desc.present? }
  scope :in_state, lambda { |st|
    where(state: st) if st.present? }
  scope :title_has, lambda { |ttl|
    _title_has(ttl) if ttl.present? }
private
  pg_search_scope :_desc_has, against: :desc
  pg_search_scope :_title_has, against: :title
end
Bogon #2: ONE per Query!
ActiveRecord::StatementInvalid in Jobs#index
Showing /Users/dave/jobboard/app/views/jobs/index.html.erb where line #49 raised:
PG::Error: ERROR: ORDER BY "pg_search_rank" is ambiguous
LINE 1: ...nglish', ''' ' || 'senator' || ' ''')))) ORDER BY pg_search_...
                                                                    ^

- I didn't ask for pg_search_rank!
- Look at the generated SQL:
    SELECT "jobs".*, ([gobbledygook]) AS
    pg_search_rank, "jobs".*, ([similar
    gobbledygook]) AS pg_search_rank FROM
    "jobs" WHERE ([more gobbledygook]) ORDER
    BY pg_search_rank DESC, "jobs"."id" ASC
- It doesn't know which one we want!
Wrong Fix: add .order
class JobsController < ApplicationController
  def index # also serves as a search screen
    @jobs = Job.in_category(params[:category]).
                desc_has(params[:desc]).
                in_state(params[:state]).
                title_has(params[:title]).
                order(:id)
  end
end

- Generated SQL now ends in:
   ORDER BY pg_search_rank DESC,
   "jobs"."id" ASC, id
- Didn't clear; just tacked it on!
Right Fix: add .reorder
class JobsController < ApplicationController
  def index # also serves as a search screen
    @jobs = Job.in_category(params[:category]).
                desc_has(params[:desc]).
                in_state(params[:state]).
                title_has(params[:title]).
                reorder(:id)
  end
end

- Generated SQL now ends in:
   ORDER BY id
- No longer barfs; query now works right.
Summary

To do full-text search for free on Heroku,
you can use pg_search, BUT...

To make its scopes optional, wrap them in
normal scopes with no-argument detection

To use two or more in one query, tack
.reorder onto the query, using a
non-ambiguous column or value
Questions/Contact/Etc.

Any questions?
Contact information:
  YourNameHere.2.trex [at] codosaur [dot] us
   (yes, put your name there, without spaces, punctuation, etc.)
   +1-571-308-6622
   http://www.codosaur.us/ (main site)
   http://blog.codosaur.us/ (code blog)                             Brought to
   http://www.dare2XL.com/ (excellence blog)                          you by
                                                                   Codosaurus,
   http://linkedin.com/in/davearonson                                  LLC
   http://facebook.com/dare2xl
   @davearonson

Más contenido relacionado

Más de Dave Aronson

Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021Dave Aronson
 
Kill All Mutants KCDC 2021
Kill All Mutants KCDC 2021Kill All Mutants KCDC 2021
Kill All Mutants KCDC 2021Dave Aronson
 
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020Dave Aronson
 
ACRUMEN Slides for DevConf Poland
ACRUMEN Slides for DevConf PolandACRUMEN Slides for DevConf Poland
ACRUMEN Slides for DevConf PolandDave Aronson
 
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)Dave Aronson
 
ACRUMEN Slides for RubyNation and CapitalGo, 2018
ACRUMEN Slides for RubyNation and CapitalGo, 2018ACRUMEN Slides for RubyNation and CapitalGo, 2018
ACRUMEN Slides for RubyNation and CapitalGo, 2018Dave Aronson
 
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)Dave Aronson
 

Más de Dave Aronson (8)

Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
 
Kill All Mutants KCDC 2021
Kill All Mutants KCDC 2021Kill All Mutants KCDC 2021
Kill All Mutants KCDC 2021
 
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
 
ACRUMEN Slides for DevConf Poland
ACRUMEN Slides for DevConf PolandACRUMEN Slides for DevConf Poland
ACRUMEN Slides for DevConf Poland
 
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
 
ACRUMEN Slides for RubyNation and CapitalGo, 2018
ACRUMEN Slides for RubyNation and CapitalGo, 2018ACRUMEN Slides for RubyNation and CapitalGo, 2018
ACRUMEN Slides for RubyNation and CapitalGo, 2018
 
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
 
Ruby Gotchas
Ruby GotchasRuby Gotchas
Ruby Gotchas
 

Último

Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 

Último (20)

Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 

Full text search on Heroku -- for FREE!

  • 1. Full-Text Search on Heroku . . . FOR FREE! (by working around bogons in pg_search) by Dave Aronson, T. Rex of Codosaurus, LLC
  • 2. What's the Problem? :-) Heroku gives lots of stuff for free. :-( But not its full-text search add-ons. :-) But we still have the database. :-( But using LIKE for that, sucks. :-) But PostgreSQL has full text search. :-( But that feature is a royal pain to use. :-) But the pg_search gem makes it easy. :-( But its scopes have problems. :-) But I've figured out workarounds.
  • 3. Example: Job Model class JobsController < ApplicationController def index # also serves as a search screen @jobs = Job.in_category(params[:category]). desc_has(params[:desc]). in_state(params[:state]). title_has(params[:title]) # quick and dirty example, # ignoring order and pagination end end
  • 4. Making pg_search_scopes class Job < ActiveRecord::Base include PgSearch attr_accessible :category, :desc, :state, :title scope :in_category, lambda { |cat| where(category: cat) if cat.present? } pg_search_scope :desc_has, against: :desc scope :in_state, lambda { |st| where(state: st) if st.present? } pg_search_scope :title_has, against: :title end
  • 5. Bogon #1: Not Optional - Fails SILENTLY (as far as user knows) -- no results; only clue is in logs/screen: Started GET "/jobs" [...] Processing by JobsController#index as HTML NOTICE: text-search query doesn't contain lexemes: "" LINE 1: ...glish', coalesce("jobs"."desc"::text, ''))) @@ (''))) - "query doesn't contain lexemes" means: "I need something to search for".
  • 6. Fix: Wrap it up, I'll take it class Job < ActiveRecord::Base include PgSearch attr_accessible :category, :desc, :state, :title scope :in_category, lambda { |cat| where(category: cat) if cat.present? } scope :desc_has, lambda { |desc| _desc_has(desc) if desc.present? } scope :in_state, lambda { |st| where(state: st) if st.present? } scope :title_has, lambda { |ttl| _title_has(ttl) if ttl.present? } private pg_search_scope :_desc_has, against: :desc pg_search_scope :_title_has, against: :title end
  • 7. Bogon #2: ONE per Query! ActiveRecord::StatementInvalid in Jobs#index Showing /Users/dave/jobboard/app/views/jobs/index.html.erb where line #49 raised: PG::Error: ERROR: ORDER BY "pg_search_rank" is ambiguous LINE 1: ...nglish', ''' ' || 'senator' || ' ''')))) ORDER BY pg_search_... ^ - I didn't ask for pg_search_rank! - Look at the generated SQL: SELECT "jobs".*, ([gobbledygook]) AS pg_search_rank, "jobs".*, ([similar gobbledygook]) AS pg_search_rank FROM "jobs" WHERE ([more gobbledygook]) ORDER BY pg_search_rank DESC, "jobs"."id" ASC - It doesn't know which one we want!
  • 8. Wrong Fix: add .order class JobsController < ApplicationController def index # also serves as a search screen @jobs = Job.in_category(params[:category]). desc_has(params[:desc]). in_state(params[:state]). title_has(params[:title]). order(:id) end end - Generated SQL now ends in: ORDER BY pg_search_rank DESC, "jobs"."id" ASC, id - Didn't clear; just tacked it on!
  • 9. Right Fix: add .reorder class JobsController < ApplicationController def index # also serves as a search screen @jobs = Job.in_category(params[:category]). desc_has(params[:desc]). in_state(params[:state]). title_has(params[:title]). reorder(:id) end end - Generated SQL now ends in: ORDER BY id - No longer barfs; query now works right.
  • 10. Summary To do full-text search for free on Heroku, you can use pg_search, BUT... To make its scopes optional, wrap them in normal scopes with no-argument detection To use two or more in one query, tack .reorder onto the query, using a non-ambiguous column or value
  • 11. Questions/Contact/Etc. Any questions? Contact information: YourNameHere.2.trex [at] codosaur [dot] us (yes, put your name there, without spaces, punctuation, etc.) +1-571-308-6622 http://www.codosaur.us/ (main site) http://blog.codosaur.us/ (code blog) Brought to http://www.dare2XL.com/ (excellence blog) you by Codosaurus, http://linkedin.com/in/davearonson LLC http://facebook.com/dare2xl @davearonson