SlideShare una empresa de Scribd logo
1 de 11
Descargar para leer sin conexión
Full-Text Search
on Heroku . . .
FOR FREE!
(by working around bogons in pg_search)


by Dave Aronson, T. Rex of Codosaurus, LLC
What's the Problem?

:-)   Heroku gives lots of stuff for free.
:-(   But not its full-text search add-ons.
:-)   But we still have the database.
:-(   But using LIKE for that, sucks.
:-)   But PostgreSQL has full text search.
:-(   But that feature is a royal pain to use.
:-)   But the pg_search gem makes it easy.
:-(   But its scopes have problems.
:-)   But I've figured out workarounds.
Example: Job Model

class JobsController < ApplicationController

  def index    # also serves as a search screen

      @jobs = Job.in_category(params[:category]).
                  desc_has(params[:desc]).
                  in_state(params[:state]).
                  title_has(params[:title])
                  # quick and dirty example,
                  # ignoring order and pagination

  end

end
Making pg_search_scopes
class Job < ActiveRecord::Base

  include PgSearch

  attr_accessible :category, :desc, :state, :title

  scope :in_category, lambda { |cat|
    where(category: cat) if cat.present? }

  pg_search_scope :desc_has, against: :desc

  scope :in_state, lambda { |st|
    where(state: st) if st.present? }

  pg_search_scope :title_has, against: :title

end
Bogon #1: Not Optional

- Fails SILENTLY (as far as user knows) --
  no results; only clue is in logs/screen:
Started GET "/jobs" [...]
Processing by JobsController#index as HTML

NOTICE: text-search query doesn't contain lexemes: ""
LINE 1: ...glish', coalesce("jobs"."desc"::text, ''))) @@
('')))

- "query doesn't contain lexemes" means:
 "I need something to search for".
Fix: Wrap it up, I'll take it

class Job < ActiveRecord::Base
  include PgSearch
  attr_accessible :category, :desc, :state, :title
  scope :in_category, lambda { |cat|
    where(category: cat) if cat.present? }
  scope :desc_has, lambda { |desc|
    _desc_has(desc) if desc.present? }
  scope :in_state, lambda { |st|
    where(state: st) if st.present? }
  scope :title_has, lambda { |ttl|
    _title_has(ttl) if ttl.present? }
private
  pg_search_scope :_desc_has, against: :desc
  pg_search_scope :_title_has, against: :title
end
Bogon #2: ONE per Query!
ActiveRecord::StatementInvalid in Jobs#index
Showing /Users/dave/jobboard/app/views/jobs/index.html.erb where line #49 raised:
PG::Error: ERROR: ORDER BY "pg_search_rank" is ambiguous
LINE 1: ...nglish', ''' ' || 'senator' || ' ''')))) ORDER BY pg_search_...
                                                                    ^

- I didn't ask for pg_search_rank!
- Look at the generated SQL:
    SELECT "jobs".*, ([gobbledygook]) AS
    pg_search_rank, "jobs".*, ([similar
    gobbledygook]) AS pg_search_rank FROM
    "jobs" WHERE ([more gobbledygook]) ORDER
    BY pg_search_rank DESC, "jobs"."id" ASC
- It doesn't know which one we want!
Wrong Fix: add .order
class JobsController < ApplicationController
  def index # also serves as a search screen
    @jobs = Job.in_category(params[:category]).
                desc_has(params[:desc]).
                in_state(params[:state]).
                title_has(params[:title]).
                order(:id)
  end
end

- Generated SQL now ends in:
   ORDER BY pg_search_rank DESC,
   "jobs"."id" ASC, id
- Didn't clear; just tacked it on!
Right Fix: add .reorder
class JobsController < ApplicationController
  def index # also serves as a search screen
    @jobs = Job.in_category(params[:category]).
                desc_has(params[:desc]).
                in_state(params[:state]).
                title_has(params[:title]).
                reorder(:id)
  end
end

- Generated SQL now ends in:
   ORDER BY id
- No longer barfs; query now works right.
Summary

To do full-text search for free on Heroku,
you can use pg_search, BUT...

To make its scopes optional, wrap them in
normal scopes with no-argument detection

To use two or more in one query, tack
.reorder onto the query, using a
non-ambiguous column or value
Questions/Contact/Etc.

Any questions?
Contact information:
  YourNameHere.2.trex [at] codosaur [dot] us
   (yes, put your name there, without spaces, punctuation, etc.)
   +1-571-308-6622
   http://www.codosaur.us/ (main site)
   http://blog.codosaur.us/ (code blog)                             Brought to
   http://www.dare2XL.com/ (excellence blog)                          you by
                                                                   Codosaurus,
   http://linkedin.com/in/davearonson                                  LLC
   http://facebook.com/dare2xl
   @davearonson

Más contenido relacionado

Más de Dave Aronson

Más de Dave Aronson (8)

Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
Kill All Mutants! (Intro to Mutation Testing), Code Europe (Poland), 2021
 
Kill All Mutants KCDC 2021
Kill All Mutants KCDC 2021Kill All Mutants KCDC 2021
Kill All Mutants KCDC 2021
 
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
"Kill All Mutants! (Intro to Mutation Testing)" Slides from NDC Sydney 2020
 
ACRUMEN Slides for DevConf Poland
ACRUMEN Slides for DevConf PolandACRUMEN Slides for DevConf Poland
ACRUMEN Slides for DevConf Poland
 
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
ACRUMEN Slides for Arlington Ruby (Practice for DevConf)
 
ACRUMEN Slides for RubyNation and CapitalGo, 2018
ACRUMEN Slides for RubyNation and CapitalGo, 2018ACRUMEN Slides for RubyNation and CapitalGo, 2018
ACRUMEN Slides for RubyNation and CapitalGo, 2018
 
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
ACRUMEN: Key Aspects of Software Quality (half-hour talk slides)
 
Ruby Gotchas
Ruby GotchasRuby Gotchas
Ruby Gotchas
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Último (20)

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 

Full text search on Heroku -- for FREE!

  • 1. Full-Text Search on Heroku . . . FOR FREE! (by working around bogons in pg_search) by Dave Aronson, T. Rex of Codosaurus, LLC
  • 2. What's the Problem? :-) Heroku gives lots of stuff for free. :-( But not its full-text search add-ons. :-) But we still have the database. :-( But using LIKE for that, sucks. :-) But PostgreSQL has full text search. :-( But that feature is a royal pain to use. :-) But the pg_search gem makes it easy. :-( But its scopes have problems. :-) But I've figured out workarounds.
  • 3. Example: Job Model class JobsController < ApplicationController def index # also serves as a search screen @jobs = Job.in_category(params[:category]). desc_has(params[:desc]). in_state(params[:state]). title_has(params[:title]) # quick and dirty example, # ignoring order and pagination end end
  • 4. Making pg_search_scopes class Job < ActiveRecord::Base include PgSearch attr_accessible :category, :desc, :state, :title scope :in_category, lambda { |cat| where(category: cat) if cat.present? } pg_search_scope :desc_has, against: :desc scope :in_state, lambda { |st| where(state: st) if st.present? } pg_search_scope :title_has, against: :title end
  • 5. Bogon #1: Not Optional - Fails SILENTLY (as far as user knows) -- no results; only clue is in logs/screen: Started GET "/jobs" [...] Processing by JobsController#index as HTML NOTICE: text-search query doesn't contain lexemes: "" LINE 1: ...glish', coalesce("jobs"."desc"::text, ''))) @@ (''))) - "query doesn't contain lexemes" means: "I need something to search for".
  • 6. Fix: Wrap it up, I'll take it class Job < ActiveRecord::Base include PgSearch attr_accessible :category, :desc, :state, :title scope :in_category, lambda { |cat| where(category: cat) if cat.present? } scope :desc_has, lambda { |desc| _desc_has(desc) if desc.present? } scope :in_state, lambda { |st| where(state: st) if st.present? } scope :title_has, lambda { |ttl| _title_has(ttl) if ttl.present? } private pg_search_scope :_desc_has, against: :desc pg_search_scope :_title_has, against: :title end
  • 7. Bogon #2: ONE per Query! ActiveRecord::StatementInvalid in Jobs#index Showing /Users/dave/jobboard/app/views/jobs/index.html.erb where line #49 raised: PG::Error: ERROR: ORDER BY "pg_search_rank" is ambiguous LINE 1: ...nglish', ''' ' || 'senator' || ' ''')))) ORDER BY pg_search_... ^ - I didn't ask for pg_search_rank! - Look at the generated SQL: SELECT "jobs".*, ([gobbledygook]) AS pg_search_rank, "jobs".*, ([similar gobbledygook]) AS pg_search_rank FROM "jobs" WHERE ([more gobbledygook]) ORDER BY pg_search_rank DESC, "jobs"."id" ASC - It doesn't know which one we want!
  • 8. Wrong Fix: add .order class JobsController < ApplicationController def index # also serves as a search screen @jobs = Job.in_category(params[:category]). desc_has(params[:desc]). in_state(params[:state]). title_has(params[:title]). order(:id) end end - Generated SQL now ends in: ORDER BY pg_search_rank DESC, "jobs"."id" ASC, id - Didn't clear; just tacked it on!
  • 9. Right Fix: add .reorder class JobsController < ApplicationController def index # also serves as a search screen @jobs = Job.in_category(params[:category]). desc_has(params[:desc]). in_state(params[:state]). title_has(params[:title]). reorder(:id) end end - Generated SQL now ends in: ORDER BY id - No longer barfs; query now works right.
  • 10. Summary To do full-text search for free on Heroku, you can use pg_search, BUT... To make its scopes optional, wrap them in normal scopes with no-argument detection To use two or more in one query, tack .reorder onto the query, using a non-ambiguous column or value
  • 11. Questions/Contact/Etc. Any questions? Contact information: YourNameHere.2.trex [at] codosaur [dot] us (yes, put your name there, without spaces, punctuation, etc.) +1-571-308-6622 http://www.codosaur.us/ (main site) http://blog.codosaur.us/ (code blog) Brought to http://www.dare2XL.com/ (excellence blog) you by Codosaurus, http://linkedin.com/in/davearonson LLC http://facebook.com/dare2xl @davearonson