How Flax helped Westcoast, the UK's largest privately owned IT distributor, implement Elasticsearch for their website

1. Charlie Hull, Managing Director, Flax
Nick Gushlow, Systems Architect, Westcoast
Elasticsearch London Meetup
charlie@flax.co.uk
www.flax.co.uk/blog
+44 (0) 8700 118334
Twitter: @FlaxSearch
Search is never Simple
Elasticsearch for Westcoast

2. Building open source search applications since 2001
Independent, honest advice and analysis
Expert design & development, Apache Solr committers
Test-driven relevancy and performance tuning
Custom training & mentoring for your staff
Flexible support up to 24/7/365 with SLAs
Come to our Meetups!

4. Charlie Hull, Managing Director & co-founder of Flax
Nick Gushlow, Systems Architect at Westcoast
Who are we?
@FlaxSearch

5. Why Westcoast needed a new search engine
The source data & the plan
The trouble with....
Building an admin panel for search
The search goes live
Lessons learned
What we'll cover today
@FlaxSearch

6. Largest privately owned IT distributor in UK & Ireland
£1.5 billion turnover
Apple, HP, Lenovo, Microsoft, Samsung, Toshiba
Includes XMA / QC Supplies and Viglen
Who are Westcoast?
@FlaxSearch

7. Old SQL based search not accurate enough
8000 searches per day, 90% SKU based
You searched iPad …. did you actually want an iPad?
Customers used Google / competitors to find part numbers,
Static traffic numbers – 3500 user per day 7am to 7pm
Increase web revenue further, currently £40m
Why a new search engine?
@FlaxSearch

8. Business approved a project to implement a change to
‘improve search’
Google Search Appliance
SLI
Apptus
Fredhopper
Elasticsearch
Time for a change
@FlaxSearch

9. Live pricing
XML data sheets
Business user management interface
Synonyms / Exclusions
Boosts
Search vs Search vs Search
Requirements
@FlaxSearch

10. 0.5m products
Nested data (attributes)
Supplied as XML, one file per product
The source data
@FlaxSearch

11. 0.5m products
Nested data (attributes)
Supplied as XML, one file per product
BUT!
– Live Pricing API will restrict results at search time
– Different for every end customer
– Based on hard to explain business rules
The source data
@FlaxSearch

12. Elasticsearch
Java client
Custom Java indexer (Dropwizard)
Search application (Dropwizard)
Admin panels (AngularJS)
Agile process
The plan
@FlaxSearch

13. First, do your search
Send 5000 results to legacy pricing API
Merge the pricing information with search results
Now build your facets (including on price)
Hang on, doesn't Elasticsearch do facets for you?
The trouble with facets
@FlaxSearch

14. Front end systems built by third party
– Solution: Search app with JSON API (defined by them)
– Encrypted JSON for use during sessions
More trouble with facets
@FlaxSearch

15. Front end systems built by third party
– Solution: Search app with JSON API (defined by them)
– Encrypted JSON for use during sessions
Data for all the facets must be supplied to the UI
– Full result counts for applied facets need to be returned, in
the order they were applied
– Solution: lots of searches
More trouble with facets
@FlaxSearch

16. Front end systems built by third party
– Solution: Search app with JSON API (defined by them)
– Encrypted JSON for use during sessions
Data for all the facets must be supplied to the UI
– Full result counts for applied facets need to be returned, in
the order they were applied
– Solution: lots of searches
Custom facets for some customers
– Solution: an index of facet definitions
More trouble with facets
@FlaxSearch

17. Boost for individual items
– Easy! Define in the source data
Term boosts
– e.g. some Macbooks over other Macbooks
– Harder – but still defined in source data
The trouble with boosting
@FlaxSearch

18. A great way to run search projects!
...unless not everyone can do Agile
The trouble with Agile
@FlaxSearch

19. Allows Westcoast to adjust
– Synonyms / Exclusions
– Remove items from index
– Test searches
– Test synonyms then push to live
• Synonyms are index side as default query is AND
Built in AngularJS
Building an Admin panel
@FlaxSearch

20. @FlaxSearch

21. @FlaxSearch

22. @FlaxSearch

23. A single node for Elasticsearch
A single node for index & search applications
Ultimately mirrored for failover
Query load very low (1 QPS)
– But this may change!
Business hours support by Flax
The search goes live
@FlaxSearch

24. @FlaxSearch

25. @FlaxSearch

26. Elasticsearch results were good
Business maintenance, large, boring, never ending work
Changing customer behaviour is slow
Search results over 30% faster on average
Time savings for sales staff
Post Live
@FlaxSearch

27. Integrating with legacy systems is hard
Business rules can be hard to understand & harder to explain
Not everything can be done with search
If you want to do Agile, make sure everyone else can
Lessons learned
@FlaxSearch

28. Thankyou!
Any questions?
charlie@flax.co.uk
www.flax.co.uk/blog
+44 (0) 8700 118334
Twitter: @FlaxSearch
@FlaxSearch

29. Plug
3rd
& 4th
February 2016, Cambridge UK
Open source search for Bioinformatics
Free event near Cambridge on Wellcome Genome Campus covering
both Solr & Elasticsearch, talks & hands-on workshops
http://www.ebi.ac.uk/pdbe/about/events
@FlaxSearch

30. Plug #2
20th
March 2016, Padua, Italy
First International Workshop on Recent Trends in News
Information Retrieval
One-day workshop as part of the European Conference on Information
Retrieval (ECIR 2016) – submission deadline end of January
http://research.signalmedia.co/newsir16/index.html
(including a free test dataset of 1m news articles!)
@FlaxSearch