My talk on search and the Sitecore.ContentSearch API at the 2015 SUGNordic in Malmö.

1. Google Is Just a Two Page
SiteRelevant Results with Sitecore.ContentSearch
Martina Helene Welander
Technical Consulting Engineer, Sitecore

2. Speaker
• Technical Consulting Engineer at Sitecore
• Community and Information Enthusiast
• Ecosystem Sites with Dnepropetrovsk Team
Martina Helene
Welander

4. Hi!
• Martina Welander
• Technical Consulting Engineer
• Ecosystem sites
• mhwelander.net / @mhwelander

5. Speaker
• Technical Consulting Engineer at Sitecore
• Community and Information Enthusiast
• Ecosystem Sites with Dnepropetrovsk Team
• @mhwelander / mhwelander.net
Martina Helene
Welander

7. Speaker

8. In the direction of awesome, that’s where

9. …let’s do search!

11. Can haz
knowledge?

12. Google Is Just a Two Page
SiteRelevant Results with Sitecore.ContentSearch
Martina Helene Welander
Technical Consulting Engineer, Sitecore

13. “Google is simply a search box with
a second page of results. And those
results are from other sites!”

16. Lalala hello
world examples
lalala ten items
in my tree!

19. Sitecore.ContentSearch 101

20. Sitecore 7

21. Search and index
ALL the items
*
*

23. Search API
(LINQ-based)
Search Technology Provider
(DLLs and Configuration)
Search Technology API and Indexes
IEnumerable<DocSearchResult>

24. var index =
Sitecore.ContentSearch.ContentSearchManager.GetIndex("sitecore_master_index");
using (var context = index.CreateSearchContext())
{
var query = context.GetQueryable<ResultItem>().Where(x => x.Title == "Hej");
var executedResults = query.GetResults();
myModel.myList = executedResults.Hits.Select(x => x.Document).ToList();
}

26. Where Sitecore adds value
• Source content to index to strongly typed object – and back again!
• You can actually index anything
• Provider model – Solr, Lucene, Elastic Search, Azure Search
• Provider-agnostic LINQ-based search API
• Highly configurable

27. Sitecore.ContentSearch is an API

29. Where should I focus my efforts?

31. CONFIIIIIG!

33. Crawlers
Mappers
Converters
Sitecore Field  Index Field  Object Property
Analyzers
Sitecore Field  Searchable Data
Analyzer Wrappers

34. Back to Plain Ol’ Search
Actually kind of difficult

35. It’s all about the Pentiums analyzers
(Tokenizers and Filters)

36. Tokenizers

37. Hello my name is
Martina
“Hello”, “my”, “name”, “is”,
“Martina”

38. Types of Tokenizer
StandardTokenizer
“My name is Martina”  “My”, “name”, “is”, “Martina”
KeywordTokenizer
“My name is Martina”  “My name is Martina”
N-Gram Tokenizer (Min 4, Max 5)
“sitecore” -> “site”, “itec”, “ecor”, “core”, “siteco”, “iteco” … etc

39. Filters

40. Examples of Filters
• Standard Filter
• (Snowball) Porter Stem Filter
• Stop Filter
• Synonym Filter
• Keep Words Filter
• Pattern Replace Filter
ORDER
MATTERS!

41. Indexing Process

43. Index

44. Query

46. Results

47. “name”
”Hello”
“Hello, my name is Martina”
“Martina”
“my”
Rebuild when
analyser
changes!
Contains(“Hello, my
name is Martina”)

48. Configuring a custom analyzer

49. Lucene – What does it look like?

52. Solr – What does it look like?
<fieldType name="text" class="solr.TextField">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory"/>
<filter class="solr.EnglishPorterFilterFactory"/>
</analyzer>
</fieldType>

53. Previewing and help

57. 6492 12:54:21 INFO ExecuteQueryAgainstLucene
(sitecore_master_index): content:make~0.7
title:make~0.7 content:new~0.7 title:new~0.7
content:item~0.7 title:item~0.7 - Filter :
Debugging A Lucene-Based ContentSearch In Sitecore
- Dan Cruickshank

58. My Super-Duper Analyzer

59. …which isn’t very special at all
• Standard analyser
• Standard filter
• Porter Stem Filter
• StopWords Filter
• Synonym Filter (EXM / ECM, PXM / APS)*
• Lowercase filter

60. The Query

61. What makes something relevant? (tf.idf)
• tf – term frequency
• Idf – inverse document frequency
• coord - # of terms found in document
• fieldNorm – field length

62. My fields
• Title
• Text
• Byline
• Keywords
• Product

63. context.GetQueryable<ResultItem>()
.Where(…)

64. .Filter() vs .Where()

65. #1 – Find me a match
• Equals()
• Contains()
• StartsWith()
.Where(x => x.ResultsTitle.Contains("scaling"))
.Where(x => x["scaling"].Contains("scaling"))
.Match()
.EndsWith()

66. #2 – Slop and fuzziness!
• Like()
• Fuzzy search – fuzziness factor (float)
• Phrase search – slop (int)

67. #3 – I love you, PredicateBuilder
Expression<Func<ResultItem, bool>> predicate =
PredicateBuilder.True<ResultItem>();
foreach (var word in list)
{
predicate = predicate.Or(x =>
x.Title.Contains(word);
}
False for ‘OR’,
True for ‘AND’

68. #4 – Boost
• At query time
• At index time (type or field)
• Rules-based

69. BOOST

70. BOOST

71. ~1000 real items
storageType=“true”

72. Attempt #1: EVERYTHING

74. If the title…
• Like phrase (with slop)
• Contains phrase
• Starts with phrase
• Equals phrase
If the content…
• Like phrase (with slop)
• Contains phrase
• Starts with phrase
• Equals phrase

75. Search: xDB Scaling

76. Search: Managing engagement plans

77. Search: Create engagement plans

79. A couple of important lessons
• Whole Phrases vs Individual Terms
• Boost()
• Contains() / Equals()

80. Attempt #2: Phrase and terms

81. “engagement plan setup”
OR
“engagement” OR “plan” OR “setup”

82. “engagement” AND “plan”
OR
“engagement” AND “setup”
OR
“plan” AND “setup”
OR
“engagement” AND “plan” AND “setup”

83. Needs more boost

85. Attempt #3: Favouring titles

86. Sitecore 7 ContentSearch Tips
- Matt Burke
“Finding a user’s search term in the title or
keywords of a document is probably more
relevant than one where the term is only in
the body”

88. My work in progress

89. If nothing is working, you probably didn’t
rebuild your index

95. Search: xDB Scaling

96. Search: Manage engagement plans

97. Search: Create engagement plans

98. // TODO: On the plane home
• Keywords
• Location
• Pinning exact title matches – “scaling”
• Expected search phrases with boost – e.g. “scaling xDB”, “xDB
scaling”, “xDB scaling options”
xDB
• Key Behaviour Cache – developer or editor?
• Common searches

99. It’s not all queries and indexes
• Vague titles are a bit of a nightmare
• Review use of keywords in content
• “I would never search for that!”
• Continuous user testing and tuning

100. What I learned
• It isn’t magic
• Get to know the provider
• Content and content structure matter
• Search is actually quite hard

102. OrganizersSponsor
Thanks to our… &…