Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Personalizing Search at LinkedIn

1.080 visualizaciones

Publicado el

Bay Area search meetup (June 2015)

Publicado en: Redes sociales
  • Sé el primero en comentar

Personalizing Search at LinkedIn

  1. 1. Recruiting SolutionsRecruiting SolutionsRecruiting Solutions Ganesh Venkataraman Viet Ha-Thuc Personalizing Search @ LinkedIn
  2. 2. Bigger Picture ▪ LinkedIn’s vision – Create economic opportunity for every member of the global workforce ▪ Connect members to other members, knowledge and opportunity and help them be great at what they do
  3. 3. Economic Graph ▪ Organize people, companies, jobs, knowledge and map out the economic graph 3
  4. 4. Role of Search ▪ At the heart of the economic graph, search makes the economic graph accessible, useful and actionable ▪ Powers searching people, jobs, companies, schools etc. ▪ On linkedin.com consumer, recruiter, sales solutions 4
  5. 5. Powered by Search 5
  6. 6. Basic Nomenclature 6 TypeAhead/TYAH Full Search/SERP
  7. 7. Search is ... 7
  8. 8. 8
  9. 9. Search is about understanding the user intent 9
  10. 10. LinkedIn Search - An Overview 10 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  11. 11. Let’s talk intent - Navigational ▪ Navigational - exactly one result in mind 11
  12. 12. Two types of Intent - Exploratory ▪ Exploratory - Typically more than one entity in mind 12
  13. 13. How to handle navigational queries? Be Fast Type Less Be Lenient 13
  14. 14. Handling Navigational Queries ▪ Type Less – Index prefixes (‘ga’, ‘gan’, ‘gane’ => ‘ganesh’) ▪ Be Fast – Do not retrieve all documents – Order documents in posting list by static rank – Modify query for targeted retrieval ▪ Be Lenient – Smart spell correction 14
  15. 15. Exploratory Queries ▪ If possible guide users to more structured queries ▪ Above query could go into different verticals if these are selected ▪ User intent becomes much clearer 15
  16. 16. Exploratory Queries 16
  17. 17. Unclear intent - Federating TYAH results 17
  18. 18. LinkedIn Search - Bird’s eye view 18 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  19. 19. Query Processing - things not strings 1919 TITLE CO GEO TITLE-237 software engineer software developer programmer … CO-1441 Google Inc. Industry: Internet GEO-7583 Country: US Lat: 42.3482 N Long: 75.1890 W (RECOGNIZED TAGS: NAME, TITLE, COMPANY, SCHOOL, GEO, SKILL )
  20. 20. Retrieval ▪ Custom search engine to handle 100’s of millions of documents (Galene) ▪ Key Features: – Offline indexing pipeline – Supports live updates with fine granularity – Static Ranking ▪ Posting list organized by static rank for each document ▪ Enables early termination 20
  21. 21. LinkedIn Search - Bird’s eye view 21 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  22. 22. Ranking ▪ Manually tuning vs. Learning to Rank (LTR) ▪ Why Learning to Rank? – Hard to manually tune with very large number of features – Challenging to personalize – LTR allows leveraging large volume of click data in an automated way 22
  23. 23. Training Data: Human Label
  24. 24. What if the searcher is a job seeker? Or a recruiter? Training Data: Human Label
  25. 25. ▪ Relevance depends on who’s searching ▪ Difficult to scale Training Data: Human Label
  26. 26. Training Data: Click Stream Approach: Clicked = Relevant, Skipped = Not Relevant User eye scan direction Unfair penalized
  27. 27. Training Data: Click Stream Approach: Graded relevance Uncertain (middle level) Non-relevant Relevant
  28. 28. Feature Overview ▪ Textual features ▪ Social features ▪ Homophily features – Geo – Industry ▪ Inferred Searcher Interests ▪ etc.
  29. 29. Inferred Searcher Interests Interests * Locations * Industry ...
  30. 30. Learning Algorithm ▪ Coordinate Ascent Algorithm – Listwise approach ▪ Objective function: Normalized Discounted Cumulative Gain (NDCG) – Defined on graded relevance – Intuition: more useful to show more-relevant documents at higher positions
  31. 31. LinkedIn Search - Bird’s eye view 31 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
  32. 32. 32 Federated Search Page
  33. 33. ▪ Why do we need this? – Not to overwhelm the user with too much information –Make results personally relevant 33 Motivation
  34. 34. ▪ Challenges –Query can be ambiguous –Incomparability across vertical objects ▪Compare objects of different nature: individual job vs. people cluster ▪Objects associate with different signals 34 Motivation
  35. 35. 35 Overall Approach
  36. 36. Learning Federation Model ▪ Predicts: p(click| individual result OR vertical cluster, query, searcher) ▪ Training data: click logs ▪ Features –Relevance scores from base rankers –Searcher intent –Query intent –etc.
  37. 37. Features ▪ Searcher Intents – Mine searcher profiles and past behavior to infer intent ▪ Title recruiter -> recruiting intent ▪ Search for jobs -> job seeking intent – Machine-learned models predict member intents: ▪Job seeking ▪Recruiting ▪Content consuming 37
  38. 38. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions 38
  39. 39. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) 39
  40. 40. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) –Individual searcher → searcher group ▪p(job vertical| “software engineer”, job seeking searcher) 40
  41. 41. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable 41
  42. 42. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable ▪ Construct composite features People relevance score of searcher if result is People f 1= ⎨0, otherwise 42
  43. 43. Calibrate Signals across Verticals ▪ Verticals associate with different signals 43 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
  44. 44. Calibrate Signals across Verticals ▪ Verticals associate with different signals 44 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
  45. 45. Calibrate Signals across Verticals ▪ Verticals associate with different signals 45 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
  46. 46. Conclusions ▪ Search personalization is at the core of our economic graph vision –Connect talent with opportunity at massive scale ▪ Click data is useful sources for personalized training data –Need to correct position bias ▪ Personalized features are keys ▪ Create composite features to calibrate across verticals
  47. 47. 47 We are hiring!

×