Micro-Scholarship, What it is, How can it help me.pdf
Techniques For Deep Query Understanding
1. Techniques for
Deep Query Understanding
“Beware of the man who knows the answer before he understands the
question”
Guided By:
Dr. Dhaval Patel,
Assistant Professor,
Department Of CSE,
IIT Roorkee.
Presented By:
Abhay Prakash,
En. No. – 10211002,
CSI, V Year,
IIT Roorkee.
2. (Source: Google)
Introduction: Query Understanding
Purpose:
To understand what exactly the user is searching for – his precise intent
To correct mistakes and guide user to formulate a precise intended query
Query Refinement
Why only this phrase in Bold?
(Source: Google) Query Suggestion
3. Emerging Variety of Queries
Natural Language Queries instead of Keyword Represented Queries
“who is the best classical singer in India” instead of “best classical singer India”
Use of NL Queries increasing (Makoto et.al in [1])
Local Search Queries
“Where can I eat cheesecake right now?”
Context Dependent Queries (Interactive Question Answering
(Source: Bing – location set as US)
4. Background: How results are generated
Query Understanding
(which index parameters to
be used)
Review: Hotel ABC, Civil Lines:
I ate cheesecake, which was really
awesome. (4/5 star)
High Level Architecture of Search Mechanism (Source: Self Made)
INDEX
(Knowledge Base)
Document Understanding
(What and how to Index)
User Query
Results Ranking
Entities: Hotel ABC, cheesecake
Location: Civil Lines
Quality: 0.8
Time: 8:15 PM
“where can I eat cheesecake
right now?”
Data
(Text documents, User
Reviews, Blogs, Tweets,
Linkedin …)
[Time: 8:15 PM]
Intent: Hotel Search
Search for: cheesecake
Location: Civil lines
Time: 8:20 PM
5. Background: QU & Adv. In Search (Weotta in [3])
1. Basic Search
Direct text match based retrieval of documents
Restrict search space using facet values provided by user
Current day example: Online shopping sites
Mechanism in Basic Search (Source: Self Made)
Example of Facets (Source: Flipkart.com)
6. Background: QU & Adv. In Search (Weotta in [3])
2. Advanced Search
Ranking of result documents based on:
TF-IDF to identify more relevant documents
Website authority and popularity
Keyword weighting
Not Considered:
Context, NLP for semantic understanding
Location of query, time of query
Example: Google as was in its early stage
7. Background: QU & Adv. In Search (Weotta in [3])
3. Deep Search
What difference does it bring?
Requirements:
Semantic Understanding of Query
Knowledge of Context, previous tasks
User Understanding and Personalization
9. Architecture: Query Understanding Module
Query
i) michael jordan berkley
ii) michael jordan NBA
Query
Suggestion
Query
Correction
Query
Expansion
i) michael jordan berkley: academic
ii) michael l. Jordan Berkley: academic
Query
Classification
Semantic
Tagging
Example of purpose of each Component (Source: Self Made)
michal jrdan
michael jordan
i) michael jordan berkley
ii) michael l. jordan berkley
i) [michael jordan: PersonName]
[berkley: Location]: academic
ii) [michael l. jordan: PersonName]
[berkley: Location]: academic
10. Query Correction
Reformulates the ill-formed (mistaken) search queries
ex. Macine learning Machine Learning
Refinements:
Spelling error, Two words Merged together, One word separated
Phrase segmentation (machine + learning machine learning)
Acronym Expansion (CSE Computer Science & Engineering)
Refinement may be mutually dependent
“lectures on machne learn”
learn is a correct term, but should have been learning
Hence, different terms need to be addressed simultaneously
11. Problem Modeled by Jiafeng et.al in [10] as
Original Query (푥 = 푥1 푥2 . . . 푥푛) Corrected Query (푦 = 푦1 푦2 . . . 푦푛)
Get y(complete sequence) which has maximum probability of occurrence, given the
sequence x.
Simple Technique
Assume terms independent take 푦푖 with max Pr 푦푖 푥푖
Prime Disadvantage:
Reality deviates a lot from assumption
Ex. “Lectures on machine learning”
Independent Corrections
Query Correction
12. Query Correction
Using Conventional CRF
What is CRF?
Probabilistic graphical model, models conditional distribution
of unobserved state sequences
Trained on given observation sequence
Trained for getting Pr 푦 푠푒푞푢푒푛푐푒 푥
Why use CRF? Conditioned on?
Sequence of words matters (learning machine?)
푦푖 conditioned on other 푦푖s as well, along with 푥푖
Corrections are mutually dependent (e.g. machine learning)
Disadvantage:
Will require very large amount of data, 푦푖 candidates’ domain open
Conventional CRF
13. Restricting space of y for the given x
conditioned 푦푖 on operation also
표 = 표1 표2 … 표푛, such that 표푖 required to get 푦푖 from 푥푖
표푖 is operation like deletion, insertion of characters, etc.
Learning and Prediction
Dataset of (푥(1), 푦(1), 표(1)), . . . , (푥(푁), 푦(푁), 표(푁))
Features
log Pr(푦푖−1|푦푖 ), where the prob. calculated using corpus
Whether 푦푖 푖푠 표푏푡푎푖푛푒푑 푓푟표푚 푥푖 푎푓푡푒푟 표푝푒푟푎푡푖표푛 표푖 --{0|1}
Basic CRF-QR Model
Query Correction
Basic CRF-QR Model (Jiafeng et. al in [10])
14. What is new?
Handles scenario with more than one refinements
Machine learm learn learning
Sequence of (sequence of operation)
표 = 표푖,1, 표푖,2, . . . 표푖,푛 i.e. multiple operations on each word
Intermediate results: 푧푖 = 푧푖,1푧푖,2 . . . 푧푖,푚−1
Extended CRF-QR
Query Correction
Extended CRF-QR Model (Jiafeng et. al in [10])
16. Context aware Query Suggestion (Huanhuan et.al in [7])
Query Suggestion Mechanism (Source: [7])
Query Suggestion
Query – mapped Concept
Concept Suffix tree from log
Suggestion time: Transition on tree with each query’s concept
Suggest top queries of that state
17. Concept Suffix Tree
Concept Discovery
Queries clustered using set of clicked URLs
Feature vector 푞푖 =
푛표푟푚 푤푖푗 푖푓 푒푑푔푒 푒푖푗 푒푥푖푠푡푠
0 표푡ℎ푒푟푤푖푠푒
Each identified cluster is taken as a Concept
Concept Suffix Tree
Vertex: state after transition through a sequence of
concepts (of queries)
Transition in a session
C2C3C1: transition Beginning C1 C3 C2
Click-Through Bipartite
Query Suggestion
Context aware Query Suggestion (Huanhuan et.al in [7])
18. Query Suggestion
Task aware Query Suggestion (Allan et.al in [9])
Why task identification Important?
Considering Off-Task query in context adversely affect quality of recommendation
30% sessions contained multiple tasks (Zhen et.al in [8])
5% sessions have Interleaved tasks (Zhen et.al in [8])
Identify similar previous queries as On-Task
consider only On-Task queries as context
Effect of On-Task and Off-Task
queries
19. Query Suggestion
Task aware Query Suggestion (Allan et.al in [9])
Measures to evaluate similarity between two queries
Lexical Score: captures similarity at word level directly. Average of:
Jaccard Coefficient between trigrams from the two queries: how many common trigrams?
(1 - Levenshtein Edit Distance), which shows closeness at word level
Semantic Score: maximum of the following two
푠푤푖푘푖푝푒푑푖푒푎(푞푖 , 푞푗 ): cosine similarity of vector of tf-idf score of Wikipedia documents w.r.t the
two queries.
푠푤푖푘푡푖표푛푎푟푦 (푞푖 , 푞푗 ): similar to above on Wiktionary entries
Final Similarity(풒풊, 풒풋) = 휶 . Lexical Score + (1-휶) . Semantic Score
If Similarity(푞푖, Reference_q) greater than threshold 푞푖 is On-Task Query
20. Query Suggestion
Personalization in Query Suggestion (Milad et.al in [2])
On character hit of ‘i’
“Instagram” more popular for female below 25
“Imdb” more popular for male in 25-44.
Candidate queries generated by prior general method
Personalization by re-ranking candidate queries
Features for feedback earlier global rank
Original position
Original score
Short History Features
3-Gram similarity with just previous query
Avg. 3-gram similarity with all previous queries in the session
21. Query Suggestion
Personalization in Query Suggestion (Source: [2])
Long History Features
No. of times candidate query issued in past
Avg. 3-gram similarity with all previous queries in the past
Demographic Features
Candidate query frequency over queries by same age group
Candidate query likelihood -- same age group
Candidate query frequency -- same gender group
Candidate query likelihood -- same gender group
Candidate query frequency -- same region group
Candidate query likelihood -- same region group
22. Query Expansion
Sending more words (should generate similar result) to tackle term-miss
Ex. “Tutorial lecture on ABC” “Video Lecture on ABC”
Expansion Tasks:
Adding synonyms of words
Morphological words by stemming
Naïve Approach
Exhaustive lookup in thesaurus
Time taking
Still miss terms of similar intent (terms even semantically far)
23. Query Expansion
Path Constrained Random Walk (Jianfeng et.al in [11])
Exploiting search logs for identifying terms having similar end result
Search log data of <Query, Document> clicks
Graph Representation
Node Q: seed query
Nodes Q’: queries in search log
Nodes D: documents
Nodes W: words that occur in queries and documents
Word nodes are the candidate expansion terms
Edges have scoring function
Represents probability of transition from start node
to end node
Search Log as Graph
24. Query Expansion
Path Constrained Random Walk (Jiafeng et.al in [11])
Probability of using w as an expansion word?
Product of probabilities in Paths starting at
node Q and ending at w
Top probable words picked, obtained from
random walk
Search Log as Graph
25. Query Classification
Classifying given query in a predefined Intent Class
Ex. michael Jordan berkley: academic
Precise intent by sequence of nodes from root to leaf
More challenging than document classification
Short length
Keyword representation, makes more ambiguous
Ex. query “brazil germany”
Older basic techniques
Example Taxonomy (Source: [6])
Considering single query statistical techniques like 2-gram/3-gram inference
26. Query Classification
Context aware Query Classification (Huanhuan et.al in [6])
Resolving ambiguity using context
Previous Queries ∈ sports, then “Michael Jordan” sports (Basketball Player)
Previous Queries ∈ academic, then “Michael Jordan” academic (ML professor)
Use of CRF (because training and prediction on sequence)
Local Features
Query Terms: Each 푞푡 supports a target category
Pseudo Feedback:
푞푡 with concept 푐푡, submitted to an external web directory
How many of top M results have 푐푡 concept?
Implicit Feedback:
Instead of Top M results – only the clicked documents taken
27. Query Classification
Context aware Query Classification (Huanhuan et.al in [6])
Contextual Features
Direct association between adjacent labels
Number of occurrences of adjacent labels < 푐푡−1, 푐푡 >
Higher weight higher probability of transit from 푐푡−1to 푐푡
Taxonomy-based association between adjacent labels
Given pair of adjacent labels < 푐푡−1, 푐푡 > at level n
n-1 features of taxonomy-based association between 푐푡−1, 푐푡 considered
e.g. Computer/Software related to Computer/Hardware, matching at (n-1)th level
Computer
28. Semantic Tagging
Identifies the semantic concepts of a word or phrase
[michael jordan: PersonName] [berkley: Location]: academic
Useful only if phrases in documents also tagged
Shallow Parsing Methods
Part of Speech Tags: e.g. Clubbing consecutive nouns for Named Entity Recognition
Disadvantage: Sentence Level Long Segments can’t be identified
29. Semantic Tagging
Hierarchical Parsing Structures
Trained a semi-Markov CRF on segments
Features
Syntactic Features
Parse tree of sentence
Plot
30. Semantic Tagging
Semantic Dependency Features
leverage the information about dependencies among different segments
Ex. “show me a funny movie starring Johnny and featuring Carribbean Pirates”
‘Featuring’ takes arguments – “funny movie” and “Carribbean Pirates”
long distance semantic dependency between the object “movie” and attribute <Plot>
31. Conclusion & Future Work
End-to-End Discussion of Query Understanding Module Tasks
Semantic Understanding of queries for intent detection has lot of scope
Use of NL (grammatically correct) queries rising
Understanding at the structure level
User community detection for its application in Query Suggestion
Based on search behavior
Community/Topic specific temporal trending of search query
32. References
[1] Makoto P. Kato, Takehiro Yamamoto, Hiroaki Ohshima and Katsumi Tanaka, "Cognitive
Search Intents Hidden Behind Queries: A User Study on Query Formulations," in WWW
Companion, Seoul, Korea, 2014.
[2] Milad Shokouhi, "Learning to Personalize Query Auto-Completion," in SIGIR, Dublin,
Ireland, 2013.
[3] Weotta, "Deep Search," 10 6 2014. [Online]. Available:
http://streamhacker.com/2014/06/10/deepsearch/. [Accessed 6 8 2014].
[4] W. Bruce Croft, Michael Bendersky, Hang Li and Gu Xu, "Query Understanding and
Representation," SIGIR Forum, vol. 44, no. 2, pp. 48-53, 2010.
[5] Jingjing Liu, Panupong Pasupat, Yining Wang, Scott Cyphers and Jim Glass, "Query
Understanding Enhanced by Hierarchical Parsing Structures," in ASRU, 2013.
[6] Huanhuan Cao, Derek Hao Hu, Dou Shen and Daxin Jiang, "Context-Aware Query
Classification," in SIGIR, Boston, Massachusetts, USA, 2009.
33. References (Continued…)
[7] Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, Hang Li,
"Context-Aware Query Suggestion by Mining Click-Through and Session Data," in
KDD, Las Vegas, Nevada, USA, 2008.
[8] Zhen Liao, Yang Song, Li-wei He and Yalou Huang, "Evaluating the Effectiveness of
Search Task Trails," in WWW, Lyon, France, 2012.
[9] Allan, Henry Feild and James, "Task-Aware Query Recommendation," in SIGIR,
Dublin, Ireland, 2013.
[10] Jiafeng Guo, Gu Xu, Hang Li and Xueqi Cheng, "A Unified and Discriminative
Model for Query Refinement,“ in SIGIR, Singapore, 2008.
[11] Jianfeng Gao, Gu Xu and Jinxi Xu, "Query Expansion Using Path-Constrained
Random Walks," in SIGIR, Dublin, Ireland, 2013.
Notas del editor
Things that happen
Corrected mistaken term
Intent and target entity is dependent on the context
Add the animated structure for deep indexing and deep querying over that
Text match
Restricted document facets
Mistaken query
Corrected suggested picked first expanded …. Segments tagged with location, person name
Reformulates by correction
Segmentation collectively have meaning learn is correct but …ing
Original sequence of terms x and have to obtain another sequence of corrected terms y, such that prob(y|x)
Models cond. Dist of unobserved sequence based on observed sequences
Learning machine mean something else
Restrict by allowed operations – insertion, deletion
So defined another sequence o, in which oi is required to get yi from xi
Extended handle scenarios multiple correction
Query mapped to a concept Learn Concept Trail from log
How mapping of query to concept
Why task id imp? For that let us see ----- on task ------ off task
Because allan zhen gave stats that ----- and allan showed that performance ……..
Lexical score ----------- semantic score
Lexical : jaccard coeff of 3gram…… closeness by leven.. edit distance
Smantic: Wikipedia ….. wiktionary
Ex by milad
Candidate by general ranked for personalize
Original ----- position score
Personal ----- short 3 g prev, 3 g with all prev
Long hist count for same 3g with all
Demographic --- identify group behaviour
Similar words
Prob by multiply all scores on path
Predefined classes
Huanhuan showed use of context to classify
Software and Hardware not same, but same parent so must be related