SlideShare una empresa de Scribd logo
1 de 16
LEARNING THROUGH CONVERSATION:  TRANSCRIPT ANALYSIS OF L-NET’S CHAT REFERENCE SERVICE 
http://sites.google.com/site/ stacyjohnsmlscapstoneportfolio/home/featured-project
Purpose of Project 1. What types of questions are being asked? 2. How effectively are they being answered? 3. What resources are being used to answer them?
Scrubbed Transcript
Coding   Spreadsheet
Katz (1997) Classification Scheme, modified by Arnold & Kaske (2005) and Seeking Synchronicity (2007), as interpreted and used for this L-net study.
Number of Questions by Type
Categories of Chat Reference Answers Coding procedures derived from Arnold&Kaske (2005), developed/revised by Torsney & Radford for Seeking Synchronicity (2007), and further adapted for the purposes of this study.
Correct & Incorrect Answers
Direct vs. Indirect Answers
Top 10 ‘Frequently Used’ Sites
Breakdown by Domain
Types of Resources Used
Reference Interviews
Average time per chat
High variation in reaction to prank-type chats. Use of automated messages. That's an excellent question. Please hold while I check some sources.  Hello. You've connected to your 24x7 online reference service staffed by librarians across the state. Please wait one moment while I take a look at your question.  I’m working with another person right now. I’ll be with you as soon as possible. Thanks for waiting...  L-net answers for reference-type questions  94% correct!

Más contenido relacionado

Similar a C:\Fakepath\Learning Through Conversation

Unsupervised Software-Specific Morphological Forms Inference from Informal Di...
Unsupervised Software-Specific Morphological Forms Inference from Informal Di...Unsupervised Software-Specific Morphological Forms Inference from Informal Di...
Unsupervised Software-Specific Morphological Forms Inference from Informal Di...
Chunyang Chen
 
Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...
Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...
Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...
Preetha Chatterjee
 
Répondre à la question automatique avec le web
Répondre à la question automatique avec le webRépondre à la question automatique avec le web
Répondre à la question automatique avec le web
Ahmed Hammami
 
A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...
A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...
A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...
Cemal Ardil
 
A_Review_of_Question_Answering_Systems.pdf
A_Review_of_Question_Answering_Systems.pdfA_Review_of_Question_Answering_Systems.pdf
A_Review_of_Question_Answering_Systems.pdf
ssuser98a1af
 
Query recommendation papers
Query recommendation papersQuery recommendation papers
Query recommendation papers
Ashish Kulkarni
 
Rule Based Automatic Generation of Query Terms for SMS Based Retrieval Systems
Rule Based Automatic Generation of Query Terms for SMS Based Retrieval SystemsRule Based Automatic Generation of Query Terms for SMS Based Retrieval Systems
Rule Based Automatic Generation of Query Terms for SMS Based Retrieval Systems
Editor IJCATR
 

Similar a C:\Fakepath\Learning Through Conversation (20)

Techniques For Deep Query Understanding
Techniques For Deep Query UnderstandingTechniques For Deep Query Understanding
Techniques For Deep Query Understanding
 
IRJET- Semantic Question Matching
IRJET- Semantic Question MatchingIRJET- Semantic Question Matching
IRJET- Semantic Question Matching
 
Unsupervised Software-Specific Morphological Forms Inference from Informal Di...
Unsupervised Software-Specific Morphological Forms Inference from Informal Di...Unsupervised Software-Specific Morphological Forms Inference from Informal Di...
Unsupervised Software-Specific Morphological Forms Inference from Informal Di...
 
Open domain Question Answering System - Research project in NLP
Open domain  Question Answering System - Research project in NLPOpen domain  Question Answering System - Research project in NLP
Open domain Question Answering System - Research project in NLP
 
Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...
Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...
Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineer...
 
Testing natural language processing
Testing natural language processingTesting natural language processing
Testing natural language processing
 
Répondre à la question automatique avec le web
Répondre à la question automatique avec le webRépondre à la question automatique avec le web
Répondre à la question automatique avec le web
 
A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...
A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...
A black-box-approach-for-response-quality-evaluation-of-conversational-agent-...
 
A_Review_of_Question_Answering_Systems.pdf
A_Review_of_Question_Answering_Systems.pdfA_Review_of_Question_Answering_Systems.pdf
A_Review_of_Question_Answering_Systems.pdf
 
Question Retrieval in Community Question Answering via NON-Negative Matrix Fa...
Question Retrieval in Community Question Answering via NON-Negative Matrix Fa...Question Retrieval in Community Question Answering via NON-Negative Matrix Fa...
Question Retrieval in Community Question Answering via NON-Negative Matrix Fa...
 
Query recommendation papers
Query recommendation papersQuery recommendation papers
Query recommendation papers
 
Arabic MT Project
Arabic MT ProjectArabic MT Project
Arabic MT Project
 
Part-of-speech Tagging for Web Search Queries Using a Large-scale Web Corpus
Part-of-speech Tagging for Web Search Queries Using a Large-scale Web CorpusPart-of-speech Tagging for Web Search Queries Using a Large-scale Web Corpus
Part-of-speech Tagging for Web Search Queries Using a Large-scale Web Corpus
 
Frequently asked tcs technical interview questions and answers
Frequently asked tcs technical interview questions and answersFrequently asked tcs technical interview questions and answers
Frequently asked tcs technical interview questions and answers
 
Intro to sentiment analysis
Intro to sentiment analysisIntro to sentiment analysis
Intro to sentiment analysis
 
Recognizing named entities in
Recognizing named entities inRecognizing named entities in
Recognizing named entities in
 
Metaphic or the art of looking another way.
Metaphic or the art of looking another way.Metaphic or the art of looking another way.
Metaphic or the art of looking another way.
 
ABBREVIATION DICTIONARY FOR TWITTER HATE SPEECH
ABBREVIATION DICTIONARY FOR TWITTER HATE SPEECHABBREVIATION DICTIONARY FOR TWITTER HATE SPEECH
ABBREVIATION DICTIONARY FOR TWITTER HATE SPEECH
 
AI and Python: Developing a Conversational Interface using Python
AI and Python: Developing a Conversational Interface using PythonAI and Python: Developing a Conversational Interface using Python
AI and Python: Developing a Conversational Interface using Python
 
Rule Based Automatic Generation of Query Terms for SMS Based Retrieval Systems
Rule Based Automatic Generation of Query Terms for SMS Based Retrieval SystemsRule Based Automatic Generation of Query Terms for SMS Based Retrieval Systems
Rule Based Automatic Generation of Query Terms for SMS Based Retrieval Systems
 

C:\Fakepath\Learning Through Conversation

Notas del editor

  1. Introduction: name, recent MLIS graduate from SCSU, Newport Library, L-net for 1 1/2 years
  2. Special project address; switch to special project page.
  3. Read through 500 scrubbed transcripts . . .
  4. Worked from a list of randomized and anonymized transcripts
  5. Codes to categorize questions
  6. The high number of pranks includes a number of people who have misunderstood the term “Chat”, and who are looking for a social chatroom. It might be worthwhile to include information on the log-in page that explains more clearly that the service is intended for people who would like assistance in finding answers to genuine questions. It is important to be as welcoming as possible, so that people with odd or embarrassing questions will feel able to ask for help. It might be worthwhile to include something about an expectation for inoffensive language as well, although again, it must be carefully worded so as not to make anyone feel unwelcome
  7. The answer rubric was less effective, as the described categories of answers didn’t always apply well to non-reference type questions. For example, what is the ‘correct’ response to a complaint about another staffperson? In order to be consistent, all answers to non-reference type questions were considered ‘other’.
  8. Most questions are answered only with a link; Lnet staff do not find and paraphrase the answers.
  9. Wikipedia used with caveat; Google includes images, translator, maps, etc . . . LOC= library of congress. Total of 612 sites recommended for the 450 questions. The relatively low percentage of Wikipedia use, or multiple use of any site, points to a broad use of the internet.
  10. No oversight on .com and .org sites. .edu and .gov sites more authoritative
  11. Some reference interview attempts were ignored or elicited annoyed responses. Time lag also complicates attempt. Most cases where one was appropriate, one was attempted, but in nearly 25% of cases where clarification would have been helpful, no attempt was made. (56/239) it is difficult to hold a linear conversation in online chat, where both parties may be typing about different issues at the same time. Experienced staff may have become accustomed to this, and to the fact that many patrons seem most satisfied with a quick answer rather than more questions. However, reading many transcripts shows that some non-reference interview chats tend to have one or more false starts before the patron’s needs are understood.
  12. Overall average, 12.5 minutes. Numbers distorted by patrons signing off w/o librarian noticing, by multitasking, by followup time.