Teaching requirements analysis REET 2014 at RE2014
Semantic technology: The tourists’ voice comes alive.
1. Semantic technology:
The tourists’ voice comes alive
Luisa Mich
Department of Computer and Management Sciences, University of Trento, I
Filippo Nardelli
Cogito, Expert System Group, Italy
2. Schema
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 2
• The problem: info, info, info
• The role of Semantics
• Linguistic and Natural Language Processing
• Cogito’s semantic technology
• A case study: the tourists’ voice in Trentino
3. A flood of unstructured data &
information
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 3
• Internet is a (the) source of information and a
place where users meet to share their opinions
about travel and vacations
• Questions:
• How to effectively integrate social media monitoring in
the marketing mix?
• How to overcome limits of traditional linguistic
technology, in order to discover and manage relevant
online information?
5. 5
Hard to find ‘relevant’ information
5
Productivityofsearch
Amount of information
Databases
Files & Folders
Directories
Keyword Search (Google)
Tagging
Natural Language Search
Desktop
PC Era
Web Social Web
Semantic
Web
The increasing amount of information
• 15 Petabytes of new information a day
• 15 million searches a month
The diminishing effectiveness of search
• 1/3 of searches do not find intended results
• Over two hours a day are spent searching for information
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011
6. More Semantics!
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 6
A buzzword?
- artificial intelligence
- semantic web
- ontologies
- search engines
- information retrieval
- reputation analysis
- reviews
- sentiment analysis
- the customers’ voice, the tourists’ voice
7. Problems in text analysis
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 7
Same word,
different
meanings
Different words,
but the same
meanings
Different words,
related
meanings
Apple (fruit)
Apple (company)
Big Apple (city)
New York City
Big Apple
Organization à Company
Organization à Charity
Organization à Trade Union
8. Limitations of traditional approaches
Breaks text into single words
without considering the
context, like reading a
language that we don’t
understand:
Az IBM szokásosan nagy hangsúlyt
helyez a továbbképzésre, így
munkatársai évente számos szakmai
tanfolyamon vesznek részt.
Recognizes words and identifies
their most basic forms
(lemmas), but cannot
distinguish between different
meanings
Sell -> Selling -> Sold
Neither understands the meaning of words
Keyword Technology
or Statistics
Shallow Linguistic
Technology
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 8
9. Semantic Technology
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 9
Effective sentiment analysis and accurate detection of
opinions expressed online is a difficult task.
• Traditional technology and web monitoring tools are able to
find specific words (keywords), but unable to discover a
customer’s opinion
• Applied in the tourism sector, semantic technology provides
tourism practitioners with more qualified analytics
• Semantic tools take advantage of semantic processing to
understand the meaning of words and the conceptual
meaning of texts
“The sandwiches sold in the old, rundown bar near the
fountain are actually really great.”
10. • It understands the relationships
between words
Luke (subject) has eaten (verb)
a chicken (object)
• It understands the meaning of
words
To eat (chicken); to consume
(oil); to destroy (sweater); to
spend (money); to rust (the
tower), etc.
Why semantics is different
Semantic technology understands the meaning of
words in the same way you learned to read.
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 10
12. Cogito and ‘Semantic Valley’ in Trentino
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 12
13. What does ‘properly analyzed’ mean?
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 13
4 Requirements Definition Example
Morphological Analysis
understands word
forms
dog, dog-catcher and doggy-bag
are closely related
Grammatical Analysis
understands the
parts of a speech
"There are 40 rows in the table“
(noun), vs. "She rows 5 times a
week" (verb)
Logical Analysis
understands how
words relate to
other words
“Davey Jones, represented by
attorney Daniel Stanley, is
married to Rebecca Carter."
Rebecca is married to Davey, not
Daniel
Semantic Analysis
(disambiguation)
understands the
context of
keywords
"I used chicken broth for my soup
stock" uses stock in the context of
food, vs. "The company keeps lots
of stock on hand" uses stock in the
context of inventory
14. The semantic net, the heart of Cogito
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 14
Traditional technologies can only guess the meaning of
words using keywords, shallow linguistics and statistics
can identify
Instead, semantic networks
“San Jose is an American city”
“San Jose is a geographic
part of California”
Connections
Concepts
Terms
Abbr ev.
Phrases Meanings
Domains
15. 15
What is a semantic network?
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 15
A rich map of associations and meanings of words
• Includes all definitions of all words
• Includes relationships between words
The quality of results depends on the richness and
complexity of the semantic network
COGITO® English
Semantic Network:
• 350,000 words
• 2.8M relationships
16. 16
Technology stack
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 16
1. Morphology
2. Grammatical
4. Disambiguation
Develop and Add Custom Rules
3. Logic
Semantic technology,
tools and customization
services maximize the
quality and the
performance of the
solution.
Development
Studio
Semantic
Network
Semantic
Network
Linguistic
Query
Engine
Italian
German
90% Precision
Semantic
Network
Other Middle Eastern
English
Arabic
80% Precision
18. Case study: Culture and Vacation in
Trentino
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 18
Web monitoring
and Open source
intelligence for
the automatic and
real-time
semantic
sentiment
detection of the
tourists’ voice
19. Application of topic models
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 19
Cultural offer of Trentino: concepts, or drivers
20. Description of the drivers
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 20
Concepts are
described
attaching to them
a set of
characteristics, in
order to extract
and use
information on
them
21. Main results
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 21
Identification of User Generated Content (UGC)
relevant for:
• the DMO, marketing plans, target markets,
perception of the tourism offer with respect to
competitors
• tourism operators: quality of services (expected,
perceived)
22. Conclusion
Luisa Mich - Enter 2011 - Helsinborg, Jan. 25-27 2011 22
Semantic technology applied to social media
monitoring supports:
• identification of relevant concepts
• interpretation of meaning
• extraction of information (strategic decisions)
• identification of trends and ‘tipping points’ (viral
marketing waves)