From the "Natural Language Processing" LinkedIn group: John Kontos, Professor of Artificial Intelligence I wonder whether translating into formal logic is nothing more than transliteration which simply isolates the part of the text that can be reasoned upon using the simple inference mechanism of formal logic. The real problem I think lies with the part of text that CANNOT be translated one the one hand and the one that changes its meaning due to civilization advances. My own proposal is to leave NL text alone and try building inference mechanisms for the UNTRANSLATED text depending on the task requirements. All the best John"

1. Semantic Analysis in Language Technology
Lecture 2: From Semantics to Semantic-Oriented Applications
Course Website: http://stp.lingfil.uu.se/~santinim/sais/sais_fall2013.htm
MARINA SANTINI
PROGRAM: COMPUTATIONAL LINGUISTICS AND LANGUAGE TECHNOLOGY
DEPT OF LINGUISTICS AND PHILOLOGY
UPPSALA UNIVERSITY, SWEDEN
14 NOV 2013

2. From Formal Systems to Natural Language Semantics
2
 The past:


Aristotelean Logic
Prepositional Logic
[huge temporal gap]


Predicate Logic (FOL & co.)
Formal Semantics
 The present:

Computational Semantics & Semantic-Oriented Applications
 The future:

Actionable Intelligence
Lecture 2: From Semantics to Applications

3. Aristotelian Logic
3
 The fundamental assumption behind the theory is that propositions are
composed of two terms – hence the name "two-term theory" or "term logic"
– and that the reasoning process is in turn built from propositions:


Aristotle distinguishes singular terms such as Socrates and general terms such as Greeks.
Aristotle further distinguishes (a) terms that could be the subject of predication, and (b)
terms that could be predicated of others by the use of the copula ("is a").
A proposition consists of two terms, in which one term (the "predicate") is
"affirmed" or "denied" of the other (the "subject"), and which is capable of
truth or falsity.


Socrates is a man
Socrates is not immortal
 The syllogism is an inference in which one proposition (the "conclusion")
follows of necessity from two others (the "premises").



Socrates is a man,
all men are mortal,
therefore Socrates is mortal = new knowledge (inferential knowledge)
Lecture 2: From Semantics to Applications

4. Syllogistic fallacies
4
 People often make mistakes when reasoning
syllogistically and mathematically with natural language:
• A=B
• B=C
• A=C



some cats (A) are black things (B),
some black things (B) are televisions (C),
it does not follow from the parameters that some cats (A) are
televisions (C).
 Existential fallacy (use of quantifiers)

The existential fallacy, or existential instantiation, is a formal
fallacy: "Everyone in the room is pretty and smart". It does not
imply that there is a pretty, smart person in the room, because it
does not state that there is a person in the room.
Lecture 2: From Semantics to Applications

5. Prepositional Logic
5
 It was developed into a formal logic by Chrysippus and
expanded by the Stoics.
 The logic was focused on propositions.
 This advancement was different from the traditional syllogistic
logic which was focused on terms.
 It represents any given proposition with a letter.
 It requires that all propositions have exactly one of two truth-
values: true or false.

To take an example, let be the proposition that it is raining outside.
This will be true if it is raining outside and false otherwise.
Lecture 2: From Semantics to Applications

6. The father of Predicate Logic
6
 In 1879 Frege published his Begriffsschrift (Concept
Script). This introduced a calculus, a method of
representing statements by the use of quantifiers
and variables.
Lecture 2: From Semantics to Applications

7. Predicate Logic (aka FOL, etc.)
7
 Predicate logic is also known as first-order predicate
calculus, the lower predicate calculus,quantification
theory, and first-order logic.
 First-order logic is a formal system used
in mathematics, philosophy, linguistics, and computer
science.
 First-order logic is distinguished from propositional
logic by its use of quantified variables.
 First-order logic is distinguished from propositional
logic by its use of quantified variables.
Lecture 2: From Semantics to Applications

8. Quantifiers
8
 The two fundamental kinds of quantification
in predicate logic are universal
quantification and existential quantification. The
traditional symbol for the universal quantifier "all" is
"∀", an inverted letter "A", and for the existential
quantifier "exists" is "∃", a rotated letter "E".
Lecture 2: From Semantics to Applications

9. Propositional Logic vs Predicate Logic
9
 A predicate takes an entity or entities in the domain of discourse as input and
outputs either True or False.
Consider the two sentences "Socrates is a philosopher" and "Plato is a philosopher".
In propositional logic, these sentences are viewed as being unrelated and are denoted, for example,
by p and q. However, the predicate "is a philosopher" occurs in both sentences which have a common
structure of "a is a philosopher". The variable a is instantiated as "Socrates" in first sentence and is
instantiated as "Plato" in the second sentence
"There exist a such that a is a philosopher" .
Predicates can be also compared.
Ex "if a is a philosopher, then a is a scholar". This formula is a conditional statement with "a is
philosopher" as hypothesis and "a is a scholar" as conclusion.
The truth of this formula depends on which object is denoted by a, and on the interpretations of the
predicates "is a philosopher" and "is a scholar".
Variables can be quantified over. "For every a, if a is a philosopher, then a is a scholar". The universal
quantifier "for every" in this sentence expresses the idea that the claim "if a is a philosopher, then a is
a scholar" holds for all choices of a.
Lecture 2: From Semantics to Applications
a

10. Formal Semantics (wikipedia)
10
 In linguistics, formal semantics seeks to understand linguistic
meaning by constructing precise mathematical models of the
principles that speakers use to define relations between
expressions in a natural language and the world which
supports meaningful discourse.
 The mathematical tools used are the confluence of formal
logic and formal language theory, especially lambda calculus.
 Linguists rarely employed formal semantics until Richard
Montague showed how English (or any natural
language) could be treated like a formal language. His
contribution to linguistic semantics, which is now known as
Montague grammar, was the basis for further developments.
Lecture 2: From Semantics to Applications

11. Translating Natural Language to Formal Language by:
11
 Lamba calculus:
 is a formal system in mathematical
logic and computer science for
expressing computation based on
function abstraction and application
via variable binding and substitution.
(Cf also J&M: 593)
 Prolog:
 Prolog is a general purpose logic
programming language associated
with artificial
ntelligence and computational
linguistics.
 Prolog has its roots in first-order logic,
a formal logic, and unlike many
other programming languages, Prolog
is declarative: the program logic is
expressed in terms of relations,
represented as facts and rules.
Lecture 2: From Semantics to Applications
Top-down rule-based systems

12. Syntax-based compositionality of meaning
12
Lecture 2: From Semantics to Applications

13. Stumbling block: meaning is not always compositional…
13
Lecture 2: From Semantics to Applications

14. Multi-Word Expressions
14
MWEs (a.k.a multiword units or MUs) are lexical
units encompassing a wide range of linguistic
phenomena, such as:






idioms (e.g. kick the bucket = to die),
collocations (e.g. cream tea = a small meal eaten in Britain,
with small cakes and tea),
regular compounds (cosmetic surgery),
graphically unstable compounds (e.g. self-contained <> self
contained <> selfcontained - all graphical variants have huge
number of hits in Google),
light verbs (e.g. do a revision vs. revise),
lexical bundles (e.g. in my opinion), etc.
Lecture 2: From Semantics to Applications

15. Stumbling Block: Ambiguity
15
 Lexical ambiguity: ex polisemy
 Ex: bank
 Referential ambiguity: ex anaphoric ambiguity
… it was funded by a tycoon
 Scopal ambiguity:
 I can’t find a piece of paper

(a particular piece of paper or any piece of paper? Existential or
universal quantifier "∀”or "∃“?)
Lecture 2: From Semantics to Applications

16. Computational Semantics (wikipedia)
16
 Computational semantics is the study of how to automate the process of
constructing and reasoning with meaning representations of natural
language expressions. It consequently plays an important role in natural
language processing and computational linguistics.
 Some traditional topics of interest are:





construction of meaning representations,
semantic underspecification,
anaphora resolution,
presupposition projection,
quantifier scope resolution.
 Methods employed usually draw from formal semantics or statistical
semantics. Computational semantics has points of contact with the areas of
lexical semantics (word sense disambiguation and semantic role labeling),
discourse semantics, knowledge representation and automated reasoning
…
Lecture 2: From Semantics to Applications

17. What is Semantics? ---- What is LT?
17
 Students’ intuition about
semantics:
1.
2.
3.
4.
5.
6.
7.
Meaning of language
(words, phrases, etc.)
Break down complex
meaning into simpler
blocks of meaning
Content understanding
Disambiguation
Understanding a phrase
Understanding the
meaning of phrases
depending on different
contexts
Meaning and
connotation
Lecture 2: From Semantics to Applications
Language technology is often
called human language
technology (HLT) or natural
language processing (NLP) and
consists of computational
linguistics (or CL) and speech
technology as its core but
includes also many application
oriented aspects of them.
Language technology is closely
connected to computer
science and general
linguistics. (wikipedia)
Must add:
•Statistics
•Machine learning

18. What shall we keep from the past?
18
 Computational semantics must be….
Lecture 2: From Semantics to Applications

19. Computational semantics must address open issues:
19



Ambiguity
Overcome compositionality
Etc.
Lecture 2: From Semantics to Applications

20. Our definition of semantics for LT must include:
20
1.
2.
3.
4.
5.
6.
7.
Meaning of language (words,
phrases, etc.)
Break down complex meaning
into simpler blocks of meaning
Content understanding
Disambiguation
Understanding a phrase
Understanding the meaning of
phrases depending on different
contexts
Meaning and connotation
Semantics for Language Technolgy
must now take also these aspects into
account.
Lecture 2: From Semantics to Applications
Continuity with the past approaches
 Must be computationally tractable
More advanced than past systems:
 Must address ambiguity
 Must address non
compositional meaning
Above all, must tackle new media.
In less than 50 years, new media (internet,
web, social networks) have completely
scrambled ”traditional” semantics and
human communication by creating :
•New meanings (sentiment, opionion, etc)
•New language (unconvetional texts and
syntax and many sublanguages, like
tweets, FB posts, etc.)
•Big amounts of wild data

21. In conclusion
21
 More than creating a ”understanding system”, currently the stress in
how to automatically extract meaningful and actionable information
depending on specifc tasks….
Lecture 2: From Semantics to Applications

22. Visual Insight into big data around us…
22
 Big Data Video: http://youtu.be/qqfeUUjAIyQ (2:21 min)
Lecture 2: From Semantics to Applications

23. New meanings: the so-called sentiment
23
 Sentiment Analysis’s purpose: detect and extract
emotions, attitudes, opnions from text… People
behaviour and choices (politics, products, reactions)
are driven by sentiment rather than ”sensibility” 
 (Sense and Sensibility by J. Austin well describe
these two opposite behaviours)
 A basic ML algorithm underlying many (but not all)
applications detecting sentiments: Daniel Jurafsky,
Coursera, NLP – Stanford University (video, 13 min)
Lecture 2: From Semantics to Applications

24. Conclusions
24
 Think about semantics, computational semantics
and big data
 Think how ML is important for semantic-oriented
applications (be proud of the many things you
learned during the previous course)
 Next time we will continue with Sentiment Analysis,
which is a semantic-oriented application…
Lecture 2: From Semantics to Applications

25. 25
This is the end… Thanks for your attention !
Lecture 2: From Semantics to Applications