1. Bounded Rationality:
Thinking Is Costly
A baseball and bat together cost $11.
The bat costs $10 more than the ball.
How much does the ball cost?
Write down your answer.
Half of Harvard students said $1, which is the intuitive
answer but wrong!
Correct answer is 50 cents: $10.50-$.50 = $10.00
People tend to use “intuitive thinking” or rules of thumb

2. CONCURRENT SCHEDULES OF
REINFORCEMENT
Study of choice centers on
how operant behavior is
affected its reinforcement
history and by reinforcement
history of other operant
behaviors.
 Research uses concurrent schedules:
two or more schedules that operate
simultaneously and independently,
each for a different response.

3. Concurrent schedules
of reinforcement
Two schedules are in effect at the same time and the
subject is free to switch from one response alternative
to the other
Key A Key B
Schedule A
VI 60 s
Schedule B
FR 10

4. Choice Behavior
and the
Matching Law
 Relative rate of responding on a particular lever
equals the relative rate of reinforcement on that lever
The Matching Law is a mathematical statement describing
the relationship between the rate of responding and the
rate of reward
 developed by Herrstein

5. Matching Law
RA/(RA + RB) = rA/(rA + rB) or
RA /rA = RB /rB
RA = Responses to A
RB = Responses to B
rA = Reinforcers to A
rB = Reinforcers to B

6. Matching Law
Provides accurate description of behavior
of many organisms in many different
choice situations.
Different species
Appetitive and aversive stimuli
Frequency, magnitude, and delay of
reinforcement

7. People Aren’t Always Rational
Studies find that people make systematic
mistakes:
 People are overconfident.
 People give too much weight to a small number of
vivid observations.
 People are reluctant to change their minds.
Even though people are not always rational,
the assumption that they are is usually
a good approximation for economic
modeling.

8. Deviations from Matching

9. Impatience: the desire for instant
gratification
Read, Loewenstein & Kalyanaraman (1999)
Choose among 24 movie videos
Some are “low brow”: My Cousin Vinny
Some are “high brow”: Schindler’s List
 Picking for tonight: 56% of subjects choose low brow.
 Picking for next Thursday: 37% choose low brow.
 Picking for second Thursday: 29% choose low brow.
Tonight I want sugar-coated
entertainment… next week I

10. Choice with Commitment
In a standard concurrent schedule of reinforcement, two
(or more) response alternatives are available at the same
time and the subject is free to switch from one to the other
at any time
However, in some (real-life) situations, choosing one
alternative makes other alternatives unavailable
In these cases, the choice may involve assessing complex,
long-range goals
Can study these types of situations in the lab using a
Concurrent-chain schedule of reinforcment

11. Terminal link
BA
Time
Choice link
Reinforcement
schedule A
(VR 10)
Reinforcement
schedule B
(FR 10)
Pecking the left key in the choice link puts into effect reinforcement
schedule A in the terminal link. Pecking the right key in the choice
link puts into effect reinforcement schedule B in the terminal link.
Concurrent-chain schedule

13. BA
Time
Small
reward
Large
reward
Delay
A B
Small
reward
Large
reward
Delay
Direct-choice procedure
Pigeon chooses immediate,
small reward
Concurrent-chain procedure
Pigeon chooses the schedule
with the delayed, larger
reward

14. Self-Control
Concurrent chain schedules have been used to study
‘self-control’ in the lab
e.g., choosing a large delayed reward over an
immediate small reward
With direct choice procedures, animals often lack
self-control. That is, they choose the immediate, but
smaller reward
With concurrent-chain procedures, animals do show
self-control. That is, they choose the larger, but delayed
reward

15. Choosing fruit vs. chocolate
Time
Choosing Today Eating Next Week
If you were
deciding today,
would you choose
fruit or chocolate
for next week?

16. Patient choices for the future:
Time
Choosing Today Eating Next Week
Today, subjects
typically choose
fruit for next week.
74%
choose
fruit

17. Impatient choices for today:
Time
Choosing and Eating
Simultaneously
If you were
deciding today,
would you choose
fruit or chocolate
for today?

18. Time Inconsistent Preferences:
Time
Choosing and Eating
Simultaneously
70%
choose
chocolate

19. Rachlin and Green (1972)
If you delay the reinforcer for both a larger and smaller reinforcer and then
you give organism a choice, it may well choose the larger delayed
reinforcer
10 second delay
10 second delay
2 seconds food
immediate
4 seconds food
Must wait 4
seconds
4 seconds food
Must wait 4
seconds
FR - 15
FR - 15

20. Self-Control
 Why does anyone choose a
smaller reward part of the time?
 Animals and people typically choose a
small immediate reward over a larger
delayed reward.
 Large rewards are selected when:
 The choice is made in advance of
reward.
 Reinforcers are not visible or reward
is already present (pleasurable
activity).

22. What mechanism may account for
this short temporal horizon?
 Two Brain System
Theory of Addiction
 Increased activity of the
Impulsive System
(motivational)
 Extented Amygdala
 Ventral Striatum
 Decreased activity of
Executive System
(rationality)
 Ventromedial Prefrontal
Cortex (VMPC)
 Dorsolaterial Prefrontal
Cortex (DLPC)

23. Self-Control
 Temporal Issue
 Lack of self-control arises from the fact that our behavior
is more heavily influenced by immediate consequences
as opposed to delayed consequences.
Immediate Consequence Delayed Consequence
quitting withdrawal Improved health
smoking Nicotine high Deterioration of health
Self-control – preference for
larger later reward
Impulsiveness – preference for
smaller sooner reward

24. Which do you prefer
$500 now or $1,000 in two years
$500 in four years or $2,000 in six years

25. People Are Inconsistent Over Time
People tend to prefer instant gratification,
even when delaying would increase the
gratification.
Result: People fail to follow through on
plans to do things that are dreary, take
effort, or cause discomfort.
 e.g., people often save less than they plan
To help follow through, people look for
ways to commit themselves to their plans.
 e.g., worker has money taken out of paycheck
before he ever sees it

26. Choice and Foraging
Laboratory paradigms
are often criticized
because they do not
capture essence of
natural contingencies.
 More recent research has
tried to bring natural
contingencies of
reinforcement into laboratory.
 This move has been prompted
by optimal foraging theory.

27. Choice and Foraging
 Optimal foraging theory:
feeding behavior is sensitive
to relation between amount
of energy expended in
finding, securing, and
consuming food, and amount
of energy or nutrition of food.
 Patterns of foraging optimize
relation between energy gain
and energy expenditure.
Optimal foraging theory:
feeding behavior is sensitive to
relation between amount of
energy expended in finding,
securing, and consuming food,
and amount of energy or
nutrition of food.
Patterns of foraging optimize
relation between energy gain
and energy expenditure

28. Behavioral Economics
 Behavioral economics: a new field in which
economists apply basic insights from psychology
 People aren’t always as rational as traditional
economic models assume.
 Herbert Simon viewed humans as satisficers,
people who make choices that are merely
“good enough” rather than optimal.
 Other economists have suggested that people
are only “near rational” or exhibit “bounded
rationality.”

29. OPERANT BEHAVIOR AND
ECONOMICS
It is not surprising that behavior theorists
have applied economic concepts to their
own domain.
Matching Law is viewed by some theorists
as special case of general economic
principles.

30. OPERANT BEHAVIOR AND
ECONOMICS: BASICS
Demand (amount purchased at given
price):
 Elastic (luxuries)
 Inelastic (necessities)
Commodities:
 Substitutes (more of one, less of other)
 Complements (more of one, more of other)
Income (money or responses required to
purchase commodities at given price)

32. OPERANT BEHAVIOR AND
ECONOMICS: BASICS
Demand (amount purchased at given
price):
 Elastic (luxuries)
 Inelastic (necessities)
Commodities:
 Substitutes (more of one, less of other)
 Complements (more of one, more of other)
Income (money or responses required to
purchase commodities at given price)
Rodents = Bar press

33. OPERANT BEHAVIOR AND
ECONOMICS: BASICS
Economy (relationship between
commodities and income:
 Closed (fixed income)
 Open (extra income): most operant studies
Deprivation

34. People Care About Fairness
People’s choices are sometimes
influenced more by their sense of
fairness than self-interest.
Example: the ultimatum game
The rules
 Two players who do not know each
other
have a chance to share a prize of $100.
 Player A decides what portion of the
prize to give to player B.
 B must accept the split or both get
nothing.

35. People Care About Fairness
Predicted outcome if both players rational
 A would propose a 99-1 split and B would accept,
because $1 is better than nothing.
Actual outcome from experiments with real
people
 B usually rejects lopsided splits like 99-1
as wildly unfair.
 Expecting this, A usually proposes giving
$30 or $40 to B.
 B views this as unfair, but not so much as to
abandon his self-interest, so B accepts.

36. People Care About Fairness
The results of the ultimatum game apply in
other situations.
Example:
A firm may pay above-equilibrium wages
during profitable years to be fair, or to
avoid appearing unfair and risking
retaliation from workers.

37. OPERANT BEHAVIOR AND
ECONOMICS
 Evidence suggests behavior of animals in
conditioning experiments conforms to what
economic theory says people will do when
confronted with similar choices.
 Implies that operant behavior involves a kind of
economic decision making.
 Animals must decide how to allocate scarce
behavioral resources; rules by which they do so
are like those people use.
 Economic theory and behavior theory may jointly
explain human and animal choice.