1. Guessing the Unknown

2. The Quest
What can we
say about this
black box?
E.g., What is the
probability that it
generates a number
5 12 3 9 28
bigger than 5?
Observations

3. Distributions
What if we had many many
observations?
Value Frequency
-1 0.3
0 0.2
Sum of
1 0.1 frequencies
This table is the 2 0.1 is 1
distribution 3 0.1
associated with
4 0.1
this black box
5 0.1

4. Distributions Graphically
-1 0 1 2 3 4 5
Area under the
curve is 1

5. The Challenge
We do not have many many
observations
So we cannot infer the
distribution from the
observations
What can we do then?

6. What can we do with few
observations?
Assume distribution is known
E.g., Normal, Binomial
(from prior knowledge or
etc
other means)
I.e., model approximately using
a canonical distribution
But the parameters are not
known
Can these parameters be
determined from the
observations?

7. Why Canonical Distributions
Value Frequency
-1 0.3
0 0.2
1 0.1 Too verbose a
description for the
2 0.1 distribution
3 0.1
4 0.1
5 0.1
Can the entire distribution be
described (even approximately) by just
a few parameters, while modeling the
data accurately

8. Example: Binomial Distribution
A coin that yields 1 with Observations
probability p and 0 with
probability 1- p, tossed n 1 0 1 1 1 ….
times, independently
Value Frequency
Number of 1’s? 0
1
Distribution, 2
μ=np,σ2=np(1-p)
n-1
Can one determine p
from the (few) n
observations?

9. Other Canonical Distributions
Normal μ, σ2
Poisson μ =r,σ2=r
Negative Binomial μ =rp/(1-p),
σ2= rp/(1-p)2
What are
these? Later
talk
Gamma μ=kθ, σ2 =kθ2

10. Back to the Quest
We have few observations
Assume these are from a
known distribution family
But with unknown
parameters
How do we determine the
parameters?
How do we determine μ,
σ2?

11. Estimating Mean
μ, σ2
Estimate for the
mean; a good
estimate??

12. μ, σ2
What is the mean and variance
Normal!! For of this distribution?
modest n.

13. μ, σ2
Unbiased
Tight as n
grows larger

14. Estimating Variance
μ, σ2
Estimate for the
variance; a good
estimate??

15. μ, σ2
μ, σ2
Bias

16. Estimating Variance Correctly
μ, σ2
Unbiased!!

17. A Mind Reading Game
• Your friend chooses a number (one of 1,3,5) in his/her
mind
– Call this i
• He/She then rolls a 6-faced die 30 times, privately
– For each roll, he/she declares Heads if the number on the
die is <=i, and Tails otherwise
• Your goal is to guess i solely from this sequence of n
Heads and Tails.
• Can you read your friend’s mind?

18. Thank You