On this website people get all information of schools

1. “Monte Carlo method”
generate M random permutations of {1,…,20}
let H be the number of those in which
exactly one person gets his/her card back
output H/M

2. Lower bounds
number from {1,2,3,…,9}
3 yes/no questions
Can you always figure out the number?

3. Lower bounds
number from {1,2,3,…,8}
3 yes/no questions
Can you always figure out the number?

4. Lower bounds
number from {1,2,3,…,8}
3 yes/no questions
in {1,2,3,4} ?
Y N
in {1,2} ? in {5,6} ?
Y
Y N
N
=1 ? =3 ? =5 ? =7 ?
Y Y Y Y
N N
N N
1 2 3 4 5 6 7 8

5. Lower bounds
number from {1,2,3,…,n}
k yes/no questions

6. Lower bounds
number from {1,2,3,…,n}
k yes/no questions
k = é log2 n ù

7. Lower bounds
number from {1,2,3,…,n}
k yes/no questions
k = é log2 n ù
INFORMATION THEORETIC LOWER BOUND:
k³ log2 n

8. Lower bounds
n animals = {dog,cat,fish,eagle,snake, …}
yes/no questions
INFORMATION THEORETIC LOWER BOUND:
k³ log2 n

9. Lower bounds for sorting
sorting by comparisons
yes/no questions: is A[i]>A[j] ?
data are not used to “control”
computation in any other way
A[1..n]
1 2 3
1 3 2
2 1 3
• 3 1
3 1 2
3 2 1

10. Lower bounds for sorting
sorting by comparisons
yes/no questions: is A[i]>A[j] ?
A[1..n]
log a*b = log a + log b
k ³ log2 n! ³
log2 n + log2 (n-1) + … log2 1 ³
(n/2) log2 (n/2) = Q(n log n)

11. Lower bounds for sorting
Theorem:
Any comparison based sorting algorithm
requires W(n ln n) comparisons in the
worst-case.

12. Lower bounds for search in sorted array
INPUT: array A[1..n], element x
OUTPUT: a position of x in A if x is in A
^ otherwise

13. Lower bounds for search in sorted array
INPUT: array A[1..n], element x
OUTPUT: a position of x in A if x is in A
^ otherwise
Theorem:
Any comparison based algorithm for
searching an element in a sorted array
requires W(ln n) comparisons in the
worst-case.

14. Lower bounds for minimum
INPUT: array A[1..n]
OUTPUT: the smallest element of A

15. Lower bounds for minimum
INPUT: array A[1..n]
OUTPUT: the smallest element of A
INFORMATION THEORETIC LOWER BOUND:
at least W(ln n) comparisons
ADVERSARY LOWER BOUND:
at least W(n) comparisons

16. Counting sort
array A[1..n] containing numbers from {1,…,k}
1st pass:
count how many times iÎ {1,…,k} occurs
2nd pass:
put the elements in B

17. Counting sort
array A[1..n] containing numbers from {1,…,k}
for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=0
for I ¬ 1 to k+1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
D[A[j]]++
B[ D[A[j]] ] ¬ A[j]

18. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
1 2 3 2 2 4 2 1 1 2 3 3 2 1 2 4
C 4 7 3 2
D 1 5 12 15

19. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
1 2 3 2 2 4 2 1 1 2 3 3 2 1 2 4
C 4 7 3 2
D 2 5 12 15
1

20. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
1 2 3 2 2 4 2 1 1 2 3 3 2 1 2 4
C 4 7 3 2
D 2 6 12 15
1 2

21. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
1 2 3 2 2 4 2 1 1 2 3 3 2 1 2 4
C 4 7 3 2
D 2 6 13 15
1 2 3

22. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
1 2 3 2 2 4 2 1 1 2 3 3 2 1 2 4
C 4 7 3 2
D 2 7 13 15
1 2 2 3

23. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
1 2 3 2 2 4 2 1 1 2 3 3 2 1 2 4
C 4 7 3 2
D 5 12 15 17
1 1 1 1 2 2 2 2 2 2 2 3 3 3 4 4

24. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
stable sort = after sorting the items with the
same key don’t switch order
running time = O(n+k)

25. Counting sort for i ¬ 1 to k do C[i] ¬ 0
for j ¬ 1 to n do C[A[j]]++
D[1]=1
for I ¬ 1 to k-1 do D[i+1] ¬ D[i]+C[i]
for j ¬ 1 to n do
B[ D[A[j]] ] ¬ A[j]
D[A[j]]++
stable sort = after sorting the items with the
same key don’t switch order
running time = O(n+k)
What if e.g., k = n2 ?

26. Radix sort
array A[1..n] containing numbers from {0,…,k2 - 1}
1) sort using counting sort with
key = x mod k
2) sort using counting sort with
key = ë x/k û
Running time = ?

27. Radix sort
array A[1..n] containing numbers from {0,…,k2 - 1}
1) sort using counting sort with
key = x mod k
2) sort using counting sort with
key = ë x/k û
Running time = O(n + k)

28. Radix sort
array A[1..n] containing numbers from {0,…,k2 - 1}
example k=10
28
70
21
20
42
21
43
42
23
32
32
43
70
23
18
28
29
18
20
29

29. Radix sort
array A[1..n] containing numbers from {0,…,k2 - 1}
example k=10
28
70
21
20
42
21
43
42
23
32
32
43
70
23
18
28
29
18
20
29

30. Radix sort
array A[1..n] containing numbers from {0,…,k2 - 1}
example k=10
28
70
21
20
42
21
43
42
23
32
32
43
70
23
18
28
29
18
20
29
18
20
21
23
28
29
32
42
43
70

31. Radix sort
array A[1..n] containing numbers from {0,…,kd - 1}
1) sort using counting sort with
key = x mod k
2) sort using counting sort with
key = ë x/k û mod k
3) sort using counting sort with
key = ë x/k2 û mod k
…
d) sort using counting sort with
key = ë x/kd-1 û mod k

32. Radix sort
array A[1..n] containing numbers from {0,…,kd - 1}
Correctness: after s-th step the numbers
are sorted according to x mod ks
Proof: By induction. Base case s=1 is trivial.
1) sort using counting sort with
key = x mod k

33. Radix sort
array A[1..n] containing numbers from {0,…,kd - 1}
Correctness: after s-th step the numbers
are sorted according to x mod ks
Proof: Now assume IH and execute s+1st step. Let x,y
be such that x mod ks+1 < y mod ks+1. Then either
ë x/ks û mod k < ë y/ks û mod k or
ë x/ks û mod k = ë y/ks û mod k and x mod ks < y mod ks

34. Bucket sort
linear time sorting algorithm on average
Assume some distribution on input.
INPUT: n independently random numbers from the
uniform distribution on the interval [0,1].

35. Bucket sort
INPUT: n independently random numbers from the
uniform distribution on the interval [0,1].
for i ¬ 1 to n do
insert A[i] into list B[ ëA[i]*nû ]
for i ¬ 0 to n-1 do
sort list B[i]
output lists B[0],…,B[n-1]

36. Bucket sort
INPUT: n independently random numbers from the
uniform distribution on the interval [0,1].
0.13, 0.23, 0.56, 0.74, 0.18, 0.34, 0.12, 0.82, 0.14, 0.19
for i ¬ 1 to n do
insert A[i] into list B[ ëA[i]*nû ]
for i ¬ 0 to n-1 do
sort list B[i]
output lists B[0],…,B[n-1]

37. Bucket sort
INPUT: n independently random numbers from the
uniform distribution on the interval [0,1].
0.13, 0.23, 0.56, 0.74, 0.18, 0.34, 0.12, 0.52, 0.14, 0.19
1 2 5 7 1 3 1 5 1 1
B[1]: 0.13, 0.18, 0.12, 0.14, 0.19
B[2]: 0.23
B[3]: 0.34
B[5]: 0.56, 0.52
B[7]: 0.74
for i ¬ 1 to n do
insert A[i] into list B[ ëA[i]*nû ]
for i ¬ 0 to n-1 do
sort list B[i]
output lists B[0],…,B[n-1]

38. Bucket sort
INPUT: n independently random numbers from the
uniform distribution on the interval [0,1].
0.13, 0.23, 0.56, 0.74, 0.18, 0.34, 0.12, 0.52, 0.14, 0.19
1 2 5 7 1 3 1 5 1 1
B[1]: 0.12, 0.13, 0.14, 0.18, 0.19
B[2]: 0.23
B[3]: 0.34
B[5]: 0.52, 0.56
B[7]: 0.74
for i ¬ 1 to n do
insert A[i] into list B[ ëA[i]*nû ]
for i ¬ 0 to n-1 do
sort list B[i]
output lists B[0],…,B[n-1]

39. Bucket sort for i ¬ 1 to n do
insert A[i] into list B[ ëA[i]*nû ]
for i ¬ 0 to n-1 do
sort list B[i]
output lists B[0],…,B[n-1]
assume we use insert-sort
worst-case running time?

40. Bucket sort for i ¬ 1 to n do
insert A[i] into list B[ ëA[i]*nû ]
for i ¬ 0 to n-1 do
sort list B[i]
output lists B[0],…,B[n-1]
assume we use insert-sort
average-case running time?
X0, X1, … , Xn-1 where Xi is the number of
items that fall inside the i-th bucket

41. Bucket sort
X0, X1, … , Xn-1 where Xi is the number of
items that fall inside the i-th bucket
X0
2 + X1
2 + … + Xn-1
2
What is E[X0
2 + X1
2 + … + Xn-1
2] ?
E[X0
2 + … + Xn-1
2 ] =
E[X0
2] + … + E[Xn-1
2 ] =
n E[X0
2]
linearity of expectation
symmetry of the problem

42. Bucket sort
E[X0
2] What is E[X0] ?
value of X0
0 (1-p)n
1 n (1-p) n-1
k binomial(n,k) pk (1-p)n-k
n pn
p=1/n

43. Bucket sort
E[X0
2] E[X0] = 1
0 (1-p)n
1 n (1-p) n-1
k binomial(n,k) pk (1-p)n-k
n pn
p=1/n
n
E[X0] = å k * binomial(n,k) pk (1-p)n-k
k=0

44. Bucket sort
E[X0
2] E[X0] = 1
p=1/n
n
E[X0] = å k * binomial(n,k) pk (1-p)n-k
k=1
binomial (n,k) = (n/k) * binomial (n-1,k-1)
n
å binomial(n,k) pk (1-p)n-k = 1
k=0

45. Bucket sort
E[X0
2] E[X0] = 1
p=1/n
n
E[X0] = å n* binomial(n-1,k-1) pk (1-p)n-k
k=1
n
n
= n*p å binomial(n-1,k-1) pk-1(1-p)n-k
= n*p
k=1
å binomial(n,k) pk (1-p)n-k = 1
k=0

46. Bucket sort
E[X0
2]
0 (1-p)n
1 n (1-p) n-1
k binomial(n,k) pk (1-p)n-k
n pn
p=1/n
2]= å k2 * binomial(n,k) pk (1-p)n-k
k=0
E[X0
n

47. Bucket sort
E[X0
2]
p=1/n
E[X0
n
2]= å k2 * binomial(n,k) pk (1-p)n-k
k=0
n
= å k * binomial(n,k) pk (1-p)n-k
k=0
+
n
å k(k-1) * binomial(n,k) pk (1-p)n-k
k=0

48. Bucket sort
E[X0
2]
p=1/n
E[X0
n
2]= å k2 * binomial(n,k) pk (1-p)n-k
k=0
n
= å k * binomial(n,k) pk (1-p)n-k
k=0
+
n
n*p
å k(k-1) * binomial(n,k) pk (1-p)n-k
k=0

49. Bucket sort
E[X0
2]
p=1/n
n
2]= å k(k-1) * binomial(n,k) pk (1-p)n-k
E[X0
+n*p
k=2
binomial (n,k) = (n/k) * binomial (n-1,k-1) =
(n/k) * ((n-1)/(k-1)) * binomial (n-2,k-2)
E[X0
2] = n* (n-1) * p2 + n*p