Growth of Functions CMSC 56 | Discrete Mathematical Structure for Computer Science October 6, 2018 Instructor: Allyn Joy D. Calcaben College of Arts & Sciences University of the Philippines Visayas

1. Growth of Functions
Lecture 8, CMSC 56
Allyn Joy D. Calcaben

2. Big – O Notation

3. Used to measure how well a computer algorithm scales as the
amount of data involved increases.
Big – O Notation

4. Used to measure how well a computer algorithm scales as the
amount of data involved increases.
Not a measure of speed but a measure of how well an
algorithm scales.
Big – O Notation

5. Big – O Notation is also known as Landau Notation
Named after Edmund Landau
He didn’t invent the notation, he popularized it
Ironic that he popularized Big – O as he was known to be an
exact and meticulous mathematician
Landau Notation

6. If n = 1: 45n3 + 20n2 + 19 =
Example

7. If n = 1: 45n3 + 20n2 + 19 = 84
Example

8. If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 =
Example

9. If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 = 459
Example

10. If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 = 459
If n = 10: 45n3 + 20n2 + 19 =
Example

11. If n = 1: 45n3 + 20n2 + 19 = 84
If n = 2: 45n3 + 20n2 + 19 = 459
If n = 10: 45n3 + 20n2 + 19 = 47, 019
45,000 + 2,000 + 19
Example

12. If n = 100: 45n3 + 20n2 + 19
Removing Unimportant Things

13. If n = 100: 45n3 + 20n2 + 19
= 45, 000, 000 + 200, 000 + 19
Removing Unimportant Things

14. If n = 100: 45n3 + 20n2 + 19
= 45, 000, 000 + 200, 000 + 19
= 45, 200, 019
Removing Unimportant Things

15. If n = 100: 45n3 + 20n2 + 19
= 45, 000, 000 + 200, 000 + 19
= 45, 200, 019
= O(N3)
Removing Unimportant Things

16. “Leaving away all the unnecessary things that we don’t care
about when comparing algorithms.”
Big – O Notation

17. 2n2 + 23 vs. 2n2 + 27
Example

18. 2n2 + 23 vs. 2n2 + 27
Essentially they have the same running time.
This is another simplification.
Solution

19. Definition 1
Given A = f(n), B = g(n) grows faster than f(n), then
there exists a number n’ and a constant c so that c•g(n’) ≥ f(n’)
for all n’’ > n, we have g(n’’) ≥ f(n’’).
Big – O Notation

20. Definition 1
Given A = f(n), B = g(n) grows faster than f(n), then
there exists a number n’ and a constant c so that c•g(n’) ≥ f(n’)
for all n’’ > n, we have g(n’’) ≥ f(n’’).
Since we don’t want to care about constants, we include c so that we can
scale g(n). That is, even if we multiply g(n) with a constant c, and it still
outgrows f(n), we can still say that g(n) runs at least as fast as f(n).
Big – O Notation

21. If the two conditions hold true, we say that f(n) ϵ O(g(n))
f(n) is contained in Big O of g(n)
Big O means that g(n) is a function that runs at least as fast as f(n).
Big – O Notation

22. f(n)
g(n)h(n)
n
Comparing the Growth Rates

23. Get ¼ and answer. Which of the ff. is true? Write True or False
for each number.
1. h(n) ϵ O(f(n))
2. h(n) ϵ O(g(n))
3. g(n) ϵ O(f(n))
4. g(n) ϵ O(h(n))
5. f(n) ϵ O(g(n))
6. f(n) ϵ O(h(n))
Comparing the Growth Rates
f(n)
g(n)h(n)
n

24. Get ¼ and answer. Which of the ff. is true? Write True or False
for each number.
1. h(n) ϵ O(f(n))
2. h(n) ϵ O(g(n))
3. g(n) ϵ O(f(n))
4. g(n) ϵ O(h(n))
5. f(n) ϵ O(g(n))
6. f(n) ϵ O(h(n))
Comparing the Growth Rates
f(n)
g(n)h(n)
n

25. Allow us to concentrate on the fastest growing part of the
function and leave out the involved constants
Importance

26. Simply about looking at which part of the function grows the
fastest.
A convenient way of describing the growth of the function
while ignoring the distracting and unnecessary details
Big – O Notation

27. f(n) = O(g(n)) is also an accepted notation
f(n) ϵ O(g(n)) is more accurate, since O(g(n)) is a set of functions.
Big – O Notation

28. Algorithm A RT: 3n2 – n + 10
Algorithm B RT: 2n – 50n + 256
Which algorithm is preferable in general?
Exercise

29. Algorithm A RT: 3n2 – n + 10
Algorithm B RT: 2n – 50n + 256
Which algorithm is preferable in general?
But, this is for general cases.
For small inputs, algorithm B might be equal to algorithm A or
better.
Solution

30. Algorithm A RT: 3n2 – n + 10
Algorithm B RT: 2n – 50n + 256
If n = 5: 3n2 – n + 10 = 80
2n – 50n + 256 = 38
If n = 100: 3n2 – n + 10 = 29910
2n – 50n + 256 = 1267650600228229401496703200632
Analysis

31. 1. 3n + 1 ϵ _______
2. 18n2 – 50 ϵ _______
3. 9n4 + 18 ϵ _______
4. 30n6 + 2n + 123 ϵ _______
5. 2nn2 + 30n6 + 123 ϵ _______
More Example

32. 1. 3n + 1 ϵ O(n)
2. 18n2 – 50 ϵ O(n2)
3. 9n4 + 18 ϵ O(n4)
4. 30n6 + 2n + 123 ϵ O(2n)
5. 2nn2 + 30n6 + 123 ϵ O(2n n2)
More Example

33. O(1)
O(log N)
O(N)
O(N log N)
O(N2)
O(N3)
O(2n)
Big – O Notation

34. 3n + 1 ϵ O(n2)
True / False?
More Examples

35. 3n + 1 ϵ O(n2)
True / False? TRUE, but NOT TIGHT
More Examples

36. 18n2 – 50 ϵ O(n3)
True / False?
If true, tightly bound / not?
If not tightly bound, what is the tighter bound?
More Examples

37. 18n2 – 50 ϵ O(n3)
True / False? TRUE
If true, tightly bound / not? NOT TIGHTLY BOUND
If not tightly bound, what is the tighter bound? O(n2)
More Examples

38. Write Correct / Incorrect, Tight / Not Tight
1. 4n2 - 300n + 12 ∈ O(n2)
2. 4n2 - 300n + 12 ∈ O(n3)
3. 3n + 5n2 - 3n ∈ O(n2)
4. 3n + 5n2 - 3n ∈ O(4n)
5. 3n + 5n2 - 3n ∈ O(3n)
6. 50•2nn2 + 5n - log(n) ∈ O(2n)
On Your Seats (1/4)

39. Write Correct / Incorrect, Tight / Not Tight
1. 4n2 - 300n + 12 ∈ O(n2) CORRECT, TIGHT
2. 4n2 - 300n + 12 ∈ O(n3) CORRECT, NOT TIGHT
3. 3n + 5n2 - 3n ∈ O(n2) INCORRECT, NOT TIGHT
4. 3n + 5n2 - 3n ∈ O(4n) CORRECT, NOT TIGHT
5. 3n + 5n2 - 3n ∈ O(3n) CORRECT, TIGHT
6. 50•2nn2 + 5n - log(n) ∈ O(2n) INCORRECT, NOT TIGHT
On Your Seats (1/4)

40. O(1)
O(log N)
O(N)
O(N log N)
O(N2)
O(N3)
O(2n)
Big – O Notation

41. O(1)

42. Constant
Description: Statement
Example: Adding two numbers
c = a + b
O(1)

43. Algorithm that will execute at the same amount of time
regardless of the amount of data.
Or simply,
Code that will execute at the same amount of time no matter
how big the array is.
O(1)

44. Example: Adding element to array
list.append(x)
O(1)

45. O(log N)

46. Logarithmic
Description: Divide in Half
Data is decreased roughly 50% each time through the algorithm
O(log N)

47. Example: Binary Search
O(log N)

48. O(N)

49. Linear
Description: Loop
Time to complete will grow in direct proportion to the amount of
data.
Example: Factorial of N using Loops
for i in range (1, N+1):
factorial *= i
O(N)

50. Example: Finding the Maximum
O(N)

51. Example: Finding the Maximum
Look in exactly each item in the array
Big difference if it was a 10 – item array vs. a 10
thousand – item array
O(N)

52. Example: Linear Search
O(N)

53. O(n log N)

54. Linearithmic / Loglinear
Description: Divide and Conquer
O(n log N)

55. Example: Merge Sort
O(n log N)

56. Example: Merge Sort
O(n log N)

57. O(N2)

58. Quadratic
Description: Double Loop
Time to complete will grow proportional to the square of amount
of data
O(N2)

59. Example: Bubble Sort
O(N2)

60. Example: Insertion Sort
O(N2)

61. O(N3)

62. Cubic
Description: Triple Loop
O(N3)

63. O(2n)

64. Exponential
Description: Exhaustive Search (Brute Force*)
O(2n)

65. Implications

66. Given an algorithm that runs in ___ time, the computer can solve a
problem size of _____ in a matter of minutes.
Practical Implications

67. Given an algorithm that runs in ___ time, the computer can solve a
problem size of _____ in a matter of minutes.
Constant: any
Logarithmic: any
Linear: billions
Loglinear: hundreds of millions
Quadratic: tens of thousands
Cubic: thousands
Exponential: 30
Practical Implications

68. Best Case, Worst Case

69. count = 0
for each character in string:
if character == ‘a’:
count += 1
Best Case, Worst Case

70. Best case: 2n + 1
Worst case: 3n
Running Time

71. We observe the ff. from the algorithm:
1. Each character in the string is considered at most once
2. For each character in the input string, a constant no. of steps
is performed (2 or 3)
Running Time

72. We observe the ff. from the algorithm:
1. Each character in the string is considered at most once
2. For each character in the input string, a constant no. of steps
is performed (2 or 3)
With Big – O, we say c1n + c2 ϵ O(n)
We say that the running time of the algorithm is O(n) or linear
time
Running Time

73. Any Questions?