SlideShare a Scribd company logo
1 of 37
Lecture 03-04

PROGRAM EFFICIENCY &
COMPLEXITY ANALYSIS
Department of Computer Science
Islamia College Univerisity Peshawar

1

Fall 2012 Semester
BCS course: CS 00 Analysis of Algorithms
Course Instructor: Mr. Zahid
RUNNING TIME OF AN
ALGORITHM


Depends upon



Input Size
Nature of Input



Generally time grows with size of input, so running time of an
algorithm is usually measured as function of input size.



Running time is measured in terms of number of
steps/primitive operations performed



Independent from machine, OS
2
FINDING RUNNING TIME OF AN
ALGORITHM / ANALYZING AN
ALGORITHM


Running time is measured by number of steps/primitive
operations performed



Steps means elementary operation like
 ,+,



*,<, =, A[i] etc

We will measure number of steps taken in term of size of
input

3
SIMPLE EXAMPLE (1)
// Input: int A[N], array of N integers
// Output: Sum of all numbers in array A
int Sum(int A[], int N)
{
int s=0;
for (int i=0; i< N; i++)
s = s + A[i];
return s;
}
How should we analyse this?
4
SIMPLE EXAMPLE (2)
// Input: int A[N], array of N integers
// Output: Sum of all numbers in array A
int Sum(int A[], int N){
int s=0;
1
for (int i=0; i< N; i++)

2
5

s = s + A[i];

return s;
}

6
8

3

7

4
1,2,8: Once
3,4,5,6,7: Once per each iteration
of for loop, N iteration
Total: 5N + 3
The complexity function of the
algorithm is : f(N) = 5N +3
5
SIMPLE EXAMPLE (3) GROWTH
OF 5N+3
Estimated running time for different values of N:
N = 10
N = 100
N = 1,000
N = 1,000,000

=> 53 steps
=> 503 steps
=> 5003 steps
=> 5,000,003 steps

As N grows, the number of steps grow in linear proportion to N for
this function “Sum”

6
WHAT DOMINATES IN PREVIOUS
EXAMPLE?
What about the +3 and 5 in 5N+3?
As N gets large, the +3 becomes insignificant
 5 is inaccurate, as different operations require varying amounts of time and
also does not have any significant importance


What is fundamental is that the time is linear in N.
Asymptotic Complexity: As N gets large, concentrate on the
highest order term:
Drop lower order terms such as +3
 Drop the constant coefficient of the highest order term i.e. N


7
ASYMPTOTIC COMPLEXITY


The 5N+3 time bound is said to "grow asymptotically"
like N



This gives us an approximation of the complexity of the
algorithm



Ignores lots of (machine dependent) details, concentrate
on the bigger picture

8
COMPARING FUNCTIONS:
ASYMPTOTIC NOTATION


Big Oh Notation: Upper bound



Omega Notation: Lower bound



Theta Notation: Tighter bound

9
BIG OH NOTATION [1]
If f(N) and g(N) are two complexity functions, we say
f(N) = O(g(N))
(read "f(N) is order g(N)", or "f(N) is big-O of g(N)")
if there are constants c and N0 such that for N > N0,
f(N) ≤ c * g(N)
for all sufficiently large N.
10
BIG OH NOTATION [2]


O(f(n)) is a set of functions.



n = O(n2) means that function n belongs to the set of
functions O(n2)

11
O(F(N))

12
EXAMPLE (2): COMPARING
FUNCTIONS


Which function is better?

10 n2 Vs n3

4000
3500
3000
2500
10 n^2

2000

n^3

1500
1000
500
0
1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

13
COMPARING FUNCTIONS
As inputs get larger, any algorithm of a smaller order
will be more efficient than an algorithm of a larger order
0.05 N2 = O(N2)
Time (steps)



3N = O(N)

N = 60

Input (size)
14
BIG-OH NOTATION


Even though it is correct to say “7n - 3 is O(n3)”, a better
statement is “7n - 3 is O(n)”, that is, one should make the
approximation as tight as possible



Simple Rule:
Drop lower order terms and constant factors
7n-3 is O(n)
8n2log n + 5n2 + n is O(n2log n)

15
BIG OMEGA NOTATION


If we wanted to say “running time is at least…” we use Ω



Big Omega notation, Ω, is used to express the lower bounds on a
function.



If f(n) and g(n) are two complexity functions then we can say:

f(n) is Ω(g(n)) if there exist positive numbers c and n0 such that 0<=f(n)>=cΩ (n) for all n>=n0

16
BIG THETA NOTATION


If we wish to express tight bounds we use the theta notation, Θ



f(n) = Θ(g(n)) means that f(n) = O(g(n)) and f(n) = Ω(g(n))

17
WHAT DOES THIS ALL MEAN?


If f(n) = Θ(g(n)) we say that f(n) and g(n) grow at the same
rate, asymptotically



If f(n) = O(g(n)) and f(n) ≠ Ω(g(n)), then we say that f(n) is
asymptotically slower growing than g(n).



If f(n) = Ω(g(n)) and f(n) ≠ O(g(n)), then we say that f(n) is
asymptotically faster growing than g(n).

18
WHICH NOTATION DO WE USE?


To express the efficiency of our algorithms which of the
three notations should we use?



As computer scientist we generally like to express our
algorithms as big O since we would like to know the
upper bounds of our algorithms.



Why?



If we know the worse case then we can aim to improve it
and/or avoid it.
19
PERFORMANCE CLASSIFICATION
f(n)

Classification

1

Constant: run time is fixed, and does not depend upon n. Most instructions are executed once, or
only a few times, regardless of the amount of information being processed

log n

Logarithmic: when n increases, so does run time, but much slower. Common in programs which
solve large problems by transforming them into smaller problems.

n

Linear: run time varies directly with n. Typically, a small amount of processing is done on each
element.

n log n

When n doubles, run time slightly more than doubles. Common in programs which break a problem
down into smaller sub-problems, solves them independently, then combines solutions

n2

Quadratic: when n doubles, runtime increases fourfold. Practical only for small problems; typically
the program processes all pairs of input (e.g. in a double nested loop).

n3

Cubic: when n doubles, runtime increases eightfold

2n

Exponential: when n doubles, run time squares. This is often the result of a natural, “brute force”
solution.

20
SIZE DOES MATTER[1]
What happens if we double the input size N?
N
8
16
32
64
128
256

log2N
3
4
5
6
7
8

5N
40
80
160
320
640
1280

N log2N
N2
24
64
64
256
160
1024
384
4096
896
16384
2048
65536

2N
256
65536
~109
~1019
~1038
~1076
21
) s p et s( e m T
i

COMPLEXITY CLASSES

22
SIZE DOES MATTER[2]


Suppose a program has run time O(n!) and the run time for
n = 10 is 1 second
For n = 12, the run time is 2 minutes
For n = 14, the run time is 6 hours
For n = 16, the run time is 2 months
For n = 18, the run time is 50 years
For n = 20, the run time is 200 centuries
23
STANDARD ANALYSIS
TECHNIQUES


Constant time statements

 Analyzing

Loops

 Analyzing

Nested Loops

 Analyzing

Sequence of Statements

 Analyzing

Conditional Statements
24
CONSTANT TIME STATEMENTS


Simplest case: O(1) time statements



Assignment statements of simple data types
int x = y;



Arithmetic operations:
x = 5 * y + 4 - z;



Array referencing:
A[j] = 5;



Array assignment:
∀ j, A[j] = 5;



Most conditional tests:
if (x < 12) ...

25
ANALYZING LOOPS[1]


Any loop has two parts:
 How

many iterations are performed?
 How many steps per iteration?
int sum = 0,j;
for (j=0; j < N; j++)
sum = sum +j;
 Loop

executes N times (0..N-1)
 4 = O(1) steps per iteration


Total time is N * O(1) = O(N*1) = O(N)

26
ANALYZING LOOPS[2]


What about this for loop?
int sum =0, j;
for (j=0; j < 100; j++)
sum = sum +j;



Loop executes 100 times



4 = O(1) steps per iteration



Total time is 100 * O(1) = O(100 * 1) = O(100) = O(1)
27
ANALYZING LOOPS – LINEAR
LOOPS


Example (have a look at this code segment):



Efficiency is proportional to the number of iterations.
Efficiency time function is :
f(n) = 1 + (n-1) + c*(n-1) +( n-1)
= (c+2)*(n-1) + 1
= (c+2)n – (c+2) +1
Asymptotically, efficiency is : O(n)





28
ANALYZING NESTED LOOPS[1]



Treat just like a single loop and evaluate each level of nesting as
needed:
int j,k;
for (j=0; j<N; j++)
for (k=N; k>0; k--)
sum += k+j;



Start with outer loop:
 How

many iterations? N
 How much time per iteration? Need to evaluate inner loop


Inner loop uses O(N) time



Total time is N * O(N) = O(N*N) = O(N2)

29
ANALYZING NESTED LOOPS[2]


What if the number of iterations of one loop depends on the
counter of the other?
int j,k;
for (j=0; j < N; j++)
for (k=0; k < j; k++)
sum += k+j;



Analyze inner and outer loop together:

Number of iterations of the inner loop is:

0 + 1 + 2 + ... + (N-1) = O(N2)


30
HOW DID WE GET THIS
ANSWER?


When doing Big-O analysis, we sometimes have to compute a
series like: 1 + 2 + 3 + ... + (n-1) + n



i.e. Sum of first n numbers. What is the complexity of this?



Gauss figured out that the sum of the first n numbers is always:

31
SEQUENCE OF STATEMENTS


For a sequence of statements, compute their complexity
functions individually and add them up



Total cost is O(n2) + O(n) +O(1) = O(n2)
32
CONDITIONAL STATEMENTS


What about conditional statements such as
if (condition)
statement1;
else
statement2;



where statement1 runs in O(n) time and statement2 runs in O(n2)
time?



We use "worst case" complexity: among all inputs of size n, what is
the maximum running time?
33



The analysis for the example above is O(n2)
DERIVING A RECURRENCE
EQUATION


So far, all algorithms that we have been analyzing have been non
recursive



Example : Recursive power method



If N = 1, then running time T(N) is 2



However if N ≥ 2, then running time T(N) is the cost of each step taken plus time
required to compute power(x,n-1). (i.e. T(N) = 2+T(N-1) for N ≥ 2)
34



How do we solve this? One way is to use the iteration method.
ITERATION METHOD


This is sometimes known as “Back Substituting”.



Involves expanding the recurrence in order to see a pattern.



Solving formula from previous example using the iteration method
:



Solution : Expand and apply to itself :
Let T(1) = n0 = 2
T(N) = 2 + T(N-1)
= 2 + 2 + T(N-2)
= 2 + 2 + 2 + T(N-3)
= 2 + 2 + 2 + ……+ 2 + T(1)
= 2N + 2 remember that T(1) = n0 = 2 for N = 1
35



So T(N) = 2N+2 is O(N) for last example.
SUMMARY


Algorithms can be classified according to their
complexity => O-Notation
 only



relevant for large input sizes

"Measurements" are machine independent
 worst-,

average-, best-case analysis

36
REFERENCES
Introduction to Algorithms by Thomas H. Cormen
Chapter 3 (Growth of Functions)

37

More Related Content

What's hot

THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION
THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION
THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION Alvin Obligacion
 
Pigeonhole Principle
Pigeonhole PrinciplePigeonhole Principle
Pigeonhole Principlenielsoli
 
AtCoder Beginner Contest 026 解説
AtCoder Beginner Contest 026 解説AtCoder Beginner Contest 026 解説
AtCoder Beginner Contest 026 解説AtCoder Inc.
 
SINGLE SOURCE SHORTEST PATH.ppt
SINGLE SOURCE SHORTEST PATH.pptSINGLE SOURCE SHORTEST PATH.ppt
SINGLE SOURCE SHORTEST PATH.pptshanthishyam
 
02. chapter 3 lexical analysis
02. chapter 3   lexical analysis02. chapter 3   lexical analysis
02. chapter 3 lexical analysisraosir123
 
AtCoder Beginner Contest 034 解説
AtCoder Beginner Contest 034 解説AtCoder Beginner Contest 034 解説
AtCoder Beginner Contest 034 解説AtCoder Inc.
 
Probability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdfProbability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdfVedant Srivastava
 
asymptotic notations i
asymptotic notations iasymptotic notations i
asymptotic notations iAli mahmood
 
Greedy algorithm activity selection fractional
Greedy algorithm activity selection fractionalGreedy algorithm activity selection fractional
Greedy algorithm activity selection fractionalAmit Kumar Rathi
 
08 Machine Learning Maximum Aposteriori
08 Machine Learning Maximum Aposteriori08 Machine Learning Maximum Aposteriori
08 Machine Learning Maximum AposterioriAndres Mendez-Vazquez
 
Asymptotic Notation and Complexity
Asymptotic Notation and ComplexityAsymptotic Notation and Complexity
Asymptotic Notation and ComplexityRajandeep Gill
 
Complexity analysis - The Big O Notation
Complexity analysis - The Big O NotationComplexity analysis - The Big O Notation
Complexity analysis - The Big O NotationJawad Khan
 
Analysis of algorithn class 2
Analysis of algorithn class 2Analysis of algorithn class 2
Analysis of algorithn class 2Kumar
 
Deterministic context free grammars &non-deterministic
Deterministic context free grammars &non-deterministicDeterministic context free grammars &non-deterministic
Deterministic context free grammars &non-deterministicLeyo Stephen
 

What's hot (20)

THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION
THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION
THE PIGEON HOLE PRINCIPLE or also known as DRAWER PRINCIPLE BY ALVIN OBLIGACION
 
Pigeonhole Principle
Pigeonhole PrinciplePigeonhole Principle
Pigeonhole Principle
 
Ch08
Ch08Ch08
Ch08
 
AtCoder Beginner Contest 026 解説
AtCoder Beginner Contest 026 解説AtCoder Beginner Contest 026 解説
AtCoder Beginner Contest 026 解説
 
SINGLE SOURCE SHORTEST PATH.ppt
SINGLE SOURCE SHORTEST PATH.pptSINGLE SOURCE SHORTEST PATH.ppt
SINGLE SOURCE SHORTEST PATH.ppt
 
Binomial queues
Binomial queuesBinomial queues
Binomial queues
 
02. chapter 3 lexical analysis
02. chapter 3   lexical analysis02. chapter 3   lexical analysis
02. chapter 3 lexical analysis
 
AtCoder Beginner Contest 034 解説
AtCoder Beginner Contest 034 解説AtCoder Beginner Contest 034 解説
AtCoder Beginner Contest 034 解説
 
Normalization
NormalizationNormalization
Normalization
 
Probability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdfProbability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdf
 
asymptotic notations i
asymptotic notations iasymptotic notations i
asymptotic notations i
 
Greedy algorithm activity selection fractional
Greedy algorithm activity selection fractionalGreedy algorithm activity selection fractional
Greedy algorithm activity selection fractional
 
Asymptotic notation
Asymptotic notationAsymptotic notation
Asymptotic notation
 
Generating function
Generating functionGenerating function
Generating function
 
Recursion DM
Recursion DMRecursion DM
Recursion DM
 
08 Machine Learning Maximum Aposteriori
08 Machine Learning Maximum Aposteriori08 Machine Learning Maximum Aposteriori
08 Machine Learning Maximum Aposteriori
 
Asymptotic Notation and Complexity
Asymptotic Notation and ComplexityAsymptotic Notation and Complexity
Asymptotic Notation and Complexity
 
Complexity analysis - The Big O Notation
Complexity analysis - The Big O NotationComplexity analysis - The Big O Notation
Complexity analysis - The Big O Notation
 
Analysis of algorithn class 2
Analysis of algorithn class 2Analysis of algorithn class 2
Analysis of algorithn class 2
 
Deterministic context free grammars &non-deterministic
Deterministic context free grammars &non-deterministicDeterministic context free grammars &non-deterministic
Deterministic context free grammars &non-deterministic
 

Viewers also liked

Aad introduction
Aad introductionAad introduction
Aad introductionMr SMAK
 
Data Structure Assignment help , Data Structure Online tutors
Data Structure Assignment help , Data Structure Online tutorsData Structure Assignment help , Data Structure Online tutors
Data Structure Assignment help , Data Structure Online tutorsjohn mayer
 
Mapping of one model into other model
Mapping of one model into other modelMapping of one model into other model
Mapping of one model into other modelratikaagarwal
 
1 d 2d 3d 4d
1 d 2d 3d 4d1 d 2d 3d 4d
1 d 2d 3d 4ddr.f
 
Learning C++ - Pointers in c++ 2
Learning C++ - Pointers in c++ 2Learning C++ - Pointers in c++ 2
Learning C++ - Pointers in c++ 2Ali Aminian
 
Two dimensional array
Two dimensional arrayTwo dimensional array
Two dimensional arrayRajendran
 
Big o notation
Big o notationBig o notation
Big o notationkeb97
 
Data structures and Big O notation
Data structures and Big O notationData structures and Big O notation
Data structures and Big O notationMuthiah Abbhirami
 
Theory of Automata and formal languages unit 2
Theory of Automata and formal languages unit 2Theory of Automata and formal languages unit 2
Theory of Automata and formal languages unit 2Abhimanyu Mishra
 
Introduction to fa and dfa
Introduction to fa  and dfaIntroduction to fa  and dfa
Introduction to fa and dfadeepinderbedi
 

Viewers also liked (20)

Chapter 11
Chapter 11Chapter 11
Chapter 11
 
Aad introduction
Aad introductionAad introduction
Aad introduction
 
Data Structure Assignment help , Data Structure Online tutors
Data Structure Assignment help , Data Structure Online tutorsData Structure Assignment help , Data Structure Online tutors
Data Structure Assignment help , Data Structure Online tutors
 
Fp201 unit4
Fp201 unit4Fp201 unit4
Fp201 unit4
 
2-D array
2-D array2-D array
2-D array
 
Mapping of one model into other model
Mapping of one model into other modelMapping of one model into other model
Mapping of one model into other model
 
Pf presntation
Pf presntationPf presntation
Pf presntation
 
1 d 2d 3d 4d
1 d 2d 3d 4d1 d 2d 3d 4d
1 d 2d 3d 4d
 
Two Dimentional Array
Two Dimentional ArrayTwo Dimentional Array
Two Dimentional Array
 
Learning C++ - Pointers in c++ 2
Learning C++ - Pointers in c++ 2Learning C++ - Pointers in c++ 2
Learning C++ - Pointers in c++ 2
 
Two dimensional array
Two dimensional arrayTwo dimensional array
Two dimensional array
 
Big o notation
Big o notationBig o notation
Big o notation
 
Data structures and Big O notation
Data structures and Big O notationData structures and Big O notation
Data structures and Big O notation
 
Theory of Automata and formal languages unit 2
Theory of Automata and formal languages unit 2Theory of Automata and formal languages unit 2
Theory of Automata and formal languages unit 2
 
Pointers in c++
Pointers in c++Pointers in c++
Pointers in c++
 
Time complexity
Time complexityTime complexity
Time complexity
 
Pointer in c++ part1
Pointer in c++ part1Pointer in c++ part1
Pointer in c++ part1
 
Introduction to fa and dfa
Introduction to fa  and dfaIntroduction to fa  and dfa
Introduction to fa and dfa
 
C++ Pointers
C++ PointersC++ Pointers
C++ Pointers
 
Unit 6 pointers
Unit 6   pointersUnit 6   pointers
Unit 6 pointers
 

Similar to Lec03 04-time complexity

Algorithm And analysis Lecture 03& 04-time complexity.
 Algorithm And analysis Lecture 03& 04-time complexity. Algorithm And analysis Lecture 03& 04-time complexity.
Algorithm And analysis Lecture 03& 04-time complexity.Tariq Khan
 
Unit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdfUnit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdfAmayJaiswal4
 
Basic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV SyllabusBasic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV SyllabusNANDINI SHARMA
 
Data Structure & Algorithms - Mathematical
Data Structure & Algorithms - MathematicalData Structure & Algorithms - Mathematical
Data Structure & Algorithms - Mathematicalbabuk110
 
Analysis Of Algorithms I
Analysis Of Algorithms IAnalysis Of Algorithms I
Analysis Of Algorithms ISri Prasanna
 
Data Structures- Part2 analysis tools
Data Structures- Part2 analysis toolsData Structures- Part2 analysis tools
Data Structures- Part2 analysis toolsAbdullah Al-hazmy
 
Daa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithmsDaa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithmssnehajiyani
 
DAA-Unit1.pptx
DAA-Unit1.pptxDAA-Unit1.pptx
DAA-Unit1.pptxNishaS88
 
04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptx04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptxarslanzaheer14
 
Asymptotic Notations
Asymptotic NotationsAsymptotic Notations
Asymptotic NotationsNagendraK18
 
Asymptotic notation
Asymptotic notationAsymptotic notation
Asymptotic notationsajinis3
 
Chapter One.pdf
Chapter One.pdfChapter One.pdf
Chapter One.pdfabay golla
 
01 - DAA - PPT.pptx
01 - DAA - PPT.pptx01 - DAA - PPT.pptx
01 - DAA - PPT.pptxKokilaK25
 
Computational Complexity.pptx
Computational Complexity.pptxComputational Complexity.pptx
Computational Complexity.pptxEnosSalar
 
lecture 1
lecture 1lecture 1
lecture 1sajinsc
 
Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Vai Jayanthi
 
Introduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic NotationIntroduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic NotationAmrinder Arora
 

Similar to Lec03 04-time complexity (20)

Algorithm And analysis Lecture 03& 04-time complexity.
 Algorithm And analysis Lecture 03& 04-time complexity. Algorithm And analysis Lecture 03& 04-time complexity.
Algorithm And analysis Lecture 03& 04-time complexity.
 
Time complexity.ppt
Time complexity.pptTime complexity.ppt
Time complexity.ppt
 
Unit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdfUnit-1 DAA_Notes.pdf
Unit-1 DAA_Notes.pdf
 
Basic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV SyllabusBasic Computer Engineering Unit II as per RGPV Syllabus
Basic Computer Engineering Unit II as per RGPV Syllabus
 
Data Structure & Algorithms - Mathematical
Data Structure & Algorithms - MathematicalData Structure & Algorithms - Mathematical
Data Structure & Algorithms - Mathematical
 
Analysis Of Algorithms I
Analysis Of Algorithms IAnalysis Of Algorithms I
Analysis Of Algorithms I
 
Data Structures- Part2 analysis tools
Data Structures- Part2 analysis toolsData Structures- Part2 analysis tools
Data Structures- Part2 analysis tools
 
Daa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithmsDaa unit 6_efficiency of algorithms
Daa unit 6_efficiency of algorithms
 
DAA-Unit1.pptx
DAA-Unit1.pptxDAA-Unit1.pptx
DAA-Unit1.pptx
 
04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptx04. Growth_Rate_AND_Asymptotic Notations_.pptx
04. Growth_Rate_AND_Asymptotic Notations_.pptx
 
analysis.ppt
analysis.pptanalysis.ppt
analysis.ppt
 
Asymptotic Notations
Asymptotic NotationsAsymptotic Notations
Asymptotic Notations
 
Lec1
Lec1Lec1
Lec1
 
Asymptotic notation
Asymptotic notationAsymptotic notation
Asymptotic notation
 
Chapter One.pdf
Chapter One.pdfChapter One.pdf
Chapter One.pdf
 
01 - DAA - PPT.pptx
01 - DAA - PPT.pptx01 - DAA - PPT.pptx
01 - DAA - PPT.pptx
 
Computational Complexity.pptx
Computational Complexity.pptxComputational Complexity.pptx
Computational Complexity.pptx
 
lecture 1
lecture 1lecture 1
lecture 1
 
Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1Cs6402 daa-2 marks set 1
Cs6402 daa-2 marks set 1
 
Introduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic NotationIntroduction to Algorithms and Asymptotic Notation
Introduction to Algorithms and Asymptotic Notation
 

Recently uploaded

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024SynarionITSolutions
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesBoston Institute of Analytics
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024The Digital Insurer
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 

Recently uploaded (20)

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 

Lec03 04-time complexity

  • 1. Lecture 03-04 PROGRAM EFFICIENCY & COMPLEXITY ANALYSIS Department of Computer Science Islamia College Univerisity Peshawar 1 Fall 2012 Semester BCS course: CS 00 Analysis of Algorithms Course Instructor: Mr. Zahid
  • 2. RUNNING TIME OF AN ALGORITHM  Depends upon   Input Size Nature of Input  Generally time grows with size of input, so running time of an algorithm is usually measured as function of input size.  Running time is measured in terms of number of steps/primitive operations performed  Independent from machine, OS 2
  • 3. FINDING RUNNING TIME OF AN ALGORITHM / ANALYZING AN ALGORITHM  Running time is measured by number of steps/primitive operations performed  Steps means elementary operation like  ,+,  *,<, =, A[i] etc We will measure number of steps taken in term of size of input 3
  • 4. SIMPLE EXAMPLE (1) // Input: int A[N], array of N integers // Output: Sum of all numbers in array A int Sum(int A[], int N) { int s=0; for (int i=0; i< N; i++) s = s + A[i]; return s; } How should we analyse this? 4
  • 5. SIMPLE EXAMPLE (2) // Input: int A[N], array of N integers // Output: Sum of all numbers in array A int Sum(int A[], int N){ int s=0; 1 for (int i=0; i< N; i++) 2 5 s = s + A[i]; return s; } 6 8 3 7 4 1,2,8: Once 3,4,5,6,7: Once per each iteration of for loop, N iteration Total: 5N + 3 The complexity function of the algorithm is : f(N) = 5N +3 5
  • 6. SIMPLE EXAMPLE (3) GROWTH OF 5N+3 Estimated running time for different values of N: N = 10 N = 100 N = 1,000 N = 1,000,000 => 53 steps => 503 steps => 5003 steps => 5,000,003 steps As N grows, the number of steps grow in linear proportion to N for this function “Sum” 6
  • 7. WHAT DOMINATES IN PREVIOUS EXAMPLE? What about the +3 and 5 in 5N+3? As N gets large, the +3 becomes insignificant  5 is inaccurate, as different operations require varying amounts of time and also does not have any significant importance  What is fundamental is that the time is linear in N. Asymptotic Complexity: As N gets large, concentrate on the highest order term: Drop lower order terms such as +3  Drop the constant coefficient of the highest order term i.e. N  7
  • 8. ASYMPTOTIC COMPLEXITY  The 5N+3 time bound is said to "grow asymptotically" like N  This gives us an approximation of the complexity of the algorithm  Ignores lots of (machine dependent) details, concentrate on the bigger picture 8
  • 9. COMPARING FUNCTIONS: ASYMPTOTIC NOTATION  Big Oh Notation: Upper bound  Omega Notation: Lower bound  Theta Notation: Tighter bound 9
  • 10. BIG OH NOTATION [1] If f(N) and g(N) are two complexity functions, we say f(N) = O(g(N)) (read "f(N) is order g(N)", or "f(N) is big-O of g(N)") if there are constants c and N0 such that for N > N0, f(N) ≤ c * g(N) for all sufficiently large N. 10
  • 11. BIG OH NOTATION [2]  O(f(n)) is a set of functions.  n = O(n2) means that function n belongs to the set of functions O(n2) 11
  • 13. EXAMPLE (2): COMPARING FUNCTIONS  Which function is better? 10 n2 Vs n3 4000 3500 3000 2500 10 n^2 2000 n^3 1500 1000 500 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 13
  • 14. COMPARING FUNCTIONS As inputs get larger, any algorithm of a smaller order will be more efficient than an algorithm of a larger order 0.05 N2 = O(N2) Time (steps)  3N = O(N) N = 60 Input (size) 14
  • 15. BIG-OH NOTATION  Even though it is correct to say “7n - 3 is O(n3)”, a better statement is “7n - 3 is O(n)”, that is, one should make the approximation as tight as possible  Simple Rule: Drop lower order terms and constant factors 7n-3 is O(n) 8n2log n + 5n2 + n is O(n2log n) 15
  • 16. BIG OMEGA NOTATION  If we wanted to say “running time is at least…” we use Ω  Big Omega notation, Ω, is used to express the lower bounds on a function.  If f(n) and g(n) are two complexity functions then we can say: f(n) is Ω(g(n)) if there exist positive numbers c and n0 such that 0<=f(n)>=cΩ (n) for all n>=n0 16
  • 17. BIG THETA NOTATION  If we wish to express tight bounds we use the theta notation, Θ  f(n) = Θ(g(n)) means that f(n) = O(g(n)) and f(n) = Ω(g(n)) 17
  • 18. WHAT DOES THIS ALL MEAN?  If f(n) = Θ(g(n)) we say that f(n) and g(n) grow at the same rate, asymptotically  If f(n) = O(g(n)) and f(n) ≠ Ω(g(n)), then we say that f(n) is asymptotically slower growing than g(n).  If f(n) = Ω(g(n)) and f(n) ≠ O(g(n)), then we say that f(n) is asymptotically faster growing than g(n). 18
  • 19. WHICH NOTATION DO WE USE?  To express the efficiency of our algorithms which of the three notations should we use?  As computer scientist we generally like to express our algorithms as big O since we would like to know the upper bounds of our algorithms.  Why?  If we know the worse case then we can aim to improve it and/or avoid it. 19
  • 20. PERFORMANCE CLASSIFICATION f(n) Classification 1 Constant: run time is fixed, and does not depend upon n. Most instructions are executed once, or only a few times, regardless of the amount of information being processed log n Logarithmic: when n increases, so does run time, but much slower. Common in programs which solve large problems by transforming them into smaller problems. n Linear: run time varies directly with n. Typically, a small amount of processing is done on each element. n log n When n doubles, run time slightly more than doubles. Common in programs which break a problem down into smaller sub-problems, solves them independently, then combines solutions n2 Quadratic: when n doubles, runtime increases fourfold. Practical only for small problems; typically the program processes all pairs of input (e.g. in a double nested loop). n3 Cubic: when n doubles, runtime increases eightfold 2n Exponential: when n doubles, run time squares. This is often the result of a natural, “brute force” solution. 20
  • 21. SIZE DOES MATTER[1] What happens if we double the input size N? N 8 16 32 64 128 256 log2N 3 4 5 6 7 8 5N 40 80 160 320 640 1280 N log2N N2 24 64 64 256 160 1024 384 4096 896 16384 2048 65536 2N 256 65536 ~109 ~1019 ~1038 ~1076 21
  • 22. ) s p et s( e m T i COMPLEXITY CLASSES 22
  • 23. SIZE DOES MATTER[2]  Suppose a program has run time O(n!) and the run time for n = 10 is 1 second For n = 12, the run time is 2 minutes For n = 14, the run time is 6 hours For n = 16, the run time is 2 months For n = 18, the run time is 50 years For n = 20, the run time is 200 centuries 23
  • 24. STANDARD ANALYSIS TECHNIQUES  Constant time statements  Analyzing Loops  Analyzing Nested Loops  Analyzing Sequence of Statements  Analyzing Conditional Statements 24
  • 25. CONSTANT TIME STATEMENTS  Simplest case: O(1) time statements  Assignment statements of simple data types int x = y;  Arithmetic operations: x = 5 * y + 4 - z;  Array referencing: A[j] = 5;  Array assignment: ∀ j, A[j] = 5;  Most conditional tests: if (x < 12) ... 25
  • 26. ANALYZING LOOPS[1]  Any loop has two parts:  How many iterations are performed?  How many steps per iteration? int sum = 0,j; for (j=0; j < N; j++) sum = sum +j;  Loop executes N times (0..N-1)  4 = O(1) steps per iteration  Total time is N * O(1) = O(N*1) = O(N) 26
  • 27. ANALYZING LOOPS[2]  What about this for loop? int sum =0, j; for (j=0; j < 100; j++) sum = sum +j;  Loop executes 100 times  4 = O(1) steps per iteration  Total time is 100 * O(1) = O(100 * 1) = O(100) = O(1) 27
  • 28. ANALYZING LOOPS – LINEAR LOOPS  Example (have a look at this code segment):  Efficiency is proportional to the number of iterations. Efficiency time function is : f(n) = 1 + (n-1) + c*(n-1) +( n-1) = (c+2)*(n-1) + 1 = (c+2)n – (c+2) +1 Asymptotically, efficiency is : O(n)   28
  • 29. ANALYZING NESTED LOOPS[1]  Treat just like a single loop and evaluate each level of nesting as needed: int j,k; for (j=0; j<N; j++) for (k=N; k>0; k--) sum += k+j;  Start with outer loop:  How many iterations? N  How much time per iteration? Need to evaluate inner loop  Inner loop uses O(N) time  Total time is N * O(N) = O(N*N) = O(N2) 29
  • 30. ANALYZING NESTED LOOPS[2]  What if the number of iterations of one loop depends on the counter of the other? int j,k; for (j=0; j < N; j++) for (k=0; k < j; k++) sum += k+j;  Analyze inner and outer loop together: Number of iterations of the inner loop is:  0 + 1 + 2 + ... + (N-1) = O(N2)  30
  • 31. HOW DID WE GET THIS ANSWER?  When doing Big-O analysis, we sometimes have to compute a series like: 1 + 2 + 3 + ... + (n-1) + n  i.e. Sum of first n numbers. What is the complexity of this?  Gauss figured out that the sum of the first n numbers is always: 31
  • 32. SEQUENCE OF STATEMENTS  For a sequence of statements, compute their complexity functions individually and add them up  Total cost is O(n2) + O(n) +O(1) = O(n2) 32
  • 33. CONDITIONAL STATEMENTS  What about conditional statements such as if (condition) statement1; else statement2;  where statement1 runs in O(n) time and statement2 runs in O(n2) time?  We use "worst case" complexity: among all inputs of size n, what is the maximum running time? 33  The analysis for the example above is O(n2)
  • 34. DERIVING A RECURRENCE EQUATION  So far, all algorithms that we have been analyzing have been non recursive  Example : Recursive power method  If N = 1, then running time T(N) is 2  However if N ≥ 2, then running time T(N) is the cost of each step taken plus time required to compute power(x,n-1). (i.e. T(N) = 2+T(N-1) for N ≥ 2) 34  How do we solve this? One way is to use the iteration method.
  • 35. ITERATION METHOD  This is sometimes known as “Back Substituting”.  Involves expanding the recurrence in order to see a pattern.  Solving formula from previous example using the iteration method :  Solution : Expand and apply to itself : Let T(1) = n0 = 2 T(N) = 2 + T(N-1) = 2 + 2 + T(N-2) = 2 + 2 + 2 + T(N-3) = 2 + 2 + 2 + ……+ 2 + T(1) = 2N + 2 remember that T(1) = n0 = 2 for N = 1 35  So T(N) = 2N+2 is O(N) for last example.
  • 36. SUMMARY  Algorithms can be classified according to their complexity => O-Notation  only  relevant for large input sizes "Measurements" are machine independent  worst-, average-, best-case analysis 36
  • 37. REFERENCES Introduction to Algorithms by Thomas H. Cormen Chapter 3 (Growth of Functions) 37