Lect07

Algorithm Analysis
Searching

Dr.Haitham A. El-Ghareeb

Information Systems Department
Faculty of Computers and Information Sciences
Mansoura University
helghareeb@gmail.com

November 25, 2012

Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 1 / 55

Algorithms

l  Algorithms are designed to solve problems.
l  A problem can have multiple solutions.

How do we determine which
solution is the most efficient?

Chapter 4: Algorithm Analysis –2
© 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.


Execution Time

l  Measure execution time:
l  construct a program for a given solution.
l  execute the program.
l  time it using a “wall clock”.

l  Dependent on:
l  amount of data
l  type of hardware and time of day
l  programming language and compiler



Complexity Analysis

l  What if we examine the solution itself and
measure critical operations:
l  logical comparisons
l  assignments
l  arithmetic operations



Example Algorithm

l  Given a matrix of size n x n, compute the:
l  sum of each row of a matrix.
l  overall sum of the entire matrix.



Example Algorithm (v.1)

l  How many additions are performed?
= 2n (n) = 2n2
rowSum = Array(n)
totalSum = 0
for i in range( n ) :
rowSum[i] = 0

n for j in range( n ) :

n rowSum[i] = rowSum[i] + matrix[i,j]
2
totalSum = totalSum + matrix[i,j]



Example Algorithm (v.2)

l  How many additions are performed?
=(n + 1) n = n2 + n
rowSum = Array(n)
totalSum = 0
rowSum[i] = 0

n for j in range( n ) :
n
1 rowSum[i] = rowSum[i] + matrix[i,j]
totalSum = totalSum + matrix[i,j]
1



Compare the Results

l  Number of additions: v1: 2n2 v2: n2 + n
l  Second version has fewer additions (n > 1)
l  Will execute faster than the first.
l  Difference will not be significant.

Both algorithms execute on the
same order of magnitude, n2



Growth Rates

l  As n increases, both algorithms increase at approx
the same rate:

n 2n2 n2 + n
10 200 110
100 20,000 10,100
1000 2,000,000 1,001,000
10,000 200,000,000 100,010,000
100,000 20,000,000,000 10,000,100,000



Growth Rates



Big-O Notation

l  No need to count precise number of steps.
l  Classify algorithms by order of magnitude.
l  execution time
l  space requirements

Approximates actual number of
steps or actual storage in terms
of variable-sized data sets.



Big-O Definition

l  Given a function T(n)
l  # of steps required for an input of size n.
l  Ex: T2(n) = n2 + n

l  Suppose there exist a function f(n) for all
integers n > 0 such that
T(n) < c f(n)

for some constant c and for all large values of n
> m (a constant).


Big-O Definition

l  Then, the algorithm has a time-complexity of or
executes “on the order of” f(n)
l  We use the notation: O( f(n) )
l  Big-O is intended for large values of n.

f(n) indicates the rate of growth
at which the run time increases
as the input size increases.



Big-O Example (v.1)

l  Consider the previous sample algorithms.
l  Version 1: T1(n) = 2n2

T1(n) < c f(n) Let c = 2
2n2 < 2n2

O( n2 )



Big-O Example (v.2)

l  Consider the previous sample algorithms.
l  Version 2: T2(n) = n2 + n

T2(n) < c f(n) Let c = 2
n 2 + n < n2 + n2
n2 + n < 2n2

O( n2 )



Upper Bound

l  There is more than one f(n) for an algorithm.
n2 + n < c f(n)
l  n2 is not the only choice
l  f(n) could be n2, n3, n4

Objective: find an f(n) that
provides the tightest (lowest)
upper bound.



Constant of Proportionality

l  Is it important?
l  Consider two algorithms:
l  O(n2), with c = 1
l  O(2n), with c = 2

n n2 2n
10 100 20
100 10,000 200
1000 1,000,000 2,000
10,000 100,000,000 20,000
100,000 10,000,000,000 200,000



Constant of Proportionality



Constructing T(n)

l  We don't count total number of specific instructions
(math operations, comparisons, etc)
l  Assume each basic statement takes the same time,
constant time.
l  Total number of steps required:

T(n) = f1(n) + f2(n) + ... + fk(n)



Constructing T(n) Example

1 rowSum = Array(n)
1 totalSum = 0
1 for i in range( n ) :
1 rowSum[i] = 0

n 1 for j in range( n ) :
n 1 rowSum[i] = rowSum[i] + matrix[i,j]
1 totalSum = totalSum + matrix[i,j]



Constructing T(n) Example

l  Constant time steps are generally omitted.

rowSum = Array(n)
totalSum = 0
...
rowSum[i] = 0 ...
for j in range( n ) :
n
n rowSum[i] = rowSum[i] + matrix[i,j]
totalSum = totalSum + matrix[i,j] ...



Choosing the Function

l  Given T(n), choose the dominant term.

T(n) = n2 + log2n + 3n

n2 dominates the other terms (for n > 3)

n2 + log2n + 3n < n2 + n2 + n2
n2 + log2n + 3n < 3n2



Choosing the Function

l  What is the dominant term for the following
expression?

T(n) = 2n2 + 15n + 500

When n < 16, 500 dominates.
When n > 16, n2 is the dominate term.



Classes of Algorithms

l  Many algorithms have a time-complexity selected
from a common set of functions.

f() Common Name
1 constant
log n logarithmic
n linear
n log n log linear
n2 quadratic
n3 cubic
an exponential



Classes of Algorithms



Evaluating Python Code

l  Basic operations only require constant time:
l  x = 5
l  z = x + y * 6
l  if x > 0 and x < 100

l  What about function calls?
y = ex1(n)



Code Evaluation #1

def ex1( n ):
count = 0
for i in range( n ):
count += i
return count



Code Evaluation #2

def ex2( n ):
count = 0
count += 1
for j in range( n ):
count += 1
return count



Code Evaluation #3

def ex3( n ):
count = 0
for j in range( n ):
count += 1
return count



Code Evaluation #3b

def ex3b( n ):
count = 0
count += ex2( n )
return count



Code Evaluation #4

def ex4( n ):
count = 0
for j in range( 25 ):
count += 1
return count



Code Evaluation #5

def ex5( n ):
count = 0
for j in range( i+1 ):
count += 1
return count



Code Evaluation #6

def ex6( n ):
count = 0
i = n
while i >= 1 :
count += 1
i = i // 2
return count



Code Evaluation #7

def ex7( n ):
count = 0
count += ex6( n )
return count



Different Cases

l  Some algorithms have different run times for
different sets of inputs of the same size.
l  best case
l  worst case
l  average case



Different Cases

def findNeg( intSeq ):
n = len( intSeq )
if intSeq[i] < 0 :
return i
return None

L = [ 72, 4, 90, 56, 12, 67, 43, 17, 2, 86, 33 ]
p = findNeg( L )

L = [ -12, 50, 4, 67, 39, 22, 43, 2, 17, 28 ]
p = findNeg( L )



Searching
Searching


Searching
There are two fundamental ways to search for data in a list: the sequential
search and the binary search.
Sequential search is used when the items in the list are in random
order.
Binary search is used when the items are sorted in the list.


Sequential Search
The most obvious type of search
begin at the beginning of a set of records and move through each
record until you ﬁnd the record you are looking for or you come to the
end of the records.
A sequential search (also called a linear search) is very easy to
implement.


Sequential Search

bool SeqSearch ( int [ ] arr , int sValue ) {
for ( int index = 0 ; index < arr . Length −1; index++)
i f ( arr [ index ] == sValue )
return true ;
return false ;
}


Modiﬁed Sequential Search
You can write the sequential search function so that the function returns
the position in the array where the searched-for value is found or a 1 if the
value cannot be found. First, lets look at the new function:


Modiﬁed Sequential Search Code

static int SeqSearch ( int [ ] arr , int sValue ) {
for ( int index = 0 ; index < arr . Length −1; index++)
i f ( arr [ index ] == sValue )
return index ;
return −1;
}


Searching for Minimum and Maximum Values
Computer programs are often asked to search a data structure for
minimum and maximum values.
In an ordered array, searching for these values is a trivial task.
Searching an unordered array, however, is a little more challenging.
Lets look at how to ﬁnd the minimum value in an array. The
algorithm is:
1 Assign the ﬁrst element of the array to a variable as the minimum
value.
2 Begin looping through the array, comparing each successive array
element with the minimum value variable.
3 If the currently accessed array element is less than the minimum value,
assign this element to the minimum value variable.
4 Continue until the last array element is accessed.
5 The minimum value is stored in the variable.


FindMin Function

static int FindMin ( int [ ] arr ) {
int min = arr [ 0 ] ;
for ( int index = 1 ; index < arr . Length −1; i++)
i f ( arr [ index ] < min )
min = arr [ index ] ;
return min ;
}


FindMax Function

static int FindMax ( int [ ] arr ) {
int max = arr [ 0 ] ;
for ( int i = 0 ; i < arr . Length −1; i++)
i f ( arr [ index ] > max )
max = arr [ index ] ;
return max ;
}


Assignment
An alternative version of these two functions could return the position of
the maximum or minimum value in the array rather than the actual value.


Self-Organizing Data
The fastest successful sequential searches occur when the data
element being searched for is at the beginning of the data set.
You can ensure that a successfully located data item is at the
beginning of the data set by moving it there after it has been found.
The concept behind this strategy is that we can minimize search
times by putting frequently searched-for items at the beginning of the
data set.
Eventually, all the most frequently searched-for data items will be
located at the beginning of the data set.
This is an example of self-organization, in that the data set is
organized not by the programmer before the program runs, but by the
program while the program is running.


Binary Search
When the records you are searching through are sorted into order, you
can perform a more eﬃcient search than the sequential search to ﬁnd
a value.
This search is called a binary search.


Binary Search in Action
To understand how a binary search works, imagine you are trying to
guess a number between 1 and 100 chosen by a friend.
For every guess you make, the friend tells you if you guessed the
correct number, or if your guess is too high, or if your guess is too low.
The best strategy then is to choose 50 as the ﬁrst guess. If that guess
is too high, you should then guess 25. If 50 is to low, you should
guess 75.
Each time you guess, you select a new midpoint by adjusting the
lower range or the upper range of the numbers (depending on if your
guess is too high or too low), which becomes your next guess.
As long as you follow that strategy, you will eventually guess the
correct number.
Next slide demonstrates how this works if the number to be chosen is
82.


Binary Search in C#

static int binSearch ( int value ) {
int upperBound , lowerBound , mid ;
upperBound = arr . Length −1;
lowerBound = 0 ;
w h i l e ( lowerBound <= upperBound ) {
mid = ( upperBound + lowerBound ) / 2 ;
i f ( arr [ mid ] == value )
return mid ;
else
i f ( value < arr [ mid ] )
upperBound = mid − 1 ;
else
lowerBound = mid + 1 ;
}
return −1;
}


Recursive Binary Search

public int RbinSearch ( int value , int lower , int upper ) {
i f ( lower > upper )
return −1;
else {
int mid ;
mid = ( int ) ( upper+lower ) / 2;
i f ( value < arr [ mid ] )
RbinSearch ( value , lower , mid −1) ;
e l s e i f ( value = arr [ mid ] )
return mid ;
else
RbinSearch ( value , mid +1 , upper )
} }


Iterative vs. Recursive Binary Search
The main problem with the recursive binary search algorithm, as
compared to the iterative algorithm, is its eﬃciency.
When a 1,000-element array is sorted using both algorithms, the
recursive algorithm is consistently 10 times slower than the iterative
algorithm


Excercises
Just a Reminder


Lect07

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Lect07

Similar a Lect07 (20)

Más de Haitham El-Ghareeb

Más de Haitham El-Ghareeb (20)

Lect07