Sorting algorithms

Algorithms

Sorting
Vicente García Díaz – garciavicente@uniovi.es
University of Oviedo, 2013

2

Table of contents
Sorting
1. Basic concepts
2. Sorting algorithms
▫ Simple algorithms
 Sorting by direct exchange
 Sorting by insertion
 Sorting by selection
▫ Sophisticated algorithms
 Sorting by direct exchange
 Sorting by insertion
 Sorting by selection
▫ Constrained algorithms
 Radix
▫ External algorithms

4

Basic concepts

Introduction
• Given a set of n elements a1, a2, a3, …, an
and an order relation (≤) the problem of sorting is
to sort these elements increasingly

• What can influence the sorting?
▫ The type
▫ The size
▫ The device on which they are

• Integers stored in a vector (simplification)

5

Basic concepts

Sorting integers
• In practice, the problems tend to be much more
complex
▫ However they can be reduced to the same problem
that the integers (using registry keys, indexes, etc.)

• In essence it is the same to sort numbers or listed
streets, houses, cars or any numerically
quantifiable property

6

Basic concepts

Classification criteria for algorithms
1. Total number of steps
▫ Complexity
2. Number of comparisons performed
3. Number of interchanges performed
▫ Much more expensive than the comparison
4. Stability
5. Type of memory used
▫ Internal algorithms
▫ External algorithms

7

Basic concepts

Preliminary considerations (I)
• We will sort arrays of integers using the Java
programming language
• We will make use of utility methods to facilitate
the work and make the code more readable

8

Basic concepts

Preliminary considerations (II)

10

Sorting algorithms Simple algorithms Sorting by direct exchange
Sophisticated algorithms Sorting by insertion
Constrained algorithms Sorting by selection

Bubble External algorithms

• Description
▫ Based on the successive 1
comparison and exchange of
2
adjacent elements
▫ At each step, each item is 3
compared with the previous
4
one and in case of being 6 5
disordered, they are exchanged 7
▫ In the first step, the smallest 8
element will be placed in the
leftmost position, and so on…

11



• Initial vector: 4 5 6 1 3 2 7 8

1 1 4 5 6 2 3 7 8
2 1 2 4 5 6 3 7 8
3 1 2 3 4 5 6 7 8
4 1 2 3 4 5 6 7 8
5 1 2 3 4 5 6 7 8
6 1 2 3 4 5 6 7 8
7 1 2 3 4 5 6 7 8

12



Best case: O(n2)
• High number of exchanges
• High number of comparisons
Worst case: O(n2)
Average case: O(n2)

13


Bubble with sentinel External algorithms

• Description
▫ Observing the previous result 1
you can see that the last steps
2
are repeated unnecessarily
▫ To avoid this you can enter a 3
stop condition when the vector
4
is sorted 6 5
 If you make a pass and you do 7
not make any exchange it
8
means that it is already sorted

14



• Initial vector: 4 5 6 1 3 2 7 8

1 1 4 5 6 2 3 7 8
2 1 2 4 5 6 3 7 8
3 1 2 3 4 5 6 7 8
4 1 2 3 4 5 6 7 8

15



Best case: O(n)
• Complexity whether the input would be 8 1 2 3 4 5 6 7? Worst case: O(n2)
Average case: O(n2)

16



• Initial vector: 8 1 2 3 4 5 6 7

1 1 8 2 3 4 5 6 7
2 1 2 8 3 4 5 6 7
3 1 2 3 8 4 5 6 7
4 1 2 3 4 8 5 6 7
5 1 2 3 4 5 8 6 7
6 1 2 3 4 5 6 8 7
7 1 2 3 4 5 6 7 8

17


Bidirectional bubble External algorithms

• Description
▫ Used to prevent problems when 1
the vector is almost sorted
2
▫ It operates in both directions at
the same time 3
4
6 5
7
8

18



• Initial vector: 8 1 2 3 4 5 6 7

1 1 2 3 4 5 6 7 8
2 1 2 3 4 5 6 7 8

19



Best case: O(n)
Worst case: O(n2)
• It still has a high number of comparisons and
exchanges, just as the other methods of bubble Average case: O(n2)

20


Direct insertion External algorithms

• Description
▫ The vector is divided into two
“virtual” parts: an ordered and Unordered sequence
an unordered sequence
6 8 2 5
▫ At each step we take the first
element of the unordered
sequence and insert it into the
corresponding place of the
ordered sequence 1 3 4 7
Ordered sequence

21



• Initial vector: 4 5 6 1 3 2 7 8

Ordered vector Unordered vector
1 4 5 6 1 3 2 7 8
2 4 5 6 1 3 2 7 8
3 1 4 5 6 3 2 7 8
4 1 3 4 5 6 2 7 8
5 1 2 3 4 5 6 7 8
6 1 2 3 4 5 6 7 8
7 1 2 3 4 5 6 7 8

22



• Initial vector: 4 5 6 1 3 2 7 8

1 4 5 6 1 3 2 7 8
2 4 5 6 1 3 2 7 8
3 1 4 5 6 3 2 7 8
4 1 3 4 5 6 2 7 8
5 1 2 3 4 5 6 7 8
6 1 2 3 4 5 6 7 8
7 1 2 3 4 5 6 7 8

23



Best case: O(n)
• High number of exchanges Worst case: O(n2)
• Less comparisons than the bubble method Average case: O(n2)

24


Binary insertion External algorithms

• Description
▫ The direct insertion can be
improved taking into account Unordered sequence
that the sequence in which the
6 8 2 5
elements are inserted is already
sorted
▫ We use binary search to quickly
locate the suitable position for
the insertion 1 3 4 7
▫ *** The binary search was discussed Ordered sequence
in the introduction to Algorithms

25



• Initial vector: 4 5 6 1 3 2 7 8

1 4 5 6 1 3 2 7 8
2 4 5 6 1 3 2 7 8
3 1 4 5 6 3 2 7 8
4 1 3 4 5 6 2 7 8
5 1 2 3 4 5 6 7 8
6 1 2 3 4 5 6 7 8
7 1 2 3 4 5 6 7 8

The result is exactly the same as with the direct insertion

26



• Less comparisons than the direct insertion Best case: O(n logn)
• High number of exchanges Worst case: O(n2)
• THE INSERTION MOVING ELEMENTS ONE POSITION Average case: O(n2)
IS NOT ECONOMIC

27


Direct selection External algorithms

• Description
▫ It consists on selecting the
smallest element and exchange
it with the first element
▫ Repeat the process with the 1 2
3 4 7 5 3
2
remaining elements until there
is only one element (the
greatest of all)

28



• Initial vector: 4 5 6 1 3 2 7 8

1 1 5 6 4 3 2 7 8
2 1 2 6 4 3 5 7 8
3 1 2 3 4 6 5 7 8
4 1 2 3 4 6 5 7 8
5 1 2 3 4 5 6 7 8
6 1 2 3 4 5 6 7 8
7 1 2 3 4 5 6 7 8

29



• The number of exchanges is minimum Best case: O(n2)
• It is predictable: the number of exchanges and Worst case: O(n2)
comparisons depends on n Average case: O(n2)
• The number of comparisons is very high

30


ShellSort External algorithms

• Description
▫ The idea is to divide the vector
of elements in smaller virtual Unordered sequence
vectors
6 8 2 5
▫ Each subvector is sorted using
an insertion sort algorithm
▫ Each element of the subvectors
is positioned at a fixed distance
which decreases in each 1 3 4 7
iteration (increments sequence) Ordered sequence

31



59 20 17 13 28 14 23 83 36 98 11 70 65 41 42 15

• Increments sequence: {1, 2, 4, 8} (example)
Iteration 1
8 sublists of size 2 36 20 11 13 28 14 23 15 59 98 17 70 65 41 42 83
(increment of 8)

Iteration 2
4 sublists of size 4 28 14 11 13 36 20 17 15 59 41 23 70 65 98 42 83
(increment of 4)

Iteration 3
2 sublists of size 8 11 13 17 14 23 15 28 20 36 41 42 70 59 83 65 98
(increment of 2)

Iteration 4
1 sublists of size 16 11 13 14 15 17 20 23 28 36 41 42 59 65 70 83 98
(increment of 1)

**The elements of the sequence of increments should not be multiples of each other

32



• Initial vector: 4 5 6 1 3 2 7 8 with increments
sequence {1, 3, 7}
K=7 1 4 5 6 1 3 2 7 8
K=3 2 1 3 2 4 5 6 7 8
K=1 3 1 2 3 4 5 6 7 8

33



The complexity depends on the sequence of increments
We do not know the sequence that works best
Knuth recommends:
hk = 3hk-1 + 1  1, 4, 13, 40, …
h1 = 1
hk = 2hk-1 + 1  1, 3, 7, 15, 31, …  O(n1.2)
• Can be optimized by distributing the processing of each subvector
across multiple processors

34



• Principle on which relies
▫ THEOREM: If a sequence is sorted in n by n and
subsequently ordered at intervals of j by j, it will
also be ordered in n by n
• The intervals should be decreasing
• The last interval must be 1
• The interval values should not be multiples of
each other (for efficiency)

35



ShellSort
Direct insertion

VS

36


HeapSort External algorithms

Unsorted area of the vector
• Description
▫ The idea is to create a heap of 5 4
maximums in the vector
▫ Subsequently, the first element is 4 3 3 2
chosen (maximum) and exchanged
with the final element of the 1 2
vector (sorted area)
1
▫ The heap is reorganized and we
again choose the first element
Sorted area of the vector
(maximum) to place it in the last
unsorted position of the vector
 The process is repeated until all the
4 5 6 7 9
elements of the unsorted vector are
placed in the sorted vector

37



• Idea of the algorithm
CREATE HEAP OF MAXIMUMS IN THE FIRST PART OF VECTOR 
O(n) First part

GET THE ROOT ELEMENT AND EXCHANGE IT WITH THE LAST
ELEMENT OF THE VECTOR (LAST POSITION)  O(1)
Second part

REPEAT UNTIL THE HEAP IS EMPTY  O(n)
REORGANIZE THE HEAP OF MAXIMUMS  O(log n)
GET THE ROOT ELEMENT AND EXCHANGE IT WITH THE NEXT
UNSORTED ELEMENT OF THE VECTOR  O(1)

38



4

• Initial vector: 4 5 6 1 3 2 7 8 5 6

1. Create heap of maximums 1 3 2 7
sinks 1 4 sinks 6 4
8 sinks 5
4
5 6 5 7
8 7
8 3 2 7 8 3 2 6
5 3 2 6
1 1
8 sinks 4
1
5 7

4 3 2 6

1 8 5 7 4 3 2 6 1

39



• Initial vector: 8 5 7 4 3 2 6 1
2. Exchange and restructure the heap
1 sinks 1 7

5 7 5 6

4 3 2 6 4 3 2 1

1 5 7 4 3 2 6 8 7 5 6 4 3 2 1 8

40



• Initial vector: 7 5 6 4 3 2 1 8
1 sinks 1 6

5 6 5 2

4 3 2 4 3 1

1 5 6 4 3 2 7 8 6 5 2 4 3 1 7 8

41



• Initial vector: 6 5 2 4 3 1 7 8
1 sinks 1 5

5 2 4 2

4 3 1 3

1 5 2 4 3 6 7 8 5 4 2 1 3 6 7 8

42



• Initial vector: 5 4 2 1 3 6 7 8
3 sinks 3 4

4 2 3 2

1 1

3 4 2 1 5 6 7 8 4 3 2 1 5 6 7 8

43



• Initial vector: 4 3 2 1 5 6 7 8
1 sinks 1 3

3 2 1 2

1 3 2 4 5 6 7 8 3 1 2 4 5 6 7 8

44



• Initial vector: 3 1 2 4 5 6 7 8
2 sinks 2 2

1 1

2 1 3 4 5 6 7 8 2 1 3 4 5 6 7 8

45



• Initial vector: 2 1 3 4 5 6 7 8
1 sinks 1 1

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

46



Best case: O(n logn)
• Not recommended for small vectors Worst case: O(n logn)
• Fast for large vectors Average case: O(n logn)

47



• Peculiarities
▫ First all big elements are moved to the left

▫ Then they are moved to the right

▫ The best case does not have to be the ordered vector

▫ The worst case does not have to be the unordered vector

48


QuickSort External algorithms

Partitioned element
• Description
▫ It is based on partitioning
▫ When you partition an item, 2 3 1 4 7 6
that item will be in its
corresponding position
 Then, the idea is to partitionate 1 2 3 6 7
all the elements
▫ You have to choose a good
element to partitionate (pivot) 1 3 7
so that it is in the center and it
creates a tree (if would be on
one corner it will create a list)
▫ The implementation is recursive

49



• Criteria for choosing a good pivot 1 3 8 7 2 4 6 5
▫ The median
 It is the ideal solution but it is very expensive to compute
▫ The first element
 It is "cheap" but it is a bad choice
▫ The last element
 Occurs as with the first element
▫ A random element
 Statistically it is not the worst choice, but has computational cost
▫ The central element
 Statistically it is not a bad choice
▫ A compromise solution (median-of-3)
 We obtain a sample of 3 elements and calculate the median
 It does not guarantee anything but it can be a good indicator
 We chose the first element, the last element and the center element
 We order the elements, and we assume that the median is the element which is
in the center

50



• Idea of the algorithm
REPEAT UNTIL ALL THE ELEMENTS ARE SORTED  O(log n) …O(n)

First
CHOOSE A PIVOT  Using median-of-3 is O(1) part

PARTITIONING THE PIVOT THROUGH A PARTITIONING
STRATEGY  Typical case O(n) Second part

51



• Initial vector: 4 5 6 1 3 2 7 8
Choice of pivot
1 4 5 6 1 3 2 7 8 1 1 5 6 8 3 2 7 4

Exchange 1
i j

1 1 1 2 6 8 3 5 7 4
1 5 6 4 3 2 7 8

Partitioning strategy Pivot 1 1 2 6 8 3 5 7 4
Exchange 2
1 1 5 6 8 3 2 7 4 i j
i j 1 1 2 3 8 6 5 7 4
Search for an element[i] > pivot and an
element[j] < pivot and interchange them

i is moved to the right and j is moved to the left 1 1 2 3 8 6 5 7 4
displacing elements element[j] < pivot to the left j i
Positioning

and elements element[i] > pivot to the right
(Ends when j and i se are crossed, determining i 1 1 2 3 4 6 5 7 8
the final position)

52



• Initial vector: 4 5 6 1 3 2 7 8
1 2 3 4 6 5 7 8
Choice of pivot Choice of pivot
2 1 2 3 2 6 5 7 8

2 1 2 3 2 5 6 7 8
Partitioning strategy Partitioning strategy
Pivot
2 5 8 7 6
2 1 2 3

Positioning
2 5 8 7 6 j i
i j
Being 3 items or less, the
2 5 6 7 8
elements are already sorted
when searching the median of 3
(ends the recursion in that
branch)

53



• Initial vector: 4 5 6 1 3 2 7 8
5 6 7 8
Choice of pivot
3 7 8
3 5
3 7 8
There is only one element, then
there is no need to do the Choice of pivot
median of 3 because the
element is already sorted (ends
the recursion in that branch) 3 7 8
Being 3 items or less, the
elements are already sorted
when searching the median of 3
(ends the recursion in that
branch)

54



• Recursive calls performed 4 5 6 1 3 2 7 8
1 1 2 3 4 6 5 7 8

2 1 2 3 2 5 6 7 8

3 5 3 7 8

55



Best case: O(n logn)
• Very fast for values of n > 20 Worst case: O(n2)
• Optimizable distributing the processing of each Average case: O(n logn)
partition in different threads in parallel

56

Sorting algorithms

Comparison of algorithms (I)
• Data consists of only one key

n = 256 Sorted Random Inverse n = 512 Sorted Random Inverse

Bubble 540 1026 1492 Bubble 2165 4054 5931

Bubble with sent. 5 1104 1645 Bubble with sent. 8 4270 6542

Bidirect. bubble 5 961 1619 Bidirect. bubble 9 3642 6520

Direct insertion 12 366 704 Direct insertion 23 1444 2836

Binary insertion 56 373 662 Binary insertion 125 1327 2490

Direct selection 489 509 695 Direct selection 1907 1956 2675

ShellSort 58 127 157 ShellSort 116 349 492

HeapSort 116 110 104 HeapSort 253 241 226

QuickSort 31 60 37 QuickSort 69 146 79

57

Sorting algorithms

Comparison of algorithms (II)
• Data consists of only one key VS data with a size 7 times
the key size
n = 256 Sorted Random Inverse n = 256 Sorted Random Inverse

Bubble 540 1026 1492 Bubble 610 3212 5599

Bubble with sent. 5 1104 1645 Bubble with sent. 5 3237 5762

Bidirect. bubble 5 961 1619 Bidirect. bubble 5 3071 5757

Direct insertion 12 366 704 Direct insertion 46 1129 2150

Binary insertion 56 373 662 Binary insertion 76 1105 2070

Direct selection 489 509 695 Direct selection 547 607 1430

ShellSort 58 127 157 ShellSort 186 373 435

HeapSort 116 110 104 HeapSort 264 246 227

QuickSort 31 60 37 QuickSort 55 137 75

58

Sorting algorithms Simple algorithms
Sophisticated algorithms
Constrained algorithms

Radix External algorithms

• Description 8 0 5
▫ It's actually a external sorting 3 1 2 6 4
method
▫ It is not COMPARATIVE
▫ It can be used to sort sequences of
digits that support a lexicographic
order (words, numbers, dates, ...)
▫ It consists of defining K queues[0,
k-1], being k the possible Q0 Q1 … Q9
values that can take each of the
digits that make up the sequence
▫ Then the elements are distributed in
the queues carrying different passes
(ordered from the least significant
digit to the most significant)

59



• Initial vector: 48 5 86 1 3 2 25 8 0 23 27 42

▫ Natural numbers in base 10  10 queues
Queues 0 1 2 3 4 5 6 7 8 9

First pass 0 1 2, 42 3, 23 5, 25 86 27 48, 8
(least
significant
digit -> 0) 0 1 2 42 3 23 5 25 86 27 48 8

Second last 0, 1, 2, 23, 25, 42, 48 86
(next least 3, 5, 8 27
significant
digit -> 1) 0 1 2 3 5 8 23 25 27 42 48 86

60



Best case: O(k * n) = O(n)
• Needs external data structures Worst case: O(k * n) = O(n)
• Not very good if many digits in each element
Average case: O(k * n) = O(n)
• It is very efficient
**K is the number of digits of each element

61


External algorithms External algorithms

• They are based on the principle of DIVIDE AND
CONQUER
• They use external memory
▫ The slowest of them is like Quicksort
• They use the same basic principles that internal
sorting algorithms
• Different algorithms:
▫ Direct mixture
▫ Natural mixture
▫ Balanced mixture
▫ Polyphasic mixture

62

Bibliography
JUAN RAMÓN PÉREZ PÉREZ; (2008) Introducción al diseño y análisis de algoritmos en
Java. Issue 50. ISBN: 8469105957, 9788469105955 (Spanish)

Sorting algorithms

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Más de Vicente García Díaz

Más de Vicente García Díaz (16)

Último

Último (20)

Sorting algorithms