Slides from Computational Thinking (book and class)

1. Week 8
Searching and sorting
In this lecture we cover two fundamental algorithmic problems: searching
and sorting. These are not problems you will like ever need to solve yourself
nor are they algorithms you will find yourself implementing. They are already
implemented in the programming environment you use unless you start
entirely from scratch. But the problems are easy to understand so they are a
good place to exercise thinking about algorithms.
Searching

2. The search problem is this: you have a collection of objects and you want to
check if a given element is among them. We often call the element a key.
The reason we call it a key is that we sometimes have information
associated with the elements—we call those values—and we map keys to
values. For example, in a phone book—back when we had such beasts—
you would have names associated with numbers. The names are the keys,
the phone numbers the values. You search for the key because you want to
find the associated value.

3. Linear search
The simplest way to search is to look at each element in turn, compare the
keys, and if one matches, you get the key-value pair. In the example in the
book, we just determine if a key is found in a list of numbers, but getting a
value associated with a key is just as easy. You can try to implement that
yourself. Because we simply run through each element in turn, we call this a
linear search.
Linear search
• What is the best-case running time?
• What is the worst-case running time?
• What is the memory usage?
• Which assumptions have we made about the keys and
how the data is represented?
To understand linear search, try to answer these questions.

4. Binary search
When you look up names in a phone book, you do not start at the beginning
and search each name in turn. You exploit that the names are sorted
alphabetically (we also call this lexically sorted, but that is just a different
name for the samething). In a phone book you have some intuition about
where a name is likely found so you start there, but in general you would
start in the middel. If you cut the search-space in half every time you do not
find the key, you have binary search.
Binary search
• What is the best-case running time?
• What is the worst-case running time?
• What is the memory usage?
• Which assumptions have we made about the keys and
how the data is represented?
To understand the difference between linear search and binary search, try to
answer these questions.

5. Sorting
The sorting problem is this: given a collection of element, order them with
respect to their keys.
Properties of sorting
algorithms
We often classify sorting algorithms according to these properties. They are
defined in the book, but can you explain them in your own words?

6. Comparison-based
sorting
Another classification of sorting algorithms is whether they are comparison
based or not. Comparison-based means that we can compare keys to
determine which is smaller than anther, but we do not make any other
assumptions. (The specific algorithms might need more assumptions,
though, but it is the only assumption we make about the keys).
Insertion sort
With insertion sort, you have a list of sorted elements and a list of those you
have yet to sort. You pick one of the unsorted elements in turn, and move it
to where it belongs in the sorted list. You might have to move some of the
sorted elements if they are larger than the new one.

7. Here, the blue underline indicates the already-sorted elements, the orange
arrow shows where we compare to keys, and the red arrows where we swap
two elements.

8. • Termination: Trivial — nested finite for-loops
• Correctness:
We use these two invariants to prove correctness. Can you argue why they
guarantee correctness when the algorithm finishes? Can you show why the
loop-body guarantees the invariants?
Insertion sort
• Which assumptions have we made?
• Is it stable?
• What is the worst-case number of comparisons?
• What is the best-case number of comparisons?
• What is the worst-case number of swaps?
• What is the best-case number of swaps?
Try to answer these questions. The answers are not the same for all the
algorithms, so think about them.

9. Selection sort
With selection sort you also have a list of sorted elements and a list of keys
that are yet to be sorted. This time, though, you ensure that the sorted
elements are smaller than (or equal to) the smallest in the set of elements we
haven’t sorted yet. In each iteration, we pick the smallest of those keys and
add it to the sorted list.

10. • Termination: Trivial — nested finite for-loops
• Correctness:
Can you argue why this invariant is guaranteed by the algorithm and why it
ensure correctness?
Selection sort
• Which assumptions have we made?
• Is it stable?
• What is the worst-case number of comparisons?
• What is the best-case number of comparisons?
• What is the worst-case number of swaps?
• What is the best-case number of swaps?
Now, try to answer these questions. Where do you get different answers
than for insertion sort?

11. Bubble sort
With bubble sort we do not have a collection of sorted and a collection of
not-sorted elements. Instead, we repeatedly swap pairs that are out order,
i.e. pairs where the first is larger than the second.

12. • Termination: Not trivial! (But wait for it…)
• Correctness:
j is # outer iterations
The invariants are a bit more complicated, but try to argue that they are
satisfied and that they ensure correctness.
• Termination:
• Correctness:
j is # outer iterations
While j is only used for the analysis,
and not a real "for-loop" variable we
can use it in the termination function:
t(j) = n - j
Since we do not use simple for-loops in bubble sort (at least for the
outermost loop), we need to prove that the algorithm terminates. We use a
“virtual” variable, j, for this. Can you argue why this works and guarantees
termination?

13. Bubble sort
• Which assumptions have we made?
• Is it stable?
• What is the worst-case number of comparisons?
• What is the best-case number of comparisons?
• What is the worst-case number of swaps?
• What is the best-case number of swaps?
Again, try to answer these questions.
Index-based sorting
With index-based sorting algorithms we make stronger assumptions about
the keys. We assume that they can be used as indices into an array—an
array, here, is just like a Python list, but in algorithms we make some
distinctions between “arrays” and “lists”. Lists are sequences we can iterate
through; arrays are sequences with random access. Python lists are both.

14. Count sort
Count sort is the simplest of these algorithms. We simply count how often
we see each key, and then output a list that has the right number of each
key in order.
Bucket sort
If we need to distinguish between the keys—for example if we associated
values with them—then we have bucket sort. Here I have just used colours
as the values. With bucket sort we need to keep track of the values. To get a
stable sort, we must output the key-value pairs in the same order as we saw
them in the input. How do you do that with Python lists?

15. Bucket sort
• Which assumptions have we made?
• Is it stable?
• What is the best- and worst-case running time?
• What is the best- and worst-case memory usage?
Answer these questions.
Radix sort
We can use radix sort when we have multiple keys that we can use in a
bucket sort, or when we can split our keys into sub-keys we can use in a
bucket sort. The order in which we sort the different keys matter—can you
explain why? The bucket sort must be stable. Can you explain why?

16. Exercises!
Time to put searching
and sorting into practice