•1 recomendación•580 vistas

Descargar para leer sin conexión

Denunciar

Seguir

- 2. Introduction Dynamic programming is a good technique for many optimization problems, but may be overkill for many others Relies on global optimization of all choices at all steps Greedy algorithms make the choice that seems best at the moment Choices are locally optimal, in the hopes that this will lead to a globally optimal solution Because of this, greedy algorithms do not always yield optimal solutions, but many times they do
- 3. An Activity-Selection Problem Suppose we have a set S={1, 2, …, n} of n proposed activities that wish to use a resource - a lecture hall, for example - that can only be used by one activity at a time Each activity i has a start time si, and finish time fi, where si <= fi Selected activities take place during the interval [si, fi) Activities i and j are compatible if their respective intervals [si, fi) and [sj, fj) do not overlap The activity-selection problem is to select a maximum-size set of mutually compatible activities
- 4. An Activity-Selection Problem Here is a greedy algorithm to select a maximum size set of mutually compatible activites It assume that input activities have been sorted by increasing finish time GreedyActivitySelector(time s[], time f[], stack<int> &A, int size) { int j = 0; for ( int i = 1 ; i < n ; ++i ) { if ( s[i] >= f[j] ) { A.push(i); j = i; } } } GreedyActivitySelector(time s[], time f[], stack<int> &A, int size) { int j = 0; for ( int i = 1 ; i < n ; ++i ) { if ( s[i] >= f[j] ) { A.push(i); j = i; } } }
- 5. An Activity-Selection Problem How does it work? A collects the selected activities j specifies the most recent addition to A As such, fj is always the maximum finishing time of any activity in A We use this to check compatibility - an activity is compatible (and added to A) if it’s start time is not earlier than the finish time of the most recent addition to A
- 6. An Activity-Selection Problem As the algorithm progresses, the activity picked next is always the one with the earliest finish time that can legally be scheduled This is thus a “greedy” choice in that it leaves as much opportunity as possible for the remaining activities to be scheduled i.e., it maximizes the amount of unscheduled time remaining
- 7. An Activity-Selection Problem How efficient is this algorithm? There’s only a single loop, so it can schedule a set of activities that have already been sorted in O(n) time How does this compare with a dynamic programming approach? Base the approach on computing mi iteratively for i=1, 2, …, n, where mi is the size of the largest set of mutually compatible activities among activities {1, 2, …, i}
- 8. Elements of the Greedy Strategy Greedy algorithms solve problems by making locally optimal choices This won’t always lead to a globally optimal solution, but many times will Problems that lend themselves to the greedy strategy exhibit two main properties: The greedy-choice property Optimal substructure
- 9. Elements of the Greedy Strategy The greedy-choice property A globally optimal solution can be arrived at by making a locally optimal (greedy) choice In dynamic programming, choices may depend on the answers to previously computed subproblems In a greedy algorithm, we make the best choice we can at the moment then solve the resulting subproblems May depend on choices so far, but cannot depend on future choices or on solutions to subproblems Usually progresses in top-down fashion
- 10. Elements of the Greedy Strategy Proving that greedy choices yields globally optimal solutions Examine a globally optimal solution Modify the solution to make a greedy choice at the first step, and show that this reduces the problem to a similar but smaller problem Reduces the problem of correctness to demonstrating that an optimal solution must exhibit optimal substructure Apply induction to show that a greedy choice can be used at every step
- 11. Elements of the Greedy Strategy Optimal substructure A problem exhibits optimal substructure if an optimal solution to the problem contains within it optimal solutions to subproblems This is a key of both dynamic programming and greedy algorithms
- 12. Greedy vs. Dynamic Programming Consider two similar problems The 0-1 knapsack problem A thief robbing a store finds n items; the ith item is worth vi dollars and weighs wi pounds (both integers) The thief wants to take the most valuable load he can, subject to the maximum weight W that he can carry What items does he choose? The fractional knapsack problem Same thief, same problem, except that the thief may take fractions of items rather than having to either take it or leave it
- 13. Greedy vs. Dynamic Programming Both problems exhibit the optimal substructure property Removing an item means that the items that remain must be the most valuable that can be carried within the remaining weight allowance (W- w, where w is the weight of the item/fraction of item removed)
- 14. Greedy vs. Dynamic Programming Solving the fractional knapsack problem First compute the value per pound vi/wi for each item By the greedy strategy, the thief starts by grabbing as much as possible of the item with the greatest value per pound If he runs out of the first item, he starts with the next highest value per pound and grabs as much of it as possible And so on, until no more can be carried
- 15. Greedy vs. Dynamic Programming Solving the 0-1 knapsack problem The greedy solution doesn’t work here Grabbing the item with the highest value per pound doesn’t guarantee that you get the most value, nor does grabbing the biggest item What must happen in this case is that solutions to subproblems must be compared to determine which one provides the most valuable load This is the overlapping subproblem property
- 16. Huffman Codes Huffman codes are an effective technique for compressing data The algorithm builds a table of the frequencies of each character in a file The table is then used to determine an optimal way of representing each character as a binary string
- 17. Huffman Codes Consider a file of 100,000 characters from a- f, with these frequencies: a = 45,000 b = 13,000 c = 12,000 d = 16,000 e = 9,000 f = 5,000
- 18. Huffman Codes Typically each character in a file is stored as a single byte (8 bits) If we know we only have six characters, we can use a 3 bit code for the characters instead: a = 000, b = 001, c = 010, d = 011, e = 100, f = 101 This is called a fixed-length code With this scheme, we can encode the whole file with 300,000 bits We can do better Better compression More flexibility
- 19. Huffman Codes Variable length codes can perform significantly better Frequent characters are given short code words, while infrequent characters get longer code words Consider this scheme: a = 0; b = 101; c = 100; d = 111; e = 1101; f = 1100 How many bits are now required to encode our file? 45,000*1 + 13,000*3 + 12,000*3 + 16,000*3 + 9,000*4 + 5,000*4 = 224,000 bits This is in fact an optimal character code for this file
- 20. Huffman Codes Prefix codes Huffman codes are constructed in such a way that they can be unambiguously translated back to the original data, yet still be an optimal character code Huffman codes are really considered “prefix codes” No code word is a prefix of any other codeword This guarantees unambiguous decoding Once a code is recognized, we can replace with the decoded data, without worrying about whether we may also match some other code
- 21. Huffman Codes Both the encoder and decoder make use of a binary tree to recognize codes The leaves of the tree represent the unencoded characters Each left branch indicates a “0” placed in the encoded bit string Each right branch indicates a “1” placed in the bit string
- 22. Huffman Codes 100 a :45 0 55 1 25 0 c :12 0 b :13 1 30 1 14 0 d :16 1 f :5 e :9 0 1 A Huffman Code Tree To encode: Search the tree for the character to encode As you progress, add “0” or “1” to right of code Code is complete when you find character To decode a code: Proceed through bit string left to right For each bit, proceed left or right as indicated When you reach a leaf, that is the decoded character
- 23. Huffman Codes Using this representation, an optimal code will always be represented by a full binary tree Every non-leaf node has two children If this were not true, then there would be “waste” bits, as in the fixed-length code, leading to a non-optimal compression For a set of c characters, this requires c leaves, and c-1 internal nodes
- 24. Huffman Codes Given a Huffman tree, how do we compute the number of bits required to encode a file? For every character c: Let f(c) denote the character’s frequency Let dT(c) denote the character’s depth in the tree This is also the length of the character’s code word The total bits required is then: ∑ ∈ = Cc T cdcfTB )()()(
- 25. Constructing a Huffman Code Huffman developed a greedy algorithm for constructing an optimal prefix code The algorithm builds the tree in a bottom-up manner It begins with the leaves, then performs merging operations to build up the tree At each step, it merges the two least frequent members together It removes these characters from the set, and replaces them with a “metacharacter” with frequency = sum of the removed characters’ frequencies
- 26. Constructing a Huffman Code Huffman(table C[], int n, tree T) { // Create a priority queue that has all characters // sorted by their frequencies - each entry happens // to also be a complete tree node PriorityQueue Q(C); for ( int i = 0 ; i < n-1 ; ++i ) { z = T.AllocateNode(); x = z->Left = Q.ExtractMin(); y = z->Right = Q.ExtractMin(); z->freq = x->freq + y->freq; Q.Insert(z); } } Huffman(table C[], int n, tree T) { // Create a priority queue that has all characters // sorted by their frequencies - each entry happens // to also be a complete tree node PriorityQueue Q(C); for ( int i = 0 ; i < n-1 ; ++i ) { z = T.AllocateNode(); x = z->Left = Q.ExtractMin(); y = z->Right = Q.ExtractMin(); z->freq = x->freq + y->freq; Q.Insert(z); } }
- 27. Constructing a Huffman Code f :5 e :9 c :12 b :13 d :16 a :45 The algorithm starts by placing an entry in a priority queue for each character, sorted by frequency f :5 e :9 c :12 b :13 d :16 a :4514 0 1 Next it removes the two entries with the lowest frequency and inserts them into the Huffman tree. It replaces them with a node that has a frequency = to the sum of their removed frequencies
- 28. Constructing a Huffman Code The tree after removing and merging c & b Because we’re using a priority queue, each node is inserted into it’s sorted position, based on the sum of the frequencies of its children f :5 e :9 c :12 b :13 d :16 a :4514 0 1 25 0 1
- 29. Constructing a Huffman Code f :5 e :9 c :12 b :13 d :16 a :45 14 0 1 25 0 1 30 0 1 The tree after removing the previously merged f & e, and d, and merging them
- 30. Constructing a Huffman Code f :5 e :9 c :12 b :13 d :16 a :45 14 0 1 25 0 1 30 0 1 55 0 1 The tree after removing the “25” and “30” frequency nodes
- 31. Constructing a Huffman Code f :5 e :9 c :12 b :13 d :16 a :45 14 0 1 25 0 1 30 0 1 55 0 1 100 0 1 The final tree
- 32. Constructing a Huffman Code What is the running time? If we use a binary heap for the priority queue, it takes O(n) to build The loop is executed n-1 times Each heap operation requires O(log2n) So, building the Huffman tree requires O(nlog2n)