In lecture Monday we argued that no comparison-based search method can use fewer comparisons than does binary search in the worst case. We used an adversary argument, showing that given any search procedure that uses fewer comparisons, we can construct a case in which it gives the wrong answer. (See the "dishonest guessing game" on the practice third midterm, Question 7.)
In this discussion we will apply the same argument to comparison-based sorting algorithms. Given a list of n distince elements, there are n! possible linear orders in which they might be arranged. If we have sorted the list correctly, we must have determined that one of those orders is the right one, and ruled out the other n! - 1. The only tool we have to rule out orders is to compare one element against another.
For example, consider a list with the three elements a, b, and c. They might be originally in any of six possible orders: abc, acb, bac, bca, cab, or cba. If we find out that a comes before c, we know that the order must be abc, acb, or bac. If we then find out that b comes before c, we know that it is either abc or bac. Testing a against b tells us which order it was. In this case we needed to do all three possible comparisons, but sometimes we can find out the order without doing all of them. (For example, if we first first that a is before b and then that b is before c, we know that the order is abc without needing to test a agaisnt c.)
If we ask k possible yes-or-no questions, there are 2k possible outcomes -- we divide the n! orders into 2k categories. If our sorting algorithm is correct in every case, each of the orders is in its own category, so n! must be less than or equal to 2k. The number k, therefore, must be at least log2(n!), which is O(n log n) by an argument we won't do today.
Question 1: List the 4! = 24 possible orders of four elements a, b, c, and d. As you will hear over and over in CMPSCI 240, to know that you have listed a set correctly for counting you need to verify two things -- that every element is listed and that no element is listed twice.
Question 2: Describe (in English) how you could list all the possible orders of n + 1 elements, if you had a list of all the possible orders of n elements. Given this idea, describe (in English) a recursive algorithm to list the orders of n elements, for any n. What is the base case of this recursion?
Question 3: Draw a binary tree where the internal nodes are comparisons of two elements from the set {a, b, c}, each internal node has two children for the two possible outcomes of the comparison, and the leaves are the 3! = 6 orderings of the three elements.
Question 4: Repeat Question 3 for four elements. Your tree should have depth as small as possible while still having a leaf for each of the 4! = 24 possible orders. You may be able to descrie parts of the tree using symmetry rather than drawing out every node.
Last modified 27 November 2011