CMPSCI 251 (291A): Mathematics of Computation

David Mix Barrington

Spring, 2007

This is the home page for CMPSCI 251, known officially to the university as CMPSCI 291A. CMPSCI 251 is a second course in the mathematics of computation, following CMPSCI 250. In the new computer science curriculum now in development, CMPSCI 251 will be a required course, coming before the algorithms course, CMPSCI 311. In the current curriculum it may be used as the elective "fifth mathematics course" -- for further clarification speak to me or to your computer science academic advisor.

The subject matter of CMPSCI 251 divides into four parts:

A review of combinatorics and induction
Probability and probabilistic reasoning
Information theory, including error-correcting codes, and
Models of computation: Finite-state machines and Turing machines

Some of the material of CMPSCI 251 has been in CMPSCI 250 when I have been the instructor, and some overlaps with CMPSCI 311 and CMPSCI 401. It will not replace either of those courses, but should provide useful preparation for them.

Instructor Contact Info: David Mix Barrington, 210 CMPSCI building, 545-4329, office hours Tuesday and Thursday 1:30-3:30, or by appointment. I generally answer my email fairly reliably.

TA Contact Info: There is no TA for this course. I get to grade everything.

The only formal prerequisite is CMPSCI 250 or equivalent, but 250 expects MATH 132 (Calculus II) and CMPSCI 187 (programming in data structures using Java) as corequistites, so 251 will expect familiarity with this material as well. Calculus will come up occasionally but will be treated gently.

The course will meet for three lecture meetings a week, MWF 11:15-12:05 in room 203 of Lederle GRC (the low-rise). There will be one discussion meeting per week, Wed 10:10-11:00 in room 202 LGRT (the Lederle tower). Discussion sessions will consist of students solving specific problems alone or in groups and handing answers in, so missing any of these classes will incur a grade penalty. The three-credit course thus has four contact hours per week -- I will try to compensate for this (and for the fact that the course is elective rather than core) by cutting back on the homework and pace of material relative to those of the eventual core course version.

There is no published textbook for this course -- I have a draft book in development that serves the first and fourth quarters pretty well, and I will make this text available. For the second and third quarters of the course you will get the best text I have produced at the time -- something between detailed lecture notes and the final book sections.

Announcements (16 May 2007):

(16 May) The solutions to the practice final are posted. See you tomorrow at 10:30 in room A205 LGRC. I will be answering email tonight.

(11 May) The practice final is posted.

(3 May) I have posted the notes from yesterday's discussion. I still owe you notes for Discussions four through 9.

(27 Apr) I have posted the fourth and final homework assignment, due on Monday 13 May.

(27 Apr) Our final exam has been scheduled for 10:30-12:30 on Thursday 17 May, in room A205 LGRC. Please let me know as soon as possible about any conflicts.

(23 Apr) I have posted the third midterm and its solution. The exams were returned in class today -- the scale was A=92 and C=57, and the scores in sorted order were 92, 82, 75, 71, 65, 57, and 56.

(23 Apr) The final exam doesn't seem to have been scheduled yet -- I'm working on it and will let you know when this is fixed.

(19 Apr) I've corrected an error in the solution of Question 5 (e) in the third practice exam solutions. Thanks to Thijs for catching this, I had Pr(B|A) at 16/40 instead of 16/41. See you in the morning...

(18 Apr) I've now posted solutions to the third practice midterm, which we may discuss in class later this morning.

(16 Apr) No one has asked me any questions about the homework, which is a little worrying, but here are some hints in any case:

Problem I.2.2: For the first problem, where you know whether the single bad coin is heavier or lighter, you should be able to find an exact number of weighings, depending on n, that is both necessary and sufficient to find the bad coin. The second problem is more complicated. For n=1, and n=2, explain why it is impossible to find the bad coin by only weighing the coins you are given. For n=3, show that you can do it in two weighings but not one. You cannot do n=4 in two weighings (this is more complicated to show) but in general for n=3 you should be able to explain how to do it with one more weighing than you needed for the first problem. You don't need to prove the lower bound (that you can't do better than this) as it looks pretty complicated.
Problem I.3.1: It's possible to do this by strong induction, but it's easier (though not trivial) if you use the Theorem from the section about the result of the Huffman algorithm. Show that if a tree doesn't have its leaves on two adjacent levels, it isn't optimal and thus can't be the result of Huffman.
Problem I.4.2: Use induction on k, the Third Counting Problem, and arithmetic.
Problem I.4.5: Use induction on the number of leaves. A useful lemma here is to show that if I have a discrete random source where the two smallest weight letters do not have the same probability, the entropy of the source is smaller than the entropy that the source would have if those two leaves were the same. (That is, if the smallest two weights in X are p and q with p > q, make a new source X' where those leaves both have weight (p+q)/2 and show that H(X') < H(X).
Problem I.5.2: Here the "o(1)" refers to the limit as p approaches zero.
Problem I.7.3: This is a bit complicated. To make it simpler, first look at just the case of n=2. Show that if Y consists of two outputs Y₁ and Y₂, that I(W,Y) ≤ I(W,Y₁) + I(W,Y₂). (You can use Jensen's Inequality to show that I(W,Y) - I(W,Y₁) - I(W,Y₂) ≤ 0. The fact that the channel is memoryless means that Pr(y|x) = Pr(y₁|x₁)Pr(y₂|x₂).)

(13 Apr) Sorry I've gotten behind in updating this site. I have just posted the practice test for next Friday's midterm. The current homework is due on Wednesday 18 April (sorry for the errors in the 29 March entry, which I have corrected). I'll hand out solutions to HW#3 in discussion on Wednesday.

Also remember that Monday is a holiday and that Tuesday is a Monday. So we will have lecture on Tuesday (about DFA's, as the syllabus says). I will also have my Tuesday office hours even though it's a Monday schedule.

(29 Mar) I have posted HW#3 (not HW#2), which is due on Wednesday 18 March.

(18 Mar) I have posted the second exam and its solution. I have graded the exam and HW#2 and I will mail each of you individually with your grades. The scale on the second exam was A = 87 and C = 57, and the individual scores were 95, 81, 65, 63, 61, 46, and 41, the last two being below a passing standard. The scale on HW#2 was A = 70, C = 35, and the individual scores were 58, 55, 49, 38, 16, 12, and 10, the last three being below a passing standard. There were some difficult problems on HW#2, but several of you blew it off and shouldn't expect to pass this course without working harder and starting the homework earlier . It's not unreasonable to spend 5-10 hours on one of these assignments, as there are only four for the term and they are an essential way to prepare for the exams.

(14 Mar) I have posted the solutions for the second practice exam and the notes for Discussion #2 and Discussion #3. We'll see whether I get Discussion #4 and #5 notes up before the exam -- I will do #5 first because everyone pretty much got #4 in class. See you later this morning...

(13 Mar) A note in the extra-credit question of HW#2, Problem P.5.3. The problem is not clearly stated. As in Problem P.5.2 above it, I am looking for the smallest number k such that the chance of k random people having some pair of birthdays g days apart is more than 50%. The problem asks for g to be 1, 2, 3, 4, and 5, but I will give the full ten points extra credit for the case of g=1. This means that two of the k people have birthdays on the same day or on adjacent days (including 31 December and 1 January). Here is a hint -- you need to find the number of binary strings of length 365 that do not have two ones in a row and do not both begin and end with 1. This can be solved using the same sort of idea as our solution of the Fourth Counting Problem. If you get correct answers for the cases of larger g as well, I am willing to give more than ten extra credit points.

(11 Mar) I am withdrawing the last question of HW#2, Problem P.8.5, because as it is written it is not clear how to answer it (I'll discuss this in the exam review Wednesday morning). So HW#2 has only 90 points plus ten extra credit.

(9 Mar) The second practice exam is posted. I hope to hand out solutions to the Chapter P exercises in the book on Monday. I plan to hand out solutions to HW#2 when you hand HW#2 in at Wednesday's discussion class.

(28 Feb) The HW#2 assignment is posted. I see that I'm behind on posting discussion notes, and I'll try to catch up soon.

(28 Feb) Graded HW#1 was returned in class today. The scale was 80 for an A and 45 for a C, and the seven individual grades were 95, 82, 69, 65, 55, 48, and 31.

(27 Feb) I've updated the syllabus to match some changes I made while writing Chapter P. The material of Sections P.3 and P.4 also appears for the most part in Section 7.9 of the bound portion of the book that you have.

(26 Feb) The first midterm and its solution are now available. The seven scores on the test were 89, 86, 83, 76, 73, 61, and 60. The scale was 90 for the middle of the A range, 75 for B, and 60 for C. I apologize again for the error in Question 7 -- I dealt with this by being generous with partial credit on that question and lowering the scale from the A=93, C=63 I had planned.

By Wednesday I should have HW#1 graded and have the HW#2 assignment ready. HW#2 will be due two weeks from Wednesday, 14 March, in discussion class. The second midterm will be in class Friday 16 February, the day before spring break. It will cover chapter "P" of the textbook, which I handed out in draft form today. (Note that this draft does not have a glossary or answers to the exercises -- I may fix this by the time I give you Chapter I, which will be at the end of break.)

(21 Feb) I have to miss the second half of my office hours tomorrow because I am meeting with a committee of outside evaluators of the department. I will be in my office 1:30-2:30, and will read and answer email late in the evening.

(20 Feb) The solution to the first practice exam is posted. I have also answered one question on HW#1.

(16 Feb) I have posted the practice exam for next Friday's midterm. I'll post solutions on Tuesday. On Wednesday we will have a review for the exam during the discussion period, then a lecture on sections 7.6 and 7.7 (which won't be covered on the exam). Remember that HW#1 is due Wednesday in discussion -- I will answer email over the long weekend and will have my normal office hours on Tuesday.

(14 Feb) We are snowed out today, so there will be no discussion or lecture. We won't make up the discussion, but I'll post the exercise we would have done soon. On Friday we will talk about both sections 7.5 and 7.6, and by Friday I will post a practice exam for the first midterm (which is a week from Friday). Enjoy the snow!

(6 Feb) As I mentioned in class Monday, I have adjusted the syllabus to remove the material on Catalan numbers I was going to cover on Friday. I would like to make sure by the end of this week that we are all solid on the combinatorics material of sections 6.1-6.7. This might require some of you to work pretty hard this week, to catch up from where you started the course.

(6 Feb) My primary office hours for this course will be Tuesdays and Thursdays from 1:30 to 3:30 p.m., in my office in the computer science building. I'm also available by arrangement at other times, though in general not early in the morning.

(4 Feb) I have posted the first homework assignment, which is due on 21 February, two weeks from Wednesday. You should start on the earlier problems this week so you know what kind of questions to ask me and what parts of the CMPSCI 250 material you need to review. The problems are taken from the book, which you should have by now. Note that the book has both Exercises and Problems, and the homework consists entirely of Problems. Solutions to the Exercises are included in the book.

(2 Feb) The parts of the textbook that have been written, chapters 6, 7, 9, and 10, should now be available at Collective Copies in downtown Amherst for $30. (I get none of this money.) Text drafts for chapters "P" and "I" will be made available when I have written them.

(2 Feb) I will post the first homework assignment over the weekend, to be due on Wed 21 Feb in discussion (so we can discuss it as part of our review for the first test on 23 Feb).

(2 Feb) For the time being our meeting rooms will stay the same, LGRC 203 for the 11:15 classes and LGRT 202 for the Wednesday discussion. I have asked to move the Wednesday lecture to LGRT 202, but the Monday and Friday lectures must stay where they are because another class is in LGRT 202 then.

(2 Feb) Don't forget the First Friday meeting, a general meeting for majors in and friends of computer science, at 3:30 today in room 151 of the computer science building.

(2 Feb) As I mentioned on Wednesday, the Discussion #1 assignment was taken from a Fall 2003 CMPSCI 311 discussion and the solutions are also posted.

(25 Jan) The syllabus and requirements pages are now ready.

(24 Jan) I am setting up the skeleton of the web site today, with at least the syllabus. Our first class meeting is on Monday, where I will go through an overview of the course content and the logistics. We will have a discussion meeting on Wednesday 31 January, where we will practice some induction proofs. Our current enrollment is four -- I will make one last sales pitch for more students this week.

Last modified 16 May 2007