CMPSCI 251 (291A): Mathematics of Computation
David Mix Barrington
This is the home page for CMPSCI 251, known officially to the university
as CMPSCI 291A. CMPSCI 251 is a second course in the mathematics of
computation, following CMPSCI 250. In the new computer science curriculum
now in development, CMPSCI 251 will be a required course, coming before the
algorithms course, CMPSCI 311. In the current curriculum it may be used as
the elective "fifth mathematics course" -- for further clarification speak
to me or to your computer science academic advisor.
The subject matter of CMPSCI 251 divides into four parts:
- A review of combinatorics and induction
- Probability and probabilistic reasoning
- Information theory, including error-correcting codes, and
- Models of computation: Finite-state machines and Turing machines
Some of the material of CMPSCI 251 has been in CMPSCI 250 when I have been
the instructor, and some overlaps with CMPSCI 311 and CMPSCI 401. It will not
replace either of those courses, but should provide useful preparation for them.
Instructor Contact Info:
David Mix Barrington, 210 CMPSCI
building, 545-4329, office hours Tuesday and Thursday 1:30-3:30,
or by appointment.
I generally answer my email fairly
TA Contact Info: There is no TA for this course. I get to grade
The only formal prerequisite is CMPSCI 250 or equivalent, but 250 expects
MATH 132 (Calculus II) and CMPSCI 187 (programming in data structures using
Java) as corequistites, so 251 will expect familiarity with this material as
well. Calculus will come up occasionally but will be treated gently.
The course will meet for three lecture meetings a week, MWF 11:15-12:05
in room 203 of Lederle GRC (the low-rise). There will be one discussion
meeting per week, Wed 10:10-11:00 in room 202 LGRT (the Lederle tower).
Discussion sessions will consist of students solving specific
problems alone or in groups and handing answers in, so missing any of these
classes will incur a grade penalty. The three-credit course thus has four
contact hours per week -- I will try to compensate for this (and for the fact
that the course is elective rather than core) by cutting back on the homework
and pace of material relative to those of the eventual core course version.
There is no published textbook for this course -- I have a draft book in
development that serves the first and fourth quarters pretty well, and I will
make this text available. For the second and third quarters of the course you
will get the best text I have produced at the time -- something between
detailed lecture notes and the final book sections.
Announcements (16 May 2007):
- (16 May) The solutions to the
practice final are posted. See you tomorrow at 10:30 in room A205 LGRC.
I will be answering email tonight.
- (11 May) The practice final is posted.
- (3 May) I have posted the notes from yesterday's
discussion. I still owe you notes for Discussions four through 9.
- (27 Apr) I have posted the fourth and final
homework assignment, due on Monday 13 May.
- (27 Apr) Our final exam has been scheduled for 10:30-12:30 on Thursday
17 May, in room A205 LGRC. Please let me know as soon as possible about any
- (23 Apr) I have posted the third midterm
and its solution. The exams were returned in
class today -- the scale was A=92 and C=57, and the scores in sorted order were
92, 82, 75, 71, 65, 57, and 56.
- (23 Apr) The final exam doesn't seem to have been scheduled yet -- I'm
working on it and will let you know when this is fixed.
- (19 Apr) I've corrected an error in the solution of Question 5 (e)
in the third practice exam solutions. Thanks
to Thijs for catching this, I had Pr(B|A) at 16/40 instead of 16/41. See you
in the morning...
- (18 Apr) I've now posted solutions
to the third practice midterm, which we may discuss in class later this
- (16 Apr) No one has asked me any questions about the homework, which is
a little worrying, but here are some hints in any case:
- Problem I.2.2: For the first problem, where you know whether the single
bad coin is heavier or lighter, you should be able to find an exact
number of weighings, depending on n, that is both necessary and sufficient to
find the bad coin. The second problem is more complicated. For n=1, and n=2,
explain why it is impossible to find the bad coin by only weighing the
coins you are given. For n=3, show that you can do it in two weighings but
not one. You cannot do n=4 in two weighings (this is more complicated to show)
but in general for n=3 you should be able to explain how to do it with one more
weighing than you needed for the first problem. You don't need to prove the
lower bound (that you can't do better than this) as it looks pretty complicated.
- Problem I.3.1: It's possible to do this by strong induction, but it's
easier (though not trivial) if you use the Theorem from the section about the
result of the Huffman algorithm. Show that if a tree doesn't have its
leaves on two adjacent levels, it isn't optimal and thus can't be the result
- Problem I.4.2: Use induction on k, the Third Counting Problem, and
- Problem I.4.5: Use induction on the number of leaves. A useful lemma
here is to show that if I have a discrete random source where the two smallest
weight letters do not have the same probability, the entropy of the
source is smaller than the entropy that the source would have if those two
leaves were the same. (That is, if the smallest two weights in X are p and
q with p > q, make a new source X' where those leaves both have weight
(p+q)/2 and show that H(X') < H(X).
- Problem I.5.2: Here the "o(1)" refers to the limit as p approaches zero.
- Problem I.7.3: This is a bit complicated. To make it simpler, first look
at just the case of n=2. Show that if Y consists of two outputs Y1
and Y2, that I(W,Y) ≤ I(W,Y1) + I(W,Y2).
(You can use Jensen's Inequality to show that I(W,Y) - I(W,Y1) -
I(W,Y2) ≤ 0. The fact that the channel is memoryless means that
Pr(y|x) = Pr(y1|x1)Pr(y2|x2).)
- (13 Apr) Sorry I've gotten behind in updating this site. I have just
posted the practice test for next Friday's
midterm. The current homework is due on Wednesday
18 April (sorry for the errors in the 29 March entry, which I have corrected).
I'll hand out solutions to HW#3 in discussion on Wednesday.
Also remember that Monday is a holiday and that Tuesday is a Monday. So
we will have lecture on Tuesday (about DFA's, as the syllabus says). I will
also have my Tuesday office hours even though it's a Monday schedule.
- (29 Mar) I have posted
HW#3 (not HW#2),
which is due on
Wednesday 18 March.
- (18 Mar) I have posted the second exam and
its solution. I have graded the exam and
HW#2 and I will mail each of you individually with your grades. The scale on
the second exam was A = 87 and C = 57, and the individual scores were 95, 81,
65, 63, 61, 46, and 41, the last two being below a passing standard. The scale
on HW#2 was A = 70, C = 35, and the individual scores were 58, 55, 49, 38, 16,
12, and 10, the last three being below a passing standard. There were some
difficult problems on HW#2, but several of you blew it off and shouldn't expect
to pass this course without working harder and starting the homework earlier
. It's not unreasonable to spend 5-10 hours on one of these assignments,
as there are only four for the term and they are an essential way to prepare for
- (14 Mar) I have posted the solutions for
the second practice exam and the notes for Discussion
#2 and Discussion #3. We'll see whether I get
Discussion #4 and #5 notes up before the exam -- I will do #5 first because
everyone pretty much got #4 in class. See you later this morning...
- (13 Mar) A note in the extra-credit question of
HW#2, Problem P.5.3. The problem is not clearly stated.
As in Problem P.5.2 above it, I am looking for the smallest number k such that
the chance of k random people having some pair of birthdays g days apart is
more than 50%. The problem asks for g to be 1, 2, 3, 4, and 5, but I will give
the full ten points extra credit for the case of g=1. This means that two of
the k people have birthdays on the same day or on adjacent days (including 31
December and 1 January). Here is a hint -- you need to find the number of
binary strings of length 365 that do not have two ones in a row and do not
both begin and end with 1. This can be solved using the same sort of idea as
our solution of the Fourth Counting Problem. If you get correct answers for
the cases of larger g as well, I am willing to give more than ten extra credit
- (11 Mar) I am withdrawing the last question of
HW#2, Problem P.8.5, because as it is written it is not
clear how to answer it (I'll discuss this in the exam review Wednesday morning).
So HW#2 has only 90 points plus ten extra credit.
- (9 Mar) The second practice exam is posted.
I hope to hand out solutions to the Chapter P exercises in the book on Monday.
I plan to hand out solutions to HW#2 when you hand HW#2 in at Wednesday's
- (28 Feb) The HW#2 assignment is posted. I see that
I'm behind on posting discussion notes, and I'll try to catch up soon.
- (28 Feb) Graded HW#1 was returned in class today. The scale was 80 for
an A and 45 for a C, and the seven individual grades were 95, 82, 69, 65, 55,
48, and 31.
- (27 Feb) I've updated the syllabus to match
some changes I made while writing Chapter P. The material of Sections P.3 and
P.4 also appears for the most part in Section 7.9 of the bound portion of the
book that you have.
- (26 Feb) The first midterm and its
solution are now available. The seven
scores on the test were 89, 86, 83, 76, 73, 61, and 60. The scale was 90
for the middle of the A range, 75 for B, and 60 for C. I apologize again
for the error in Question 7 -- I dealt with this by being generous with
partial credit on that question and lowering the scale from the A=93, C=63 I
By Wednesday I should have HW#1 graded and have the HW#2 assignment ready.
HW#2 will be due two weeks from Wednesday, 14 March, in discussion class. The
second midterm will be in class Friday 16 February, the day before spring
break. It will cover chapter "P" of the textbook, which I handed out in draft
form today. (Note that this draft does not have a glossary or answers to the
exercises -- I may fix this by the time I give you Chapter I, which will be
at the end of break.)
- (21 Feb) I have to miss the second half of my office hours
tomorrow because I am meeting with a committee of outside evaluators of the
department. I will be in my office 1:30-2:30, and will read and answer
email late in the evening.
- (20 Feb) The solution to the first
practice exam is posted. I have also answered one
question on HW#1.
- (16 Feb) I have posted the practice exam
for next Friday's midterm. I'll post solutions on Tuesday. On Wednesday we
will have a review for the exam during the discussion period, then a lecture
on sections 7.6 and 7.7 (which won't be covered on the exam). Remember that
HW#1 is due Wednesday in discussion -- I will answer email over the long
weekend and will have my normal office hours on Tuesday.
- (14 Feb) We are snowed out today, so there will be no discussion
or lecture. We won't make up the discussion, but I'll post the exercise we
would have done soon. On Friday we will talk about both sections 7.5 and 7.6,
and by Friday I will post a practice exam for the first midterm (which is a
week from Friday). Enjoy the snow!
- (6 Feb) As I mentioned in class Monday, I have adjusted the
syllabus to remove the material on Catalan numbers I was going
to cover on Friday. I would like to make sure by the end of this week that we
are all solid on the combinatorics material of sections 6.1-6.7. This might
require some of you to work pretty hard this week, to catch up from where you
started the course.
- (6 Feb) My primary office hours for this course will be Tuesdays and
Thursdays from 1:30 to 3:30 p.m., in my office in the computer science building.
I'm also available by arrangement at other times, though in general not early in
- (4 Feb) I have posted the first homework
assignment, which is due on 21 February, two weeks from Wednesday. You
should start on the earlier problems this week so you know what kind of
questions to ask me and what parts of the CMPSCI 250 material you need to
review. The problems are taken from the book, which you should have by now.
Note that the book has both Exercises and Problems, and the homework consists
entirely of Problems. Solutions to the Exercises are included in the book.
- (2 Feb) The parts of the textbook that have been written, chapters 6, 7,
9, and 10, should now be available at Collective Copies in downtown Amherst for
$30. (I get none of this money.) Text drafts for chapters "P" and "I" will
be made available when I have written them.
- (2 Feb) I will post the first homework assignment over the weekend, to be
due on Wed 21 Feb in discussion (so we can discuss it as part of our review for
the first test on 23 Feb).
- (2 Feb) For the time being our meeting rooms will stay the same, LGRC 203
for the 11:15 classes and LGRT 202 for the Wednesday discussion. I have asked
to move the Wednesday lecture to LGRT 202, but the Monday and Friday lectures
must stay where they are because another class is in LGRT 202 then.
- (2 Feb) Don't forget the First Friday meeting, a general meeting for majors
in and friends of computer science, at 3:30 today in room 151 of the computer
- (2 Feb) As I mentioned on Wednesday, the Discussion #1 assignment was
taken from a Fall 2003 CMPSCI 311 discussion
and the solutions are also posted.
- (25 Jan) The syllabus and
requirements pages are now ready.
- (24 Jan) I am setting up the skeleton of the web site today, with at least
the syllabus. Our first class meeting is on Monday, where I will go through
an overview of the course content and the logistics. We will have a
discussion meeting on Wednesday 31 January, where we will practice some
induction proofs. Our current enrollment is four -- I will make one last sales
pitch for more students this week.
Last modified 16 May 2007