CMPSCI 251: Mathematics of Computation

Practice Exam for Second Midterm

David Mix Barrington

13 April 2007

Directions:

• Answer the problems on the exam pages.
• There are six problems on pages 2-7, for 100 total points. Probable scale is A=93, C=63.
• If you need extra space use the back of a page.
• No books, notes, calculators, or collaboration.
• The first four questions are true/false, with five points for the correct boolean answer and up to five for a correct justification.
• Questions 5 and 6 have numerical answers -- remember that logarithms are base 2.

  Q1: 10 points
  Q2: 10 points
  Q3: 10 points
  Q4: 10 points
  Q5: 30 points
  Q6: 30 points
 Total: 100 points

• Question 1 (10): True or false with justification: Let X be a discrete random source. If we construct a variable-length binary code for X using Huffman's algorithm, the average number of bits we need to transmit a letter of X will always be equal to the entropy of X.

• Question 2 (10): True or false with justification: A string is in the intersection of the regular languages (ab)^* and (ba)^* if and only if it is the empty string.

• Question 3 (10): True or false with justification: The Cherokee writing system contains 85 symbols, one for each syllable that occurs in the language. With an approriate coding system, we could transmit any sequence of n syllables (whether it made sense in Cherokee or not) using at most 6n bits.

• Question 4 (10): True or false with justification: Let w be a string of six letters from {a,b,c} where each letter is chosen independently with probability 1/3 of each letter. Then the probability that w has exactly two occurrences of each letter is less than 5%.

• Question 5 (30): Suppose that we send bits over a symmetric binary channel using a (4,3) parity-check code. This means that after every three message bits b₁, b₂ , and b₃, we send a parity bit $p$ that is equal to b₁+b₂+ b₃, using addition modulo 2.

  • (a,5) Recall that a code word is a sequence of four bits that could be a possible valid message. How many code words are there? List them.

  • (b,5) What is the minimum Hamming weight of a nonzero word in this code? What does this imply about the error detection capacity of the code?

  • (c,5) What is the error correction capacity of this code? That is, what is the minimum number of errors that could occur and still allow the receiver to determine the most likely message to have been sent?

  • (d,10) Let the error probability of the channel be 1/3, so that a bit is transmitted correctly with probability 2/3 and incorrectly with probability 1/3. For each i in the set {0,1,2,3,4}, compute the probability of exactly i errors among the four bits. (You may express your answers as fractions.)

  • (e,5) Again assume an error probability of 1/3. Let A be the event that no error is detected (for a single packet of four bits) and B be the event that no error occurred. Compute Pr(B|A) (to within an additive error of +/- 10 percent). Is the coding scheme effective for this channel? Explain your answer.

• Question 6 (30): This problem concerns a channel C that has input alphabet X = {0,1} and output alphabet Y = {0,1}. C always transmits 0's correctly, but transmits 1's as 0's half the time and 1's half the time.

  • (a,10) Suppose we define a discrete random source X such that 0's and 1's are equally likely, and let the random variable Y be the output from sending bits from X through C. Compute the entropies H(X) and H(Y). (You may estimate the base-two log of 3 as 1.6.)

  • (b,10) Compute the joint entropy H(X,Y), the mutual information I(X,Y), and the equivocation H(X|Y) for these two variables. Again assume that 0's and 1's from X are equally likely.

  • (c,10) Assume now that we code bits using triple repetition and send them over this channel. (That is, we send 0's as 000 and 1's as 111, with 000 and 111 being equally likely.) Describe a sensible error-correction scheme for the receiver to use and find the probability (for a random input) that the three-bit packet received is interpreted correctly.

Last modified 13 April 2007