CMPSCI 251: Mathematics of Computation

David Mix Barrington

Spring, 2007

Discussion Notes #3

from Wednesday 14 February 2007

Proofs with Big-O Notation

In lecture this week we gave formal definitions of the big-O and big-Omega notation. Here f and g are functions from N to the reals, ε is a positive real, and n and n₀ are naturals.

• f = O(g) &Harr; ∃c:∃n₀∀n:(n≥n₀) → f(n) ≤ cg(n)
• f = Ω(g) &Harr; ∃ε:∃n₀:∀n: (n ≥ n₀) → f(n) ≥ εg(n)
• f = Θ(g) &Harr; f = O(g) ∧ f = Ω(g)

Questions are in black, answers in blue.

In this discussion we will use these definitions, and the proof rules for quantified statements from CMPSCI 250, to prove some of the properties of big-O and big-Omega. (Anyone who took 250 from me should remember the rules of Existence, Instantiation, Specification, and Generalization -- I'll go over these on the board as needed.)

• Question 1: Prove that if 0 < α < β, then n^α = O(n^β) but n^β ≠ O(n^α).

  To show the first statement, we can let c=1 and n₀ = 1, then because α < β implies n^α ≤ n^β for positive n, we are done. If the second statement were true for any c and any n₀, for all large n we would have n^β ≤ cn^α. But this can't be true for all n, because for large enough n, n^β-α will be bigger than any fixed c.

• Question 2: Prove that if f = Θ(1), then f² = Θ(f). You probably want two separate proof, one for f² = O(f) and one for f² = Ω(f).

  To show f² = O(f), pick c and n₀ from the fact that f = O(1) so that n ≥ n₀ implies f(n) ≤ c. Then for the same c and n₀, n ≥ n₀ implies that f²(n) ≤ cf(n).

  To show f² = Ω(f), pick ε and n₀ from the fact that f = Ω(1) so that n ≥ n₀ implies f(n) ≥ ε. Then for the same ε and n₀, n ≥ n₀ implies f²(n) ≥ εf(n).

• Question 3: Prove that log(n!) = Θ(n log n). Like Question 2, this naturally breaks into two parts. It is useful to express log(n!) as a sum.

  The sum in question is that log(n!) is the sum for i from 1 to n of log(i). To see that log(n!) = O(n log n), note that this sum is ≤ n log n because each term is ≤ log(n). To see that log(n!) = Ω(n log n), note that the sum is ≥ (1/4)(n log n) because the second half of the terms are each bigger than (1/2)(log n) (as long as n is at least 4).

• Question 4: Put the following functions in order by growth rate, so that if f is before g, then f = O(g). Indicate which functions are also related by big-Omega and thus by big-Theta:
  • n²log n, 6n log⁵ n, n^2.01, n^1/2+17, 27n+35, (n^1/2+17)², (n+17)^1/2, 3 + sin n.
  • smallest is 3 + sin n, which is Θ(1)
  • next are n^1/2+17 and (n+17)^1/2, which are both Θ(n^1/2)
  • next are 27n+35 and (n^1/2+17)², which are both Θ(n)
  • next is 6n log⁵n
  • next is n²log n
  • largest is n^2.01

Last modified 14 March 2007