Divide and Conquer

• Idea
  1. Divide the problem into a number of smaller subproblems
  2. Conquer the subproblems by solving them recursively
  3. Combine the solutions to the subproblems into the solution of the original problem
• Leads naturally to recursion
  • Recursive case: solving subproblems recursively when they are big enough
  • Base case: solving a small problem directly

Recurrences

• Result directly from divide-and-conquer algorithms
• Recall Merge-Sort

  T(n) = {	Θ(1)	if n = 1
  	2T(n/2) + Θ(n)	if n > 1

Solving Recurrences

• Substitution method: guess a bound and use mathematical induction to prove it is correct
• Recursion-tree method: convert the recurrence into a tree whose nodes represent costs incurred at various levels of the recursion
• Master method: recipe for solving recurrences of the form

  T(n) = aT(n/b) + f(n)

  where a ≥ 1, b > 1, and f(n) is a given function

• Note: we often ignore certain technicalities

Volatile Chemicals

Given stock prices into the future, allowed to make one purchase and one sale, maximize profit

Volatile Chemicals

Given stock prices into the future, allowed to make one purchase and one sale, maximize profit

What would be a brute-force method?

Maximum-subarray Problem

• Define a new array of size n−1 of daily changes
• Find the contiguous subarray with the largest sum

Will this solve the same problem?

Observation

Any contiguous subarray A[i..j] of A[low..high] must lie in exactly one of the following places:

• entirely in the subarray A[low..mid], so that low ≤ i ≤ j ≤ mid,
• entirely in the subarray A[mid+1..high], mid < i ≤ j ≤ high, or
• crossing the midpoint, so that low ≤ i ≤ mid ≤ j ≤ high

Find Max Crossing Subarray

   Find-Max-Crossing-Subarray (A, low, mid, high)

   1  left-sum = −∞
   2  sum = 0
   3  for i = mid downto low
   4    sum = sum + A[i]
   5    if sum > left-sum
   6       left-sum = sum
   7       max-left = i

   8  right-sum = −∞
   8  sum = 0
   10 for j = mid+1 to high
   11   sum = sum + A[j]
   12   if sum > right-sum
   13      right-sum = sum
   14      max-right = j
   15 return (max-left, max-right, left-sum + right-sum)
	

Maximum Crossing Subarray: Example

Find Maximum Subarray

   Find-Maximum-Subarray (A, low, high)

   1  if high == low
   2    return (low, high, A[low])
   3  else mid = floor((low+high)/2)
   4    (left-low, left-high, left-sum) = 
           Find-Maximum-Subarray(A,low,mid)
   5    (right-low,right-high,right-sum) =
           Find-Maximum-Subarray(A,mid+1,high)
   6    (cross-low,cross-high,cross-sum) =
           Find-Crossing-Subarray(A,low,mid,high)
   7  if left-sum ≥ right-sum and left-sum ≥ cross-sum
   8    return (left-low, left-high, left-sum)
   9  elseif right-sum ≥ left-sum and right-sum ≥ cross-sum
   10   return (right-low, right-high, right-sum)
   11 else return (cross-low, cross-high, cross-sum)
	

Matrix Multiply

  Square-Matrix-Multiply (A, B)
  1  n = A.rows
  2  let C be a new n×n matrix
  3  for i = 1 to n
  4    for j = 1 to n
  5      cij = 0
  6      for k = 1 to n
  7        cij = cij + aik.bkj
  8  return C
	

What is the time complexity?

Answer: Θ(n³)

Can we do better?

Multiplying using Sub-matrices

Recursive Version

  Square-Matrix-Multiply-Recursive (A, B)
  1  n = A.rows
  2  let C be a new n×n matrix
  3  if n == 1
  4    c11 = a11.b11
  5  else partition A, B, and C into A11, A12, A21, A22, etc.
  6    C11 = Square-Matrix-Multiply-Recursive(A11, B11)
          + Square-Matrix-Multiply-Recursive(A12, B21)
  7    C12 = Square-Matrix-Multiply-Recursive(A11, B12)
          + Square-Matrix-Multiply-Recursive(A12, B22)
  8    C21 = Square-Matrix-Multiply-Recursive(A11, B11)
          + Square-Matrix-Multiply-Recursive(A22, B21)
  9    C22 = Square-Matrix-Multiply-Recursive(A21, B12)
          + Square-Matrix-Multiply-Recursive(A22, B22)
  10 return C
	

Recursive Matrix Multiply Running Time

Strassen's (Straßen's) Method

Idea:

• Set up some intermediate matrices, to reduce the number of recursions from 8 to 7
• Add some Θ(n²) work to remove one recursive call

S Sub-matrices

P Sub-matrices

Compute C using P Sub-matrices

Recurrences: We Focus on Master Method

• Substitution method: guess a bound and use mathematical induction to prove it is correct
• Recursion-tree method: convert the recurrence into a tree whose nodes represent costs incurred at various levels of the recursion
• Master method: recipe for solving recurrences of the form

  T(n) = aT(n/b) + f(n)

  where a ≥ 1, b > 1, and f(n) is a given function

Theorem: Master Theorem

Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and let T(n) be defined on the nonnegative integers by the recurrence:

T(n) = a T(n/b) + f(n)

1. If f(n) = O(n^log_ba−ε) for some constant ε > 0, then

   T(n) = Θ(n^log_ba)

2. If f(n) = Θ(n^log_ba), then

   T(n) = Θ(n^log_ba log n) = &Theta(f(n) log n)

3. If f(n) = Ω(n^log_ba+ε), for some constant ε > 0, and
   if a f(n/b) ≤ c f(n) for some constant c < 1 and all sufficiently large n then

   T(n) = Θ(f(n))

Master Method: Important Special Case

An important special case is when f(n) = c^k for some k. In other words, f(n) is a polynomial function.

It turns out that the “regularity condition” (the extra condition for case 3) always holds when f(n) is a polynomial. So there is no need to check for it explicitly.

How would you prove this?

Final Remarks

• The best known square matrix-multiply algorithm to date is O(n^2.376), by Coppersmith and Winograd
• The lower bound on matrix-multiply is Ω(n²) Why?
• Reasons for not using Strassen's algorithm, in practice
  • larger constant factor hidden in O(n^{log 7}) than the constant factor in O(n³) in standard square matrix multiply
  • Specialized sparse matrix algorithms are faster
  • Numerically less stable
  • Extra space, and consequently poorer cache behavior