CS691A: Graduate Computer Vision

• Fall 2011; Monday and Wednesday, 10:35-11:50 • in CS 140


Erik Learned-Miller
elm at cs.umass.edu
(413) 545-2993

Teaching Assistant: None
Draft Syllabus: syllabus.pdf



1. Introduction to Computer Vision

2. A review of basic probability.

3. Supervised learning and Bayesian classification

4. Entropy and mutual information

5. Image Alignment, by Rick Szeliski

6. Lightness Perception and Lightness Illusions

7. Unsupervised learning in vision.


MATLAB Tutorial

Matlab Diary for Lecture on Sep. 7

Interesting Links

The Necker Cube illusion

Checker shadow illusion

Movie on optical illusions

Early color photographs by S. M. Prokudin-Gorsky


Date Lecture topic New assignments Assignments due Reading
Sep. 7 UNIT 1: Introduction. What is Computer Vision? What are the goals of computer vision? What should we take from humans? Also, Intro to Matlab. See Matlab Diary of in class session under "Resources" above. Assignment 1: Probabilistic classification.

Digit data for assignment 1

Due by midnight on Sept 14. Please email me the solution as a zipped tar file of all necessary files.

Readings 1, 2, and 3 from the "Readings" list above by Monday, September 12.
Sep. 12 UNIT 2: Probability, Statistics, and Learning Basics. Review of basic discrete probability. Samples spaces. Events. Joint Probability. Conditional Probability. Marginalization. Role of Probability and Statistics in Computer Vision. Bayes rule. Likelihoods, priors, and posteriors. Estimating likelihoods, priors, and posteriors.

Sizes of sets. Number of different images. Number of images of a person's face. Implications for estimation.

Sep. 14

Continuous probability. Parzen density estimates. In 1 dimension. In 2 dimensions. In N dimensions.

Topology of features. Discrete variables, continuous variables and, "discretized continuous variables". Examples: dice, angles, sets of rotations, pixel brightness values.

Modeling the entire joint distribution of images in a class. Assuming a parametric form. Assuming feature independence. Semi-parametric models. More on estimation and smoothing.

A few more Matlab tricks: dot-m files and Matlab calling conventions, the image toolbox, avoiding for loops, repmat, dot-times, dot-slash, dot-power, etc.

Assignment 1 Due by midnight
Sep. 19 Euclidean distance functions in 1, 2, 3, and more dimensions. Nearest neighbor. K-nearest neighbors. Consistency of K-nearest neighbors. Relationship between density estimation techniques of classification and K-nearest neighbors.

Reading number 4, for next lecture.
Sept. 21 UNIT 3: Alignment Prokudin-Gorski photographs. Exhaustive search vs. gradient methods. Correlation alignment. Mutual information alignment. Other schemes of aligment. Problems of alignment and solutions to alignment problems.

Lecture slides

Assignment 2: Automatic alignment of Prokudin-Gorsky plates.


Due by midnight on Sept. 28.

Reading number 5, SECTIONS 2.0, 2.1, 3.0, 3.1
Sept. 26 Some "low level" vision problems (tracking, backgrounding, optical flow, stereo, etc.). The core matching problem: aligning Patch J to Image I. Families of transformations. Translations. Rigid. Similarity. Affine. Linear. Homographies or perspective. Diffeomorphisms. Implementing transformations as "looking back" to original image using transform inverse.

Gradient descent as a method for solving the core matching problem.

Lecture slides

Sep. 28 Alignment continued. Continue discussion of gradient descent to solve core matching problem. Analytic gradient descent. Gradient descent using discrete approximation to the gradient. Gradient descent with respect to translation. Add rotation. Add other parameters. How to pick epsilon for approximation of partials. How to pick delta for step size.

Problems with optimization. The zero gradient problem. Local minima. Techniques for getting around local minima. Smoothing. Image pyramids. Problems with image pyramids.

Reading for next lecture: Data Driven Models through Continuous Joint Alignment.

A possible solution to local minima in alignment problems: joint alignment. Congealing. The minimum entropy criterion. Non-parametric maximum likelihood. Joint gradient descent (or joint coordinate descent). Smoothing of the optimization landscape without destroying information.

Distributions fields: "congealing without all the images".

Lecture slides

Assignment 3: Congealing implementation, due on Oct 11th (Tuesday), 11:59pm.

Oct. 5 Congealing continued.

Lecture slides

Oct. 11 (TUESDAY) Distribution fields. Exploding an image. Convolving with a Gaussian. Basin of attraction with distribution fields. Likelihood match. Sharpening match.

Lecture slides

Oct. 12 UNIT 4: Light, optics, and human vision. Electromagnetic spectrum. Multi-frequency nature of light.

Assignment 4: Video stabilization, due on Oct 26th (Wednesday), 11:59pm.
Oct. 17 Slides Basic anatomy of the eye. Anatomy of the retina. Rods and cones. Low light and normal light vision. Dynamic range of the eye. Frequency response of rods and cones. Distribution of rods and cones. Blind spot. Pupillary adjustment.

Oct. 19 Tristimulus theory of color. Color matching experiments. Linearity of color perception. Total response as the dot product of cone sensitivity curves and spectral power distrbution curves.

Oct 24 Point sources, extended sources. Radiance, irradiance, luminance, illuminance, brightness.

Handout on radiometry.
Oct 26. Slides Pinhole cameras. Bidirectional reflectance distribution functions. Lambertian surfaces. Specular surfaces. Classifying surfaces using properties related to BRDF. .

Oct 31. SNOW DAY!

Nov. 2 Slides CCDs. Grayscale vs. color. Image splitting, vs. Bayer pattern. Interpolating missing pixels in detector. Tradeoff between resolution and light sensitivity. Tiling patterns for CCDs. Bits per pixel. Linear digitization vs. other schemes. Video cameras and synchonicity vs. integrate and fire.

Nov. 7. UNIT 5: Features

Motivation for using features other than raw pixel values. Independence, informativeness, mutual information, data compression. Edge features.

Project ideas

Nov. 9 Processing by the retina. Derivatives and center surround. The role of decorrelation and independence in signal processing, compression, and meaning. SIFT features.

Read SIFT paper:http://www.cs.ubc.ca/~lowe/keypoints/

"David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004), pp. 91-110."

Nov. 14 More on SIFT. Using higher order features for classification.

Nov. 16 NO CLASS!!! friday schedule.

Nov. 21 FACE UNIT: (finish SIFT) Start Viola-Jones face detection.

Nov. 23 Exam review. Exam Review document.

Nov. 28 Viola Jones Face detection. Viola-Jones face detector slides.


Dec. 5 Project presentations.

Dec. 7 Project presentations.