Projects

Texture understanding: This project investigates various aspects of texture understanding. In the representation side of things we have looked into the connection between classical orderless pooling representations (e.g., bag-of-words models) and deep architectures (e.g., CNNs). This led to a new class of deep architectures (e.g., Fisher vector CNNs, bilinear CNNs, etc.) that generalize better than prior architectures on tasks where texture cues are useful such as fine-grained species, material, and scene classification. Here is my recent talk at the manifold learning workshop at ICCV 2017. Below are links to individual projects.

• Bilinear CNNs and their applications
• Second-order democratic pooling
• Deep filter banks for texture understanding
• Naming and describing textures
• Visualizing texture representations

3D shape understanding: 3D data is becoming common with the growth of consumer depth scanners and the increasing number of gaming and virtual/augmented reality applications. We are developing a number of fundamental techniques to improve the creation and organization of 3D shape data. Take a look at this recent talk for an overview of our work on 3D shape generation. Below are links to individual projects.

• Multiresolution tree networks (MRTNet)
• Neural constructive solid Geometry (CSGNet)
• Sparse Lattice Networks (SPLATNet)
• Unsupervised 3D shape learning
• Inferring 3D shapes from sketches
• Multiview convolutional networks
• Shape PFCNs
• Point Cloud Generation (PCAGAN)

Dark Ecology: This project will leverage large-scale cloud computing and develop novel computer vision, machine learning, and radar analysis methods to measure the densities and velocities of migrating birds across the US. Deep convolutional networks will be trained to discriminate migrating birds from precipitation and other clutter in the radar data. New techniques for domain transfer and weakly supervised training will enable the training of convolutional networks with only modest-sized training sets. Gaussian process (GP) models will be developed to create smooth national maps of migration density and velocity. Novel GP methods and cloud-computing workflows will allow us to scale to massive radar data sets and analyze the more then 200 million archived radar scans. The resulting data and tools will be curated with open access policies, and used by the research team to conduct ecological research about patterns and drivers of continent-scale migration.

Link to the project page.

Other active projects:

• Domain adaptation and transfer learning
• Learning and inference with humans in-the-loop [ICML 16 workshop, CVPR 15, WACV 15, CVPR 14]
• Describing objects using parts and attributes, a survey

Past projects:

• Objects in detail
• Efficient additive kernel SVMs
• Object detection: poselets, hough transform
• Semantic segmentation
• Pose estimation and attribute recognition
• Perturb and MAP for learning and inference
• Collecting annotations efficiently using Mechanical Turk