UMass Machine Learning and Friends Lunch | Main / Recognizing And Editing Scene Attributes

Abstract: In computer vision it is common to organize visual concepts (e.g. objects, scenes, materials, and actions) into non-overlapping categories. In recent years, attributes (nameable high-level properties) have been explored as an alternative to categorical taxonomies. I will first discuss the SUN attribute database in which images are described by over one hundred attribute labels related to materials, surface properties, lighting, affordances, and spatial layout of scenes. Then I will present more recent work which focuses on "transient attributes" -- dynamic properties of outdoor scenes. We manipulate photos so that they exhibit novel attributes related to weather, season, and lightning. For example, we might change a midday photo to sunset or add snow to a summer scene.

Bio: James Hays is the Manning assistant professor of computer science at Brown University. His research interests span computer graphics, computer vision, and computational photography. His research focuses on using "Internet-scale" data and crowds-sourcing to improve scene understanding and allow smarter image synthesis and manipulation. Before joining Brown, James worked with Antonio Torralba as a post-doc at Massachusetts Institute of Technology. He received a Ph.D. in Computer Science from Carnegie Mellon University in 2009 while working with Alexei Efros. He has a B.S. in Computer Science from Georgia Institute of Technology. James is funded by an NSF CAREER award and gifts from Microsoft, Adobe, Pixar, and Google.