UMass Machine Learning and Friends Lunch | Main / Learning To Generate Images

Abstract: Deep learning has revolutionized the field of visual recognition. Since 2012, we witnessed an enormous jump in recognition performance on the standard benchmarks as well as many real-world applications. Meantime, many people in computer vision and graphics were wondering if deep learning can help visual synthesis. Unfortunately, it turned out that using deep neural networks to generate high-dimensional data such as images was extremely difficult. In this talk, I will discuss its main challenges and present a few end-to-end learning frameworks (e.g., pix2pix, CycleGAN, pix2pixHD) for generating and manipulating natural images. Then, I will show various applications such as generating synthetic training data (computer vision), photo manipulation and synthesis (computer graphics), converting MRIs into CT scans (medical imaging), and applications in NLP and speech synthesis. Finally, I will briefly discuss our ongoing efforts on learning to synthesize 3D textured objects and high-res videos, with the ultimate goal of recreating our visual world.

Bio: Jun-Yan Zhu is a postdoctoral researcher at MIT CSAIL. He obtained his Ph.D. in Electrical Engineering and Computer Sciences from UC Berkeley in 2017 after spending five years at CMU and UC Berkeley. He received his B.E in Computer Sciences from Tsinghua University in 2012. His research interests are in computer vision, computer graphics, and machine learning, with the goal of building machines capable of understanding and recreating our visual world. His Ph.D. work was supported by a Facebook Fellowship. His dissertation won the 2018 ACM SIGGRAPH Outstanding Doctoral Dissertation Award from SIGGRAPH and 2017-18 David J. Sakrison Memorial Prize for outstanding doctoral research from the UC Berkeley EECS Department. He has served as a Technical Paper Committee member at SIGGRAPH Asia 2018 and a guest editor of International Journal of Computer Vision.

See more details at http://people.csail.mit.edu/junyanz/

The talk can be accessed using this link : https://youtu.be/0jVcWgTxfTc