CS 294-131: Special Topics in Deep Learning

Instructors

Trevor Darrell

Sergey Levine

Dawn Song

Teaching Assistants

Richard Shin

Weicheng Kuo

Lectures

Time: Monday 1–2:30 pm

Location: 306 Soda

Office hours

Richard: 3-4 PM on Tuesdays in 723 Soda.

Mailing list and Piazza

To get announcements about information about the class including guest speakers, and more generally, deep learning talks at Berkeley, please sign up for the talk announcement mailing list for future announcements.

For students enrolled in the class, please join the Piazza and the student Google Group.

Syllabus

Date	Speaker	Readings	Talk	Deadlines
1/23	Devi Parikh	No assigned readings for this lecture	Visual Question Answering (VQA)
1/30	Richard Socher	Dynamic Coattention Networks For Question Answering, by Caiming Xiong, Victor Zhong, and Richard Socher A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks, by Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher	Tackling the Limits of Deep Learning for NLP
2/6	Rahul Sukthankar	Speed/accuracy trade-offs for modern convolutional object detectors, by Jonathan Huang et al. Variable Rate Image Compression with Recurrent Neural Networks, by George Toderici et al. Full Resolution Image Compression with Recurrent Neural Networks, by George Toderici et al.	Recent Progress on CNNs for Object Detection and Image Compression
2/13	Bryan Catanzaro	Deep learning with COTS HPC systems, by Adam Coates et al. Deep Speech 2: End-to-End Speech Recognition in English and Mandarin, by Dario Amodei et al.	Scaling Deep Learning	Project proposal
2/20	No class (President’s Day)
2/27	Aaron Hertzmann	Main reading: Image Analogies, by Aaron Hertzmann et al. Learning 3D Mesh Segmentation and Labeling, by Evangelos Kalogerakis et al. Background reading: StyLit (image analogies) Mesh region annotation	Artistic Stylization and Geometry Processing
3/6	Kate Saenko	Main reading: Multimodal Video Description, by Vasili Ramanishka et al. Top-down Visual Saliency Guided by Captions, by Vasili Ramanishka et al. Background reading: Translating Videos to Natural Language Using Deep Recurrent Neural Networks, by Subhashini Venugopalan et al. Show, attend and tell: Neural image caption generation with visual attention, by Kelvin Xu et al.	Attentive Captioning without Attention: Designing and understanding LSTM-based captioning models
3/13	Diedrik Kingma	Main reading: Auto-Encoding Variational Bayes, by Diedrik Kingma and Max Welling Improving Variational Inference with Inverse Autoregressive Flow, by Diedrik Kingma et al. Optional reading: Density estimation using Real NVP, by Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio Conditional Image Generation with PixelCNN Decoders, by Aaron van den Oord et al.	Unsupervised Deep Learning with Variational Autoencoders: Recent Advances and Applications	1st project milestone
3/20	Jianxiong Xiao	Main reading: DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving, by Chenyi Chen et al. Learning from Maps: Visual Common Sense for Autonomous Driving, by Ari Seff and Jianxiong Xiao	Learning Affordance for Autonomous Driving
3/27	No class (spring break)
4/3	Le Song	Main reading: Deep Coevolutionary Network: Embedding User and Item Features for Recommendation, by Hanjun Dai et al. Learning Combinatorial Optimization Algorithms over Graphs, by Hanjun Dai et al. Background reading: Hilbert Space Embeddings of Hidden Markov Models, by Le Song et al. Discriminative Embeddings of Latent Variable Models for Structured Data, by Hanjun Dai et al.	Embedding as a Tool for Algorithm Design
4/10	Alexander Rush	Main reading: Structured Attention Networks, by Yoon Kim, Carl Denton, Luong Hoang, and Alexander M. Rush Background reading: Sequence-to-Sequence Learning as Beam-Search Optimization, by Sam Wiseman and Alexander M. Rush Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks, by Jason Weston et al. Neural Machine Translation by Jointly Learning to Align and Translate, by Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio	Attention, Seq-to-Seq, and Language Structure	2nd project milestone
4/17	Sanjeev Arora	Main reading: Generalization and Equilibrium in Generative Adversarial Nets (GANs), by Sanjeev Arora et al. Background reading: NIPS 2016 Tutorial: Generative Adversarial Networks, by Ian Goodfellow	Generalization and Equilibrium in Generative Adversarial Nets (GANs)
4/24	Bryan Russell	Main reading: ActionVLAD: Learning spatio-temporal aggregation for action classification, by Rohit Girdhar et al. Project information Background reading: Temporal Segment Networks: Towards Good Practices for Deep Action Recognition, by Limin Wang et al. Code NetVLAD: CNN architecture for weakly supervised place recognition, by Relja Arandjelović et al. Project information	ActionVLAD: Learning spatio-temporal aggregation for action classification
5/1	Poster session
5/10	Final project report due

Course description

In recent years, deep learning has enabled huge progress in many domains including computer vision, speech, NLP, and robotics. It has become the leading solution for many tasks, from winning the ImageNet competition to winning at Go against a world champion. This class is designed to help students develop a deeper understanding of deep learning and explore new research directions and applications of deep learning. It assumes that students already have a basic understanding of deep learning. In particular, we will explore a selected list of new, cutting-edge topics in deep learning, including new techniques and architectures in deep learning, security and privacy issues in deep learning, recent advances in the theoretical and systems aspects of deep learning, and new application domains of deep learning such as autonomous driving.

Class format and project

This is a lecture, discussion, and project oriented class. Each lecture will focus on one of the topics, including a survey of the state-of-the-art in the area and an in-depth discussion of the topic. Each week, students are expected to complete reading assignments before class and participate actively in class discussion.

Students will also form project groups (two to three people per group) and complete a research-quality class project.

Please also refer to the course overview slides from the first lecture for more information.

Enrollment information

For undergraduates: Please note that this is a graduate-level class. However, with instructors’ permission, we do allow qualified undergraduate students to be in the class. If you are an undergraduate student and would like to enroll in the class, please fill out this form and come to the first lecture of the class. Qualified undergraduates will be given instructor codes to be allowed to register for the class after the first lecture of the class, subject to space availability.

Students may enroll in this class for variable units.

1 unit: Participate in reading assignments (including serving as discussion lead).
2 units: Complete a project.
3 units: Both reading assignments and a project.

Listing in the Berkeley Academic Guide. Class # is 34001.

Deadlines

Reading assignment deadlines:
- For students,
  - Submit questions by Friday noon
  - Vote on the poll of discussion questions by Saturday 11:59 pm
- For discussion leads,
  - Send form to collect questions from students by Wednesday 11:59 pm
  - Summarize questions proposed by students to form the poll and send it by Friday 11:59 pm
  - Summarize the poll to generate a ranked & categorized discussion question list and send the list to teaching staff by Sunday 7pm
Project deadlines:
- February 13: project proposal due
- March 13: first project milestone report due
- April 10: second project milestone report due
- May 1: poster session
- May 10: final project report due

Grading

20% class participation
25% weekly reading assignment
- 10% discussion leads
- 15% individual reading assignments
55% project

Additional Notes

For students who need computing resources for the class project, we recommend you to look into AWS educate program for students. You’ll get 100 dollar’s worth of sign up credit. Here’s the link .