CS 294-131: Special Topics in Deep Learning

Instructors

Trevor Darrell

Dawn Song

Jacob Steinhardt

Teaching Assistants

Samaneh Azadi

Office Hours

Samaneh Azadi: By appointment

Lectures

Time: Monday 4:00-5:30pm

Location: Soda 306

Piazza

Course announcements will be announced through Piazza. If you are in the class, [ sign up on Piazza])(https://piazza.com/class/joy4z1cunad9h).

Syllabus

</td>

Date	Speaker	Topic	Readings	Talk
1/28	Jacob Steinhardt	course overview/security		Slides Video
2/4	Nicolas Papernot	security/privacy	Main Reading: Ensemble adversarial training Scalable private learning with PATE Background Reading: Understanding black-box predictions via influence functions Membership inference attacks against machine learning models	Slides Video
2/11	Justin Gilmer	adversarial examples	Main Reading: Benchmarking Neural Network Robustness To Common Corruptions And Perturbations Adversarial Examples Are a Natural Consequence of Test Error in Noise Background Reading: Motivating the Rules of the Game for Adversarial Example Research	Slides Video
2/18	Academic Holiday	-
2/25	Stefan Wager	causality	Main Reading: Beyond prediction: Using big data for policy problems Quasi-oracle estimation of heterogeneous treatment effects Background Reading: Statistics and Causal Inference Causal Diagrams for Empirical Research	Slides
3/4	Guillaume Bouchard	fake news defense	Main Reading: Relevant Document Discovery for Fact-Checking Articles Combining Similarity Features and Deep Representation Learning for Stance Detection in the Context of Checking Fake News Background Reading: Investigating Rumor News Using Agreement-Aware Search A survey on hate speech detection using natural language processing
3/11	Zachary Lipton	explainability	Main Reading: Detecting and Correcting for Label Shift with Black Box Predictors Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment Background Reading: Improving predictive inference under covariate shift by weighting the log-likelihood function Domain-Adversarial Training of Neural Networks	Video
3/18	Dustin Tran	Bayesian Deep learning	Main Reading: Automatic Differentiation Variational Inference Deep Probabilistic Programming Background Reading: Auto-Encoding Variational Bayes Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models	Slides Video
3/25	Spring Break	-
4/1	Alex Alemi, Ian Fischer	information theory	Main Reading: Uncertainty in the Variational Information Bottleneck The Conditional Entropy Bottleneck Background Reading: Deep Variational Information Bottleneck Fixing a Broken ELBO The Information Bottleneck Method Emergence of Invariance and Disentanglement in Deep Representations	Video
4/8	Been Kim	explainability	Main Reading: Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) Sanity Checks for Saliency Maps Background Reading: Towards A Rigorous Science of Interpretable Machine Learning To Trust Or Not To Trust A Classifier	Video
4/15	Balaji Lakshminarayanan	uncertainty	Main Reading: Simple and scalable predictive uncertainty estimation using deep ensembles Do deep generative models know what they don't know? Background Reading: Chapter 1 of Yarin Gal's PhD thesis "Uncertainty in deep learning" Introduction: The Importance of Knowing What We Don’t Know On Calibration of Modern Neural Networks	Video Slides
4/22	Noah Goodman	Pyro	Main Reading: Multimodal Generative Models for Scalable Weakly-Supervised Learning. Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inference Background Reading: Pyro: Deep Universal Probabilistic Programming	Video
4/29	Poster Presentation

Course description

In recent years, deep learning has enabled huge progress in many domains including computer vision, speech, NLP, and robotics. This class is designed to help students develop a deeper understanding of deep learning and explore new research directions and applications of AI/deep learning and privacy/security. It assumes that students already have a basic understanding of deep learning. In particular, in this semester, we will focus on a theme, trustworthy deep learning, exploring a selected list of new, cutting-edge topics including security and privacy issues in deep learning, explainability, generalization, reliability and robustness, fairness, causality, and theoretical understanding.

Class format and project

This is a lecture, discussion, and project oriented class. Each lecture will focus on one of the topics, including a survey of the state-of-the-art in the area and an in-depth discussion of the topic. Each week, students are expected to complete reading assignments before class and participate actively in class discussion.

Students will also form project groups (two to three people per group) and complete a research-quality class project.

Enrollment information

For undergraduates: Please note that this is a graduate-level class. However, with instructors’ permission, we do allow qualified undergraduate students to be in the class. If you are an undergraduate student and would like to enroll in the class, please fill out this form and come to the first lecture of the class. Qualified undergraduates will be given instructor codes to be allowed to register for the class after the first lecture of the class, subject to space availability.

If you have not received grades for some classes that you are currently enrolled in, please choose Currently Enrolled and then update the form when you receive your final grades. You may also be interested in this class, which is open to undergraduates.

Students may enroll in this class for variable units.

1 unit: Participate in reading assignments.
2 units: Both reading assignments and a project. Projects may fall into one of four categories:
- Distill-like Literature Review of a deep learning topic (e.g., a Distill-like blog post illustrating different optimization techniques used in deep learning)
- Reimplement research code and open source it
- Conference level research project
You may not take this class for 3 or 4 units.

Deadlines

Reading assignment deadlines:
- Submit questions about the reading material by Sunday noon.
Project deadlines:
- 2/25: Project proposal due
- 4/1: Project milestone report due
- 4/29: Poster presentation
- 5/6: (tentative) Final project report due

Grading

20% class participation
25% weekly reading assignment
55% project

Additional Notes

For students who need computing resources for the class project, we recommend you to look into AWS educate program for students. You’ll get 100 dollar’s worth of sign up credit. Here’s the link .

CS 294-131: Trustworthy Deep Learning (Special Topics in Deep Learning)

Spring 2019