**CSCI 378: Deep Learning**

###### REED COLLEGE, SPRING 2019

This course is an introduction to deep neural architectures and their training. Beginning with the fundamentals of regression, optimization, and regularization, the course will then survey a variety of architectures and their associated applications. Students will develop projects that implement deep learning systems to perform various tasks. Prerequisites: Mathematics 201, 202, and Computer Science 221.

### BASIC INFO

**Professor:** Mark Hopkins, hopkinsm@reed.edu

**Class Schedule:** MWF 3:10-4pm, Physics 240A

**Office Hours:** MW 4:10-6pm, Th 10am-noon in Library 314 (M 4:10-5 and Th 10-11am are reserved for this class; the remainder are shared with CSCI 121).

**Textbook (optional):** *Deep Learning *by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.

**Syllabus:** downloadable here

### LECTURE NOTES

**Machine Learning: A Whirlwind Guide**(pdf)**Gradient Descent**(pdf)**Gradient Descent: Now in 2D!**(pdf)**A Quick Review of Basic Probability Formulas**(pdf)**Bayesian Networks**(pdf)**How to Read a Bayesian Network**(pdf)**Functional Causal Models**(pdf)**Exponential Distributions**(pdf)**Regression Problems**(pdf)

### LECTURE CODE

https://classroom.github.com/a/h3tOU2km**Introduction to Tensors (Wednesday, Jan. 30):****Solution to Rubik (Monday, Feb. 4):**https://github.com/Mark-Hopkins-at-Reed/csci-378/blob/master/code/rubik/rubik.py

### HOMEWORK AND PROJECT LINKS

https://classroom.github.com/a/WqP7Oqmf**HW1 (“Memoize”, due Friday, Feb. 1):****HW2 (“Rubik”, due Monday, Feb. 4):**https://classroom.github.com/a/yaOkb2-4**Project 1 (“Descent”, due Friday, Feb. 8):**https://classroom.github.com/a/zidPHJ4R**Project 2 (“Lines”, due Friday, Mar. 1):**https://classroom.github.com/a/Ewo2k4xc

### SCHEDULE

**Jan 28: Introduction**

**Reading:**None**Assignment:**- Sign up for a Github account if you don’t already have one (www.github.com).
- Go through the following setup, if desired (no need to use the Anaconda setup if you prefer another one, but make sure you install torch and go through the Github Classroom tutorial): https://github.com/Mark-Hopkins-at-Reed/csci-378/blob/master/admin/setup.md
- Bring your laptop on Wednesday!

**Jan 30: Class Infrastructure/Introduction to Torch**

**Reading:**None**Assignment (due Friday, Feb. 1):**HW1 (link above, under Homework Links).

**Feb 1: A Whirlwind Guide to Machine Learning**

**Reading:**None**Assignment (due Monday, Feb. 4):**HW2 (link above, under Homework Links).

**Feb 4: Gradient Descent**

**Reading:**Review Machine Learning: A Whirlwind Guide lecture notes.**Assignment (suggested):**Find out where in the world Chekhov’s Sun might be (i.e. complete Descent1.ipynb).

**Feb 6: Gradient Descent**

**Reading:**Review first part of Gradient Descent lecture notes.**Optional Reading:**Goodfellow, Chapter 8.3 and 8.5**Assignment:**Project 1 (link above, under Homework and Project Links)

**Feb 8: Bayesian Networks**

**Reading:**Review full Gradient Descent and Gradient Descent: Now in 2D! notes.**Assignment:**None

**Feb 11: Reading a Bayesian Network**

**Reading:**Review lecture notes on Probability and Bayesian Networks .**Assignment (due Wednesday):**Complete handout distributed in class.

**Feb 13: D-Separation/Causal Models**

**Reading:**Review lecture notes on How to Read a Bayesian Network.**Assignment (due Friday):**Complete handout distributed in class.

**Feb 15: Causal Models**

**Reading:**None**Assignment:**Nothing to hand in, but please review lecture notes on Functional Causal Models.

**Feb 18: Exponential Distributions**

**Optional Reading:**Goodfellow, Chapter 3.9.3 and 3.9.4**Assignment:**Complete handout distributed in class.

**Feb 20: Regression Problems**

**Reading:**Review lecture notes on Exponential Distributions.**Assignment:**None.

**Feb 22: Linear Regression and Maximum Likelihood**

**Reading:**Review lecture notes on Regression Problems.**Optional Reading:**Goodfellow Chapter 5.1, 5.5