CSCI 378: Deep Learning

REED COLLEGE, SPRING 2019

This course is an introduction to deep neural architectures and their training. Beginning with the fundamentals of regression, optimization, and regularization, the course will then survey a variety of architectures and their associated applications. Students will develop projects that implement deep learning systems to perform various tasks. Prerequisites: Mathematics 201, 202, and Computer Science 221. 

BASIC INFO

Professor: Mark Hopkins, hopkinsm@reed.edu

Class Schedule: MWF 3:10-4pm, Physics 240A

Office Hours: MW 4:10-6pm, Th 10am-noon in Library 314 (M 4:10-5 and Th 10-11am are reserved for this class; the remainder are shared with CSCI 121).

Textbook (optional): Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.

Syllabus: downloadable here

LECTURE NOTES

  1. Machine Learning: A Whirlwind Guide (pdf)
  2. Gradient Descent (pdf)
  3. Gradient Descent: Now in 2D! (pdf)
  4. A Quick Review of Basic Probability Formulas (pdf)
  5. Bayesian Networks (pdf)
  6. How to Read a Bayesian Network (pdf)
  7. Functional Causal Models (pdf)
  8. Exponential Distributions (pdf)
  9. Regression Problems (pdf)

 

LECTURE CODE

  1. Introduction to Tensors (Wednesday, Jan. 30): https://classroom.github.com/a/h3tOU2km
  2. Solution to Rubik (Monday, Feb. 4): https://github.com/Mark-Hopkins-at-Reed/csci-378/blob/master/code/rubik/rubik.py

 

HOMEWORK AND PROJECT LINKS

  1. HW1 (“Memoize”, due Friday, Feb. 1): https://classroom.github.com/a/WqP7Oqmf
  2. HW2 (“Rubik”, due Monday, Feb. 4): https://classroom.github.com/a/yaOkb2-4
  3. Project 1 (“Descent”, due Friday, Feb. 8): https://classroom.github.com/a/zidPHJ4R
  4. Project 2 (“Lines”, due Friday, Mar. 1): https://classroom.github.com/a/Ewo2k4xc

 

SCHEDULE

Jan 28: Introduction

Jan 30: Class Infrastructure/Introduction to Torch

  • Reading: None
  • Assignment (due Friday, Feb. 1): HW1 (link above, under Homework Links).

Feb 1: A Whirlwind Guide to Machine Learning

  • Reading: None
  • Assignment (due Monday, Feb. 4): HW2 (link above, under Homework Links).

Feb 4: Gradient Descent

  • Reading: Review Machine Learning: A Whirlwind Guide lecture notes.
  • Assignment (suggested): Find out where in the world Chekhov’s Sun might be (i.e. complete Descent1.ipynb).

Feb 6: Gradient Descent

  • Reading: Review first part of Gradient Descent lecture notes.
  • Optional Reading: Goodfellow, Chapter 8.3 and 8.5
  • Assignment: Project 1 (link above, under Homework and Project Links)

Feb 8: Bayesian Networks

Feb 11: Reading a Bayesian Network

Feb 13: D-Separation/Causal Models

Feb 15: Causal Models

Feb 18: Exponential Distributions

  • Optional Reading: Goodfellow, Chapter 3.9.3 and 3.9.4
  • Assignment: Complete handout distributed in class.

Feb 20: Regression Problems

Feb 22: Linear Regression and Maximum Likelihood

  • Reading: Review lecture notes on Regression Problems.
  • Optional Reading: Goodfellow Chapter 5.1, 5.5