🧠 Learning Deep Learning: My Curated Top-Five List

First published on The Grand Janitor Blog
Maintained by Arthur Chan – Updated periodically with curated content and personal reflections.

Note at 20250721 - Time flies - This list is written 9 years ago. Inevitably some of the resources are outdated, and I am planning to write an update. Regardless, I still believe that you want to learn how to walk before you run. Some of the material will be the basics you want to learn first before you look into more advanced materials.

---

✨ Philosophy

Most deep learning resource lists aim for completeness. Mine doesn't.
This list assumes your time is limited. Each course or book here is curated based on: - Personal completion or deep audit - Pedagogical quality - Practical relevance

Think of this as “Arthur’s Roadmap” — with opinionated suggestions for people entering deep learning with serious intent.

---

🧭 Where to Start: The “Top-Five of Top-Five”

1. Take Online Classes – Best return on effort. Do the lectures + homework.
2. Read a Textbook – You need more than lectures for real depth.
3. Experiment with Frameworks – Try real-world implementation.
4. Read Select Blogs – Focus on insightful ones, not noise.
5. Engage in Forums – Pick communities that actually discuss, not spam.

---

🎓 Core Lectures (Beginner Track – "The Basic Five")

• Andrew Ng’s ML (Coursera)
• DeepLearning.ai Specialization (Coursera)
• CS231n: Convolutional Neural Networks
• CS224d: NLP with Deep Learning (Socher)
• David Silver’s Reinforcement Learning (UCL)

---

📚 Intermediate & Advanced Topics

• Geoffrey Hinton's Neural Networks for ML
• Daphne Koller’s Probabilistic Graphical Models (Coursera)
• MIT 6.S094: Deep Learning for Self-Driving Cars
• CMU 11-747: Neural Networks for NLP
• Oxford Deep NLP

---

🧠 Reinforcement Learning

• Udacity’s Deep RL Nanodegree
• CMU Deep RL and Control
• Berkeley Deep RL (CS285)

---

📘 Recommended Books

• Michael Nielsen’s Neural Networks and Deep Learning
• PRML (Bishop)
• Duda & Hart
• Goodfellow et al.’s Deep Learning Book
• Kyung Hyun Cho’s NLU with Distributed Representations

---

⚙️ (Older) Frameworks

• TensorFlow → Keras
• PyTorch
• Theano
• Caffe / Caffe2
• Torch (Lua)
• Deeplearning4j
• Neon

---

📚 Tutorials

• char-rnn
• TensorFlow Tutorials

---

📬 Mailing Lists

• Import AI (Jack Clark)
• Data Machina

---

💬 Forums

• AIDL Facebook Group
• Strong AI

---

🔢 Math You Should Know

1. Bayes’ Theorem
2. Multivariate Gaussian
3. Matrix Differentiation – Matrix Cookbook
4. Calculus of Variations
5. Information Theory

---

📝 Blogs You Should Read

• Chris Olah’s Blog
• Andrej Karpathy’s Blog
• WildML (Danny Britz)
• Tombone’s CV Blog
• The Spectator (Shakir Mohamed)