Skip to content

Education

Awesome Deep Learning Resources Awesome

This is a rough list of my favorite deep learning resources. It has been useful to me for learning how to do deep learning, I use it for revisiting topics or for reference. I (Guillaume Chevalier) have built this list and got through all of the content listed here, carefully.

Here are the all-time Google Trends, from 2004 up to now, September 2017:

You might also want to look at Andrej Karpathy's new post about trends in Machine Learning research.

I believe that Deep learning is the key to make computers think more like humans, and has a lot of potential. Some hard automation tasks can be solved easily with that while this was impossible to achieve earlier with classical algorithms.

Moore's Law about exponential progress rates in computer science hardware is now more affecting GPUs than CPUs because of physical limits on how tiny an atomic transistor can be. We are shifting toward parallel architectures [read more]. Deep learning exploits parallel architectures as such under the hood by using GPUs. On top of that, deep learning algorithms may use Quantum Computing and apply to machine-brain interfaces in the future.

I find that the key of intelligence and cognition is a very interesting subject to explore and is not yet well understood. Those technologies are promising.

Online Classes

Books

  • Clean Code - Get back to the basics you fool! Learn how to do Clean Code for your career. This is by far the best book I've read even if this list is related to Deep Learning.
  • Clean Coder - Learn how to be professional as a coder and how to interact with your manager. This is important for any coding career.
  • How to Create a Mind - The audio version is nice to listen to while commuting. This book is motivating about reverse-engineering the mind and thinking on how to code AI.
  • Neural Networks and Deep Learning - This book covers many of the core concepts behind neural networks and deep learning.
  • Deep Learning - An MIT Press book - Yet halfway through the book, it contains satisfying math content on how to think about actual deep learning.
  • Some other books I have read - Some books listed here are less related to deep learning but are still somehow relevant to this list.

Posts and Articles

Practical Resources

Librairies and Implementations

Some Datasets

Those are resources I have found that seems interesting to develop models onto.

Other Math Theory

Gradient Descent Algorithms & Optimization Theory

Complex Numbers & Digital Signal Processing

Okay, signal processing might not be directly related to deep learning, but studying it is interesting to have more intuition in developing neural architectures based on signal.

Papers

Recurrent Neural Networks

Convolutional Neural Networks

Attention Mechanisms

Other

YouTube and Videos

  • Hacker News - Maybe how I discovered ML - Interesting trends appear on that site way before they get to be a big deal.
  • DataTau - This is a hub similar to Hacker News, but specific to data science.
  • Naver - This is a Korean search engine - best used with Google Translate, ironically. Surprisingly, sometimes deep learning search results and comprehensible advanced math content shows up more easily there than on Google search.
  • Arxiv Sanity Preserver - arXiv browser with TF/IDF features.
  • Awesome Neuraxle - An awesome list for Neuraxle, a ML Framework for coding clean production-level ML pipelines.

License

CC0

To the extent possible under law, Guillaume Chevalier has waived all copyright and related or neighboring rights to this work.