Convolutional Neural Networks (From Bob Zigon)
Here is my "algorithm" for learning CNN's. As I said, it is based on 2 years of intensive study
and application. My assumption is that the student is motivated to learn this stuff, but doesn't have
daily access to a "teacher/professor" that they can run to. Feel free to ask me questions about the list.
I think the real value of this list is the "order of execution". If you start too far down the list you won't have
the basics under your belt. I didn't have anyone suggesting an order so I fumbled, but I am very tenacious.
- Read the Bishop Red Book.
- Optionally read a couple chapters of this other BishopBook.
- Take the Stanford CS 231n class (AMAZINGLY practical). I can't emphasize the importance of this class. It is amazing. I think you can take it in parallel with reading #1
- This web site is good .. but there are more.
- Review the 1 page CNN program in C . When I put this on the debugger it really helped to reinforce the mathematics and the simple data structures.
- Google on this phrase "neural network code in c".
- Review Neural 2D Github) - A CNN framework in 1 .H and 1 .CPP file. It's less of a toy and closer to Caffe, but the code size is manageable so you can put it on a debugger and see how a more complex framework functions.
- Consider using Caffe. Integrate this with NVidia DIGITS. DIGITS supplies data management to the training and testing problem. V3 also adds DeepOMatic, a viz tool.
- At some point you will look at other CNN frameworks. If you go through the steps I have outlined, you will have a basic "intuition" for how to build one. However, you will eventually realize that you need a tool to help you "debug" them. You will want to look for a tool that allows you to "visualize" the internal features and filters of the network. This paper will help. Here is another Visualizer
- When I first started using Caffe, I started with the COIL20 dataset. This helped me understand a small dataset in black and white. Then I moved to COIL100.
This is a larger dataset in color. When I worked with this dataset I eventually discovered that CNN's can be "color blind". The green donut
looked like the red donut. I eventually discovered (based on reading numerous papers) that operating on color images in RGB space is a
bad idea. You need to switch to HSV.
- Here is a ZIP file. It has 150 or so CNN papers by the important authors. The Hinton, Ng and LeCun papers are good from a historical perspective. Krizhevsky, Karpathy, Zeiler and Ciresan are their students who graduated and are pushing the technology.
- I strongly advise you move on to CNTK with Keras.