This magical time of the year, Simon’s craziest, most daring dreams come true! First, his guru from the New York University Daniel Shiffman sends Simon his book and the words he writes there are the most beautiful words anyone has ever told him. Then, on the last day of the awesome year 2017, Simon’s other hero, the glamorous knight of AI Siraj Raval materialises in our living room, directly from YouTube! Happy New Year full of miracles and discoveries everyone!
Daniel Shiffman’s book “The Nature of Code” that Simon had already largely read online and now also reads before bed. It also comforted him recently when he was in pain, he cuddled up of the sofa with this big friendly tome on his lap.
Daniel Shiffman signed the book for Simon:
Siraj Raval stepped out of the YouTube screen straight into our Antwerp apartment on December 31. Simon has been following Siraj’s channel for months, learning about the types of neural networks and the math behind machine learning. It is thanks to Siraj’s explanations that Simon has been able to build his first neural nets :
Simon has started building neural networks in Python! For the moment, he has succeeded in making two working neural nets (a Perceptron and a Feed Forward neural net). He used the sigmoid activation function for both. The code partially derived from Siraj Raval’s “The Math of Intelligence” tutorials.
The FF was tougher to build:
Simon’s nets run locally (on our home pc), but he will need more computational power for the more complex future projects, so he signed up to this wonderful online resource called FloydHub! FloydHub is sort of a heroku for deep learning, a Platform-as-a-Service for training and deploying deep learning models in the cloud. It uses Amazon, which Simon could, too, but it would have been a lot more expensive and tedious work to set up.
Simon’s next step will be another supervised learning project, a Recurrent Neural Net that will generate text. He has already started building it and fed it one book to read! In this video he explains how character-based text generators work:
Simon’s version of the Citius Invaders game in Processing is finally working! The big bug that seemed so unsolvable (enemies endlessly crossing over) has been defeated!
Siraj Raval presented this game in Python during Week 9 of his “Math of Intelligence” course. Simon’s is a simplified version that still incorporates a genetic algorithm.
Link to code on github at: https://github.com/simon-tiger/citius-invaders/
Link to our previous post on this project: https://antwerpenhomeschooling.wordpress.com/2017/11/23/problem-with-a-genetic-algorithm-game/
Link to Simon’s archived Live Stream about the project: https://antwerpenhomeschooling.wordpress.com/2017/12/01/live-stream-30-november-speechjs-and-citius-invaders/
Activation functions used in machine learning:
Simon told me today he was ready to start building his own DAE (Denoising Auto Encoder) neural network. He said he would be using a documentation page about a machine learning library called Keras at blog.keras.io/building-autoencoders-in-keras.html. He found this documentation page completely on his own, by searching the web and digging into Python fora. I just watch him google something like “How can I install Keras 1.0?” and find GitHub discussions on the subject that guide him along. Or I see him type “How to install Python on Windows?” and follow the instructions at How-to-Geek. Eventually, he came up with a list of steps that he needed to complete in order to be able to install Keras (installing Python 3 (looking up why it should be 3 and not 2), installing PIP, installing Tensor Flow, etc). It didn’t work on a mac laptop, so he tried everything on a pc and it worked! Two steps required that he used terminal. It was amazing to see him plan ahead, search and implement (notoriously difficult) steps completely independently.
He has started working on the DAE tonight. Working on a node package that makes `manifest.json` files (for Chrome extensions) at the same time, so not sure he’ll finish soon. “Mom, I’ve got so many things to do! There are so many thoughts in my head!”
Simon was working on a neural networks paper in Jupyter Notebook on Friday evening, but didn’t finish it because the Coding Train live stream started. He says he can no longer continue without having to do too much copy-pasting from this version into a new one, as his in-your-browser time expired, so I’m posting some screen shots of the unfinished paper below. This is the way Simon teaches himself: he follows lectures and tutorials online and then goes ahead to writing his own “textbook”or recording his own “lecture”. Much of the knowledge he acquires on neural networks these days comes from Siraj Raval’s YouTube series “The Math of Intelligence”.
There’s a part 3 coming!
“Mom, my ClickCharts trial period expired, so I found this Virtual Paradigm Enterprise!” (Simon independently searches for free options to make beautiful diagrams online).
Here a diagram of an LSTM neural network:
And an RNN:
Excerpts from Simon writing about distributions in an IPython Notebook, consulting Numpy Documentation. The code for the examples below is his own.
Simon had earlier created similar examples about Matplotlib, also in an IPython Notebook, but didn’t save those.
Simon has been watching a lot of Siraj Raval’s videos on neural networks lately, brushing up his Python syntax and derivatives. He has even been trying the great Jupyter editor, where one can build one’s own neural network and install libraries with pretrained networks https://try.jupyter.org/
Just like with Danel Shiffman’s videos, the remarkable thing about Siraj’s (very challenging) courses is that they also touch upon so many subjects outside programming (like art and music and stock exchange) and are compiled with a sublime sense of humour.
In this video, Simon gives an introduction to the basics of programming languages (what languages there are, which ones are front end and which ones are back end and what libraries and data interchange formats go with different languages):