Activation functions used in machine learning:

# Tag: Machine Learning

# Simon working on a neural networks paper

Simon was working on a neural networks paper in Jupyter Notebook on Friday evening, but didn’t finish it because the Coding Train live stream started. He says he can no longer continue without having to do too much copy-pasting from this version into a new one, as his in-your-browser time expired, so I’m posting some screen shots of the unfinished paper below. This is the way Simon teaches himself: he follows lectures and tutorials online and then goes ahead to writing his own “textbook”or recording his own “lecture”. Much of the knowledge he acquires on neural networks these days comes from Siraj Raval’s YouTube series “The Math of Intelligence”.

# Simon’s bedtime lectures on neural networks

There’s a part 3 coming!

“Mom, my ClickCharts trial period expired, so I found this Virtual Paradigm Enterprise!” (Simon independently searches for free options to make beautiful diagrams online).

Here a diagram of an LSTM neural network:

And an RNN:

# Simon’s own little neural network

This is one of Simon’s most enchanting and challenging projects so far: working on his own little AIs. As I’ve mentioned before, when it comes to discussing AI, Simon is both mesmerized and frightened. He watches Daniel Shiffman’s neural networks tutorials twenty times in a row and practices his understanding of the mathematical concepts underlying the code (linear regression and gradient descent) for hours. Last week, Simon built a perceptron of his own. It was based on Daniel Shiffman’s code, but Simon added his own colors and physics, and played around with the numbers and the bias. You can see Simon working on this project step by step in the six videos below.

His original plan was to build two neural networks that would be connected to each other and communicate, he has only built one perceptron so far.

# Simon gets serious with Linear Regression (Machine Learning)

Simon has been working on a very complicated topic for the past couple of days: Linear Regression. In essence, it is the math behind machine learning.

Simon was watching Daniel Shiffman’s tutorials on Linear Regression that form session 3 of his Spring 2017 ITP “Intelligence and Learning” course (ITP stands for Interactive Telecommunications Program and is a graduate programme at NYU’s Tisch School of the Arts).

Daniel Shiffman’s current weekly live streams are also largely devoted to neural networks, so in a way, Simon has been preoccupied with related stuff for weeks now. This time around, however, he decided to make his own versions of Daniel Shiffman’s lectures (a whole Linear Regression playlist), has been busy with in-camera editing, and has written a resume of one of the Linear Regression tutorials (he actually sat there transcribing what Daniel said) in the form of an interactive webpage! This Linear Regression webpage is online at: https://simon-tiger.github.io/linear-regression/ and the Gragient Descent addendum Simon made later is at: https://simon-tiger.github.io/linear-regression/gradient_descent/interactive/ and https://simon-tiger.github.io/linear-regression/gradient_descent/random/

And here come the videos from Simon’s Liner Regression playlist, the first one being an older video you may have already seen:

Here Simon shows his interactive Linear Regression webpage:

A lecture of Anscombe’s Quartet (something from statistics):

Then comes a lecture on Scatter Plot and Residual Plot, as well as combining Residual Plot with Anscombe’s Quartet, based upon video 3.3 of Intelligence and Learning. Simon made a mistake graphing he residual plot but corrected himself in an addendum (end of the video):

Polynomial Regression:

And finally, Linear Regression with Gradient Descent algorithm and how the learning works. Based upon Daniel Shiffman’s tutorial 3.4 on Intelligence and Learning:

# Simon explains Linear Regression (Machine Learning)

In the two videos below Simon writes a JavaScript program using Linear Regression in Atom and gives a whiteboard lecture on the Linear Regression algorithm, both following a tutorial on Linear Regression by Daniel Shiffman.

Simon made a mistake in the formula using the sigma operator. He corrected it later. It should be i=1 (not i=0).