Simon has been working on a very complicated topic for the past couple of days: Linear Regression. In essence, it is the math behind machine learning.
Simon was watching Daniel Shiffman’s tutorials on Linear Regression that form session 3 of his Spring 2017 ITP “Intelligence and Learning” course (ITP stands for Interactive Telecommunications Program and is a graduate programme at NYU’s Tisch School of the Arts).
Daniel Shiffman’s current weekly live streams are also largely devoted to neural networks, so in a way, Simon has been preoccupied with related stuff for weeks now. This time around, however, he decided to make his own versions of Daniel Shiffman’s lectures (a whole Linear Regression playlist), has been busy with in-camera editing, and has written a resume of one of the Linear Regression tutorials (he actually sat there transcribing what Daniel said) in the form of an interactive webpage! This Linear Regression webpage is online at: https://simon-tiger.github.io/linear-regression/ and the Gragient Descent addendum Simon made later is at: https://simon-tiger.github.io/linear-regression/gradient_descent/interactive/ and https://simon-tiger.github.io/linear-regression/gradient_descent/random/
And here come the videos from Simon’s Liner Regression playlist, the first one being an older video you may have already seen:
Here Simon shows his interactive Linear Regression webpage:
A lecture of Anscombe’s Quartet (something from statistics):
Then comes a lecture on Scatter Plot and Residual Plot, as well as combining Residual Plot with Anscombe’s Quartet, based upon video 3.3 of Intelligence and Learning. Simon made a mistake graphing he residual plot but corrected himself in an addendum (end of the video):
And finally, Linear Regression with Gradient Descent algorithm and how the learning works. Based upon Daniel Shiffman’s tutorial 3.4 on Intelligence and Learning:
Simon made a mistake in the formula using the sigma operator. He corrected it later. It should be i=1 (not i=0).
Here Simon explains how to calculate the magnitude of a 3D vector. This is something he partially figured out on his own and partially learned from Daniel Shiffman’s tutorial on Trigonometry and Polar Coordinates.
Almost every evening, before going to bed, we are reading books and Simon mostly prefers math adventures. Russian author Vladimir Levshin (1904-1984) published several books about geometry, algebra and math history, with numbers and letters as the leading characters. Most chapters contain complicated riddles that we solve along the way. Sometimes, Simon gets up to fetch some paper and pencils to write down what he thinks the formula or the geometrical pattern should be for a particular story. And because Levshin’s books often mention famous mathematicians of the past, I see Simon learn about history through math. What he knows about Ancient Greece or the 1970’s mainly comes from his interest in early math and geometry or the dawn of computer science.
Here Simon was drawing his impressions of Cantor’s set theory, inspired by a passage about him in Levshin’s book:
Levshin’s book that we’re reading now:
Passage on Boole and Cantor:
Another book by Levshin we have recently read, about Algebra:
A chapter from that book talking about finding a sum of all the members of an arithmetic progression:
Simon stormed out the bedroom and came back with a sheet of paper where he wrote down the formula, before we read about it in the book (he often tries to come up with his own formulas):
The same formula in the book:
Funny how, even when training some pretty straightforward (and boring) arithmetic or Dutch reading, Simon tries to introduce more complex notions like here,
the floor, ceiling and round functions while solving a simple arithmetic word problem:
and lexicographic order, while sequencing Dutch story sentences:
Heard Simon give his Russian grandparents a lecture in the playroom, via FaceTime. When I came in, this is what I saw on the whiteboard. Simon proudly said he figured out how to calculate the arc-tangent. Why, what do you talk to your Grandmom about?
Simon wrote a modulus counting program in Processing after we were discussing why
1 % 2 = 1 and why 2 % 4 = 2. He basically told me he simply knew 1 % 60 equals 1, but he didn’t know why. We realized later that the strange result comes from the fact that if a number (the divisor) fits into another number (the numerator) zero times then, in modulus counting, that numerator becomes the excess.
Simon loves functions. He hasn’t ever studid them in Algebra, but comes across functions regularly in programming. This is what he came up with Wednesday:
I showed Simon that there is a functions course on Khan Academy:
And he watched some videos and completed some tasks:
A minimum spanning tree or minimum weight spanning tree is a subset of the edges of a connected undirected graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight. Simon said it resembled the Traveling Salesperson problem, except that the spanning tree can have “loose ends”:
Yesterday Simon spent the whole day studying the concept of Cellular Automaton (CA), a discrete model of a system of “cell” objects used in physics, math and theoretical biology. He learned a lot from Daniel Shiffman’s tutorials on Cellular Automata and his book The Nature of Code (Chapter 7).
Simon describes Cellular Automaton as a set of rules for getting nature-like patterns. 1D cellular automata are static (think of a pattern resembling the Pascal triangle) and 2D are not static (the most famous example is Conway’s Game of Life).
The mathematician who made cellular automata a big thing (and even Turing-compatible!) was Stephen Wolfram. Simon also looked at some of Wolfram’s writings. Wolfram’s book A New Kind of Science is intimidating (in content and size) but full of graphic illustrations and available for free at http://www.wolframscience.com/nksonline/toc.html
A CA is basically a grid where cells live, each cell in a particular state (the simplest example being “1” or “0”). In Processing, it’s possible to have a CA draw a beautiful pattern this way. Simon tried to write a CA program in another language. He tried writing it in Python on his RaspberryPi first, but I overheard him say “don’t know how classes really work in Python, let’s try Ruby – I know how classes work in Ruby!” Then I saw him move over to the desktop and plug his long forgotten Arduino in. He said he now wanted to translate a CA code into C. Of course, there would be no pattern, but he wanted to have the Arduino “spit out numbers in the console” instead. Eventually he got an error and gave up, but it was intriguing to observe him.
The videos below show Simon talk about CA and program a CA in Arduino (C):