Computer repeats after Simon

Following the exciting text-to-speech and speech-to-text projects yesterday, this morning Simon made a basic speech-to-text-to-speech demo, which means that the computer can now repeat (parrot) everything Simon says.

Simon relied on what he learned during Daniel Shiffman’s two latest live streams on the Coding Train channel in building these projects.

Advertisements

Almost talking to the computer!

This is one of those wow projects, so much fun! Simon built his Text-to-Speech and Speech-to-Text demos following Daniel Shiffman’s recent live streams on working with the p5.Speech library and added some extra style features. This basically means that you can type anything on your computer and hear it say what you’ve typed (in any voice or language!) or, in what Simon said was an easier project, yell something to your computer (I love you!) and watch it type it out for you. The next step will be combining the two and including that code into a chat bot code.

You can play with Simon’s Text-to-Speech demo on GitHub at:

Basic text to speech example: https://simon-tiger.github.io/p5_speech/01_text2speech/

Example using different voices: https://simon-tiger.github.io/p5_speech/02_voices/

Basic speech to text example: https://simon-tiger.github.io/p5_speech/03_speech2text/

Code/ repo: https://github.com/simon-tiger/p5_speech

 

 

 

 

 

Simon writing a Matter.js textbook

Simon is currently working on a “Matter.js textbook”, a set of tutorials on how to use Matter.js (a physics library) that he writes on GitHub. Simon writes everything himself, not copying from anywhere. Sometimes he forgets some coding syntax and looks it up in the documentation, but for the rest this is his own text. He explains: “The reason why I am making this textbook is because there are Matter.js tutorials but they are really short and there aren’t many of them. And the only video tutorials are by Daniel Shiffman and they are using P5.js.” If it proves possible, Simon might add video to his GitHub textbook later.

It’s great to observe him type away and when I look over his shoulder, I see that he makes very decent sentences, with a nice tint of humour every now and then.

The link to the textbook:

https://github.com/simon-tiger/matter-js-tutorial/blob/master/README.md

Simon builds his own RiveScript Editor

Simon has been working on a RiveScript Editor for creating chat bots. RiveScript is a markup language, that gives chat bots the ability to respond to messages from humans using natural human language. This project will involve server side programming, which means that Simon will have to go through the whole process of hosting it on Heroku once he’s finished. So far, he decided to give himself a break in the middle of programming the syntax highlighting, which he found a little too much boring work, so not sure whether the editor will ever be published. In any case, Simon has got quite far and the interface looks good.

Simon has already built three chat bots using RiveScript online editor and Atom.

 

Simon contributes to the p5.Speech library

Simon has made a pull request to the p5.Speech GitHub repo (a milestone!) and hopes his request gets merged. In this video he explains what he wants to improve with his contribution.

Later it turned out that someone else made a similar request (with more extras) and that request will probably be merged, so Simon was definitely thinking in the right derection. He got positive response from Daniel Shiffman and it looked like Simon’s comments have sparked a discussion on GitHub.

 

Simon contrubuted to p5.Speech library. Pull request 14 Oct 2017

Simon writing on GitHub: 

This github issue is referring to pull request #7.

As you can see in commit a2a5d38, there are some comments. Which look like 
this:

// this one 'start' cycle.  if you need to recognize speech more

// than once, use continuous mode rather than firing start()

// multiple times in a single script.

The comments are right before the start() function in the p5.SpeechRec 
object. But the commit adds arguments to this function:

p5.SpeechRec.prototype.start = function(continuous, interimResults) {

  if('webkitSpeechRecognition' in window) {

    this.rec.continuous = continuous;

    this.rec.interimResults = interimResults;

    this.rec.start();

  }

}

And before, that piece of code looked like this:

p5.SpeechRec.prototype.start = function() {

  if('webkitSpeechRecognition' in window) {

    this.rec.continuous = this.continuous;

    this.rec.interimResults = this.interimResults;

    this.rec.start();

  }

}

Are the comments "unnecessary" now? In other words, Should we remove 
them or leave them there?

28 times 28

Simon programmed a presentation to explain why 28×28 is not the same as 20×20 + 8×8 geometrically. The code is quite complicated and involves some trigonometry and conditional statements: the grid is divided into different parts every time Simon clicks and depending on how many times he has already clicked. This is typical Simon – coming up with an inherently arduous and complex system to visualise the beauty of the world around him, even of the seemingly trivial things. By the way, the inspiration for the 28×28 grid came from Simon’s favourite math channel, 3Blue1Brown and its latest video on Neural Networks (the grid was used to explain computer vision).

Simon is doing quite a lot of sums in his head nowadays, looks like it’s a new trend. Today, while bathing in the fountain outside, he was calculating how long 1/16th of a minute lasted. And a couple days ago, while waiting for his appointment at the hospital, he calculated how long it would take someone to read a whole page of random numbers, taking an educated guess that one takes 4 seconds to read out one number and remembering Daniel Shiffman mentioned there were 100×5 numbers per page in his book.

Cannon Game in Processing

Simon shows the Cannon game he created in Processing (Java). He says he was inspired by the Stackoverflow forum, where he saw an example of the game and later wrote the code for a similar game himself. I saw him quickly write the code in a matter of perhaps two hours. Simon will post his code on GitHub once he has added a couple extra features.

Introducing Siraj Raval

Simon has been watching a lot of Siraj Raval’s videos on neural networks lately, brushing up his Python syntax and derivatives. He has even been trying the great Jupyter editor, where one can build one’s own neural network and install libraries with pretrained networks https://try.jupyter.org/

Just like with Danel Shiffman’s videos, the remarkable thing about Siraj’s (very challenging) courses is that they also touch upon so many subjects outside programming (like art and music and stock exchange) and are compiled with a sublime sense of humour.

dsc_2125964025814.jpg