Flocking System Painting with Pixels

Flocking painting Live Video 8 Aug 2017

This is one Simon’s most beautiful projects recently! Simon saw the idea to link the webcam image to the boids of a flocking system in a video by Daniel Shiffman, but the code featured in this project Simon wrote himself. The Flocking code is based on Daniel Shiffman’s example from his book The Nature of Code. (Flocking is a steering behavior that consists of separation, alignment and cohesion – which are also steering behaviors – combined).

Simon’s sis also posed for the camera:

Flocking painting Live Video 8 Aug 2017 2

Simon is also about to make a JavaScript version of this fun project, because JavaScript allows to host it easily online, so that everyone can play with it. With JavaScript, he may even be able to write it in an online editor, so there won’t even be a need to host it. Check in later for an update to this post!

UPDATE: Simon encountered a problem trying to translate his Flocking System Painting with Pixels into JavaScript: P5 runs much slower than Processing after Simon added steering behavior. He doesn’t know how to solve this. Simon’s JavaScript code is online at https://codepen.io/simontiger/pen/ZJKBbN?editors=0011

Translating Car On Terrain project from Phaser.io into Processing

Today Simon spent hours translating this Car On Terrain project from Phaser.io (where it appears in JavaScript) into Processing (Java). He loved doing it in a form of a lesson for me, while I was filming him and asking questions about loops, arrays, fixtures, shapes and bodies (and there are many things I don’t understand). Simon also spoke about “the three most important properties: density, friction and restitution”. The project involved a lot of Physics, using many Box2D sub-libraries and translating between pixels and mm.

In the end, he got tired of writing all the coordinates for the terrain vertices, but he did get quite far.

 

 

 

 

 

Applying Box2D to translate from pixels into mm:

CarOnTerrain translating from pixels into mm

Microsoft Kinect (v2) in Processing using the Open Kinect for Processing library

Our Kinect adapter has finally arrived! Simon had been waiting for about one and a half months and was extremely hyper to try out the code he had already written ages ago, following Daniel Shiffman’s tutorials on Computer Vision.  The code involved processing the pixels of the “depth image” and mapping depth to brightness. Simon also learned how to find the average location of a set of pixels within a minimum and maximum depth threshold (can be useful for basic hand tracking).

Computer Vision: Color Tracking and Motion Detection (Java)

On Saturday Simon picked up Computer Vision again – something he had tried back in February but got stuck. This time around, he had built up better theoretical knowledge and sketched out a rough plan in advance. He has managed to complete the first two tasks from the plan, following Daniel Shiffman’s brilliant Color Tracking and Motion Detection tutorials.

IMG_4335

Here he explains how colour tracking in computer vision works:

Simon programmed his camera to track anything red. He was careful not to wear anything red himself and tried to get the computer find the only red object within its vision – a red building block – and mark it with “a blob” (an ellipse):

Then Simon made the computer to not only track the colour and mark it with a blob, but also show all the colour pixels picked up (by changing them to white):

Simon added one more red object into the picture. The blob was now choosing the average point between the two red objects:

Simon changed the blob colour:

 

Motion detection. This basically means analyzing the pixels of a video to detect motion. This technique is also known as frame differencing. If an object is still, the computer shows it in white, and if an object is moving, it’s shown in black. Simon programmed this using a threshold and a distance squared formula.

 

Translating from JavaScript to Java (Continued)

Simon continued translating from JavaScript to Java as he followed Daniel Shiffman’s tutorial on creating a mirror based on the brightness values of the pixels from a live video feed. The tutorial was about working in p5.js and Simon worked in Processing (Java):

 

 

 

Translating from JavaScript to Java (Simon wrote this post)

– Simon, where did you learn translating from JavaScript to Java?

– From Dan, of course! I’ve compared the two formulas for converting from 2D to 1D. (The width stands for the width of the canvas).

From 2D to 1D in Processing: x + y * width
and
From 2D to 1D in p5: x + y * width * 4

But, why did I say, “from 2D to 1D”? Because these formulas relate to the formula for converting from Processing to p5 and vice versa. How do they relate? Because of the 4 in the p5 formula. Why? Here is the formula (i is the index into the pixel array):

p5 to Processing: i / 4
Processing to p5: i * 4

For example, in the video I divide 365 (i) by 4 and get 91.25. Here I wrote a table of what the decimal place means:

Decimal Value Color Value
00 red
25 green
50 blue
75 alpha (transparency)

In the example with 91.25, the decimal value is equal to 25. This is why I wrote in Processing: color(51, 255, 51, 255)

So this is how I got the green value to Processing.

This post is really about converting from p5 to Processing and vice versa. That I figured out myself. You could say that I only explained that in this post, but really I explained that and converting from 1D to 2D. That’s why I added this form:

Converting from 1D to 2D
Converting from Processing to p5 and vice versaIMG_4268

IMG_4270

Simon writing this post in html

IMG_4271

Simon wrote this table in html

Pixel Sorting in Processing (Java)

Pixel Sorting 1 Apr 2017 8

Today Simon showed me what he learned about pixel sorting in Processing (Java) by doing this coding challenge by Daniel Shiffman. Using a “selection sort” algorithm, he sorted the pixels of a sunflower image by brightness and hue. The results were amazing, it had this impressionistic effect, like a Van Gogh painting. Maybe Van Gogh also sorted pixels?

Here the pixels are sorted at random in black and white:

Pixel Sorting 1 Apr 2017 1

Here Simon added max RGB values:

Pixel Sorting 1 Apr 2017 2

Here the pixels are cloned from the image on the left:

Pixel Sorting 1 Apr 2017 3

The same, but in an easier way, by using the img.get() function:

Pixel Sorting 1 Apr 2017 4

Pixels sorted by brightness:

Pixel Sorting 1 Apr 2017 5

Pixels sorted by hue:

Pixel Sorting 1 Apr 2017 6

Beautiful, isn’t it?

Pixel Sorting 1 Apr 2017 7