Very Nervous System

David Rokeby has been making new media artwork since the early 1980s. One of his well known pieces "Very Nervous System" was directly responsible for causing me to dive down the rabbit hole into interactive technology. Here is a video of it in action:

Very Nervous System was the third generation of interactive sound installations which I have created. In these systems, I use video cameras, image processors, computers, synthesizers and a sound system to create a space in which the movements of one's body create sound and/or music. It has been primarily presented as an installation in galleries but has also been installed in public outdoor spaces, and has been used in a number of performances. I created the work for many reasons, but perhaps the most pervasive reason was a simple impulse towards contrariness. The computer as a medium is strongly biased. And so my impulse while using the computer was to work solidly against these biases. Because the computer is purely logical, the language of interaction should strive to be intuitive. Because the computer removes you from your body, the body should be strongly engaged. Because the computer's activity takes place on the tiny playing fields of integrated circuits, the encounter with the computer should take place in human-scaled physical space. Because the computer is objective and disinterested, the experience should be intimate.

The gallery onlooker is intended to stand in front of a camera and dance around. The result is a beautiful interactive musical composition that pays homage to musical styles found in traditional eastern Asian music. Although the technology and interface has changed many times over the years the basic concept has remained the same. 

The image is reduced into a monochromatic grid by a given number of pixels. The amount of light present in each pixel and the change in light that occurs over time is manipulated and mapped to a series of messages that control a musical composition. In the early stages of the installation, 64 individual photocells housed behind a makeshift lens were connected to an Apple II computer using a variety of hardware interfacing. The computer was used to generate MIDI information which was sent to a series of synthesizers to generate the audio for the installation. The complexity of the interaction between how the hardware, software, and synthesis interact with one another can be found in the following article: 

Creating Interactive Dance with the Very Nervous System - Todd Winkler

Instructions: 1.) Put on good headphones 2.) Sit in pitch black room 3.) Turn brightness all the way up on monitor 4.) Turn monitor towards wall Abstraction of Jean Painleve's "Cristaux Liquides." Each pixel represents 1 oscillator. Each oscillator's (quantized) pitch is controlled by the amount of red in it's cell.

I tried to mimic similar techniques with Max/MSP in the video above. The patch is looking for overall brightness of a specific hue in each square. Each square outputs data that behaves in a similar manner to a photocell. Normally, a photocell measures overall brightness. In the patch only a specific hue's overall brightness was being utilized, in this case the color red. Although the entire thing was done in software and is in fact not an actual photocell, the output of each pixel could be thought of in the abstract as a fluctuating analog signal like a control voltage on a synthesizer. Much like a modular synthesizer, if you were to lock this control voltage to a scale by using a quantizer, you would be able to get musical intervals out of the interaction. This patch does exactly that and assigns those quantized notes to the frequency of individual sine waves for each square. In this particular example the key chosen was A minor pentatonic. Some squares are offset from others harmonically by octaves and fifths to create a more interesting composition. 

David Rokeby currently resides in Toronto, ON and is still actively creating awesome artwork. Very Nervous System is still being exhibited.  

CLICK HERE TO VIEW DAVID ROKEBY'S WEBSITE