Network 2 (2018)

Crackle box and 10-relationship-network


  • 13-Apr-2018 - Spektrum, Berlin
  • 9-May-2018 - DeMontfort University, Leicester - Performance and lecture
  • 15-Jul-2018 - Iklectik, London
  • 27-Nov-2018 - Institut für Elektronische Musik, Graz - Performance and lecture

Network 2 is a work that uses the analog output of a crackle box (1) to drive a network of digital synthesizers and surround the performative gestures with an immersive synthetic landscape. This landscape requires no intervention other than the input sounds for its whole development. These sounds are converted into a wide variety of data which are then recombined to generate all the aspects of the computer's response, which is always a result of a combination between incoming data and automatic behaviors; there is no use of sampling or timeline sequencing.
The sounds do not avoid referenciality: there is glitch, synth patterns, percussive gong-like sounds, a part evokes an eastern bowed string instrument. It's not an intellectual operation of decontextualization, those sounds belong to my experience. It is more a chance for me to explore a particular view about emergent behavior. We are starting to get used to the "alien" sounds of the machines, it is almost if we expect them to sound that way. What happens if the machines use sounds that we are starting to consider socially codified? If the machine behavior proposes with an unpredictable logic sounds that evoke a vague sense of familiarity and then betray our expectations?

In this work there are two distinct phases of agency, which also determine two clear musical sections. The first section is responsive to the input sounds. Sometimes the association is direct, but most of the times is tangential: the spectrum analyzed is spread over a larger timespan, and often the computer recombines data coming from sounds produced at different time, blurring the association between sound and gesture. In the second section there is no input analysis. The network uses all the data accumulated during the first section to generate an independent landscape with formal ties to the sounds played in the first section; although, the relationships are mostly unintelligible. Here, the result of the mapping process concerns mostly the spectral components of one single sound that is then repeated, lengthly, with a rhythm that is the symmetric reflection of the timing of the performative gestures.

(1) The crackle box is a circuit designed by Michel Waisvisz in 1974. The original circuit consists of an operational amplifier whose connections are brought on the control surface, and left open. The user, touching them, closes the circuit using the body as conductor. The sounds produced are related to the resistance of the body, and the amount of pressure applied on the connectors. The classic crackle box used the integrated circuit LM709. Unfortunately the LM709 is not that easy to find, so instead I used the schematics of John Richards bed of nails, which uses a far more common IC, the LM358. The op amp inside the LM358 is more stable and less prone to surprises, but in the bad of nails this is compensated using both the op amps on the IC, basically creating two crackle boxes feeding back into each other. Furthermore, there is a white noise generator made amplifying the background noise of a small 10 Ohm resistance connecting pin 2 and 3. The connection is done manually, pushing with the finger the resistor on the pad, so it's possible to control the white noise in a fairly performative way.

Description of the network

This part is highly technical and it is mostly a memo for me to remember all the steps necessary to bring the piece to an end. It is also a diary where I describe all the relationships I develop, so that they can be reused for successive pieces. However it's here for whoever is curious about what happens in the piece.

• Relationship 1 - Inital additive synth.
This actor is the most independent of the first section. It starts on its own when the network is turned on. The occurrence is choosen by a beta distribution applied to the duration and it is mostly unrelated to the input. If the input sound is longer than 5 seconds the generation stops and wait for the next onset to start again. The fundamental pitches are a replica of the input signal with a 8-second-delay, enough to cancel an excessively direct perceptual association, and they are then transposed in a lower register. All the glitchy variations are independently generated by the computer but they have a simple relationship with the input: when the amp detection module sends the on message, it flips the state of a switch that controls many generative processes turning on and off most of the variations. Timber-wise, some bands of a bark list of the input are directly mapped to the amplitude of some partial of the synth.

Activating condition - Network is on

Ending condition - Detection of special evnent 1.

Occurrence - Mostly continuous. A sound longer than 5 seconds stops the generation, the next onset re-activates it.

• Relationship 2 - First train pulse.
The actor reacts when the amp detection module sends the on message. When this happens there is a high probability that the event occurs. Some initial parameters are choosen algorithmically. The pulse is also controlled with a variable mapping of the input spectral centroid, and the sound is very reactive to my gestures.

Activating condition - Detection of 1st event.

Ending condition - Detection of special event 1.

Occurrence - Probabilistic, when the amp detection module sends the on message.

• Relationship 3 - Second train pulse.
The actor reacts when the amp detection module sends the off message. When this happens there is a high probability that the event occurs. Some initial parameters are choosen algorithmically. The input continous pitch is recorded in a buffer when there is sound. When the synth plays, this buffer is played back at variable speeds and mapped to the delay of the comb filter. If the buffer is controlling the delay and I play again, the buffer stops and restarts when I'm done.

Activating condition - Detection of 1st event.

Ending condition - Detection of special event 1.

Occurrence - Probabilistic, when the amp detection module sends the off message.

• Relationship 4 - Slowly moving background.
This relationship is very tangential and uses spectral techniques to relate the spectrum of the input to the spectrum of the synth. Every 6 seconds an FFT frame of the input signal is taken, and some of the frequency/amp pair of the most prominent partials are assigned to banks of resonant filters. The banks are six and fade into each other providing a slow and dense morphing. The selection of the frequency/amplitude pairs follows a user-defined pattern based on one important parameter: how much the spectral content of the filter banks have to resemble the input (a variable tangentiality). When a spectrum more resembling the input is desired the program uses the most prominent partials; when a spectrum more tangential is desired, the program discards the loudest and selects the quietest ones. The overall amp is always normalized but it preserves the amp ratio of the FFT partials, setting the amp of the loudest used partial to 1. The resonating filter banks are excited with white noise. Slow random automations are used to control the amp and the transition from a flat sound to an impulse-like one. The decay of the resonant filter (here you can think of it as a normal Q) is directly related to the noisiness of the input signal, so when the input is more similar to white noise the decay is high and viceversa. This relation requires a continuous input, so when there is no sound to be analyzed it uses a silent loop created with the last part of the input sound.

Activating condition - Detection of 15th event.

Ending condition - Detection of special event 1.

Occurrence - Continuous.

• Relationship 5 - 2 static high-freqeuncy synths.
The actors react when the amp detection module sends the on message. The probability of occurrence is not too frequent. The synths use very high frequency content. The relationship is very simple. Here it is all about the minimal generative behavior of the synths.

Activating condition - Detection of 20th event.

Ending condition - Detection of special event 1.

Occurrence - Probabilistic, related to the amp threshold.

• Relationship 6 - Gliding bass.
This actor uses the pitches recorded on the onset of the input events, transposed in a low register. It starts playing probabilistically, when the amp detection module sends the on message. Rarely it is activated automatically. The pitch are stored in an array and selected with a shuffle among the last 5. The selection happens with an interrupt that uses the beta distribution applyed to the duration; the limits are very broad. The synth glides to the new value.

Activating condition - Detection of 20th event.

Ending condition - Detection of special event 1.

Occurrence - Probabilistic, related to the amp threshold. Rarely automatic

• Relationship 7 - 2 synths that respond to glissandi.
This relationship links the pitch, especially the glissandi of the input signal, to the gliding synths. It uses two kinds of buffer recording to store some user-defined parameters: the spectral centroid and an on/off signal when there is input sound. One kind of recording opens the buffer and closes it in correspondence with the amp threshold, so a buffer with a defined length is recorded every time there is an event in input and can be used right after. A second kind of buffer recording is continuous and these data can be used even if the buffer isn't closed, respecting the condition that its reading speed isn't faster than the real sound (reading index <= 1). This allow to sync some buffer based operations with the attack of the input sound. The use of these buffers and the related onset of the synth is triggered using probability: the reading can start in correspondence with the attack of the input sound, a few ms after, in correspondence with the end of the sound, or after the end it can wait for a random delay. The spectral centroid recorded on the buffer is applied to the pitch of two additive synths both displaying independent behavior in many other parameters. The probability works independently for the two instances of the synths. The reading speed of the buffer is random and often is extremely slowed down. On top of that there is the possibility that the reading freezes so that the timbral independent movement of the synths can emerge from the sustained sound.

Activating condition - Detection of 25th event.

Ending condition - Detection of special event 1.

Occurrence - Probabilistic, related to the amp threshold.

Relationship 8 - Electro patterns.
This relationship is the one responsible for those patterns that provide some melodic and rhythmic material. The pitches of the patterns are the most prominent partials of an FFT frame. When the actor is called, a FFT frame is taken from the input sound, and a random pattern is chosen probabilistically. If for example the pattern has a rhythm of 11 beats, the most prominent 11 partials of the FFT frame are used as pitch material for the pattern. The pattern also can have one or two notes that are two beats long. The pattern is looped and the loop can change during the occurrence of the relationship, but with a very low probability. Each occurrence has an overall duration that ranges from 30 to 60 sec.

Activating condition - Detection of 25th event. (actually, a safer equivalent: relationships 5 and 7 are active)

Ending condition - Three occurrences.

Occurrence - When the input sound is longer than 6 seconds. Probability 50% OR after 2 detections. After each occurrence the module wait 40 sec before it can be used again.

• Relationship 9 - Bowed string-like instrument.
This actor responds directly to the spectral centroid of the input. The centroid is mapped directly to the pitch of a recursive FM synth, and at the same time it is recorded into a buffer. When the input sound stops, the buffer is looped and continues to control the pitch of the synth. The loop goes through a series of generative processes: it can remain constant, slow down, or speed up. Some parameters of the synth can be changed during the looping time. When they change, they follow a direction and cannot go back until a new loop is used.

Activating condition - 3 occurrences of relationship 8.

Ending condition - Activation of relationship 10.

Occurrence - Continuous.

• Relationship 10 - Percussive sound.
This actor alone occupies the entire second section of the work. It is different from all the others in that it uses the data stored in the long term memory of the system, and operates when the performative gestures are over. This is one of the most tangetial relationship possible. The time intervals of the percussive synthetic sounds are the reverse of the intervals of the onsets of the sounds analyzed in the first section. Their spectral content is a result of data recombination of the spectra of the analyzed sounds: the most prominent partial of the last 18 events are used to create the spectrum. They are transposed in range to make sure that the spectrum is balanced but they are left unaltered, so that any possible harmonic or inharmonic relationship is the result of an emergent behavior. The overall duration of this section is the smaller part of the golden ratio of the overall duration of the first section.

Activating condition - After the special event 1 has been detected, wait for the detection of an event that is longer than 6 seconds, when that happens, wait until there is no sound, than wait for a duration that is equal to the detected event and activate the module.

Ending condition - The condition is met when the internal clock of this relationship that starts with its activation is major equal to the duration of the first section multiplied by 0.382 (golden ratio).

Occurrence - Continuous.

• Special event 1
If relationship 9 is active, detect a sound longer than 40 seconds.