/ 
The lunch talk at Max-Planck-Insitut for Empirical Aesthetics in September 2023 was a kind of warming up for the upcoming collaboration with the institutes researchers at the context of the EU funded S+T+Arts ReSilence program. Credit: imachination projects
The basic unit of the Sonapticon is formed by audio neurons, which, in contrast to nerve cells, are not connected by conductive fibres but by sounds moving in space. Credit: imachination projects
Simulation for the upcoming presentation in the Animal Anatomical Theatre, Berlin
Concept study for the Theatre of Memory within the framework of the programme Neustart Kultur.
The studio version Sonapticon premiered in 2012 at the Sounddom of the ZKM Karlsruhe. Credit: imachination projects
When music & art are getting nervous. With the Sonapticon music becomes proverbially nervous: an entire room is transformed into a network of interacting tons, reflecting fundamental processes in nerve cells that make us sentient and thinking beings. The walk-through the immersive sound space of communicating speakers not only makes it possible to immerse yourself in the network structure, but at the same time you can interact with the so called audio-neurons through tones and sounds. Once you get a feel for the processes, the Sonapticon allows you to make music in a completely new way - music that gives you an idea of the cognitive processes that still remain a mystery to us in their complexity.
Sonaptic mobilization: Theatre of Memory & Peri-Sonapticon Until now, a presentation did afford a large-scale studio such as the Klangdom at the ZKM Karlsruhe, as the neuronal activity was elaborately controlled by a central computer. The Peri-Sonapticon makes the whole system mobile and flexible by the fact, that each audio-neuron sculpture itself performs the sonic processing on a microprocessor. Here, Tim Otto Roth enhances the loudspeaker sculptures, which he has already successfully used for the sound installation [aiskju:b]. The electronics is extended by two special microphones, so the speakers can emit and record sound autonomously in 360 degrees (peri = circular). The Peri-Sonapticon is thus composed of individual autonomous units like parallel processing network of nerves and thus reacts much more directly to the analogue space - a significant difference to the studio version.
The pilot project with audio-neurons was funded in 2021 by the Beauftragten der Bundesregierung für Kultur und Medien as part of Neustart Kultur with project funds from the Bundesverbands Bildender Künstlerinnen und Künstler. An expansion of the network to 125 Audioneurons is planned for 2023, which will premiere as Theatre of Memory at the Tieranatomisches Theater in Berlin.
Most people do know feedback in acoustic systems as a cascading effect of a microcophone in combination with a loudspeaker as explored artistically by Jimmy Hendrix with his guitar. The Sonapticon uses a similar but different scheme. The basic unit of the Sonapticon are audio neurons interconnected not by wires but by sound transmitted in space. An audio neuron registers impulses spikes) from its acoustic environment by a microphone and fires its own impulse by a loudspeaker. The clue of that system is that every neuron gets assigned a sine tone with an individual frequency.
In the conference paper Sonapticon - space as an acoustic network you will find further reflections on the project idea.
Biological neurons are interconnected by organic wires and communicate by short electric impulses, so-called 'spikes'.
On the left you see a simple scheme of how a neuron works: The incoming electric impulses are summed up and change the membrane
potential of a neuron. There are two different types of incoming spikes: if the spike increases the membrane potential,
the sending neuron is called excitatory, if it decreases the potential, the sender is called inhibitory.
In the absence of external input, the membrane potential decays to a resting potential which is marked by the green ring. A neuron
sends out an electric impulse, it 'fires', when the incoming impulses from other neurons force the potential over a certain threshold,
here indicated by the red ring. After having fired the neuron is not excitable for a certain period of time and the potential goes back
to the resting potential.
This simple interaction scheme is the basis of all nervous activity.
It is still an enigma how these interactions form something like a simple thought in our brain. With latest microscopic techniques you
can observe in vivo in small probes of neuronal tissues how firing neurons self-organize and create synchronized activity patterns.
Unlike in in vivo neural tissue, where neuronal interactions occur on the timescale of milliseconds, the in silico technique of the
Sonapticon allows us to decelerate all biological processes. This generates an environment where spectators can identify and
understand intuitively the causal relationship between integration and initiation of spikes on a single neuron level.
Consequently the Sonapticon is a system of a decelerated synthetic neurons which allows
for a real time interaction with neuronal dynamics and control of the biological parameters. The Sonapticon combines two basic
components:
- digital in silico methods, with latest mathematical models of biological neurons (e. g. the conductivity based model b
Alain Destexhe) and
- an empirical environment (the Klangdom) using acoustics as an analogue space of experimentation and interaction.
Furthermore, using sound waves as the transmitting medium, the quasi-living Sonapticon evaporates the requirement of a well
defined input-structure, like e.g. the sensory system of the human that feeds information into its brain. The borders between inner
world and outer world wear out, as individual neurons in this artificial brain can be directly stimulated by playing the
corresponding frequencies!
Looking to a sound spectrum you see that most sounds do consist of a broad band of frequencies. But sine tones oscillating at a single
frequency show up characteristic peaks in a spectrum. This is how you can simply connect the audio neurons: Every audio neuron
registers a specific set of frequencies assigned to other neurons. These frequencies are marked as yellow and blue lines in the
spectrum. If an audio neuron registers a significant steep peak at one of these lines it interprets this as impulse of a connected
neuron and the line blinks in red. As you see in the change of the audio neuron's potential peaks at the blue lines are interpreted
as excitatory and detected pitches at the yellow lines are interpreted as an inhibition. On the bottom you see a plot with the history
of the changing potential which helps to understand the dynamics of an individual neuron.
Principally every computing device with a microphone and
loudspeaker can function as an audio neuron to be it a laptop, a tablet computer or a smart phone. In August 2012 a first performance
did take place at the Bernstein Centre for Computational Neuroscience on the Campus of the Charité in Berlin. The visitors installed a
little software to be it for PC or Mac and changed their laptops with small external speakers into acoustic neurons. Step by step a
network of twenty acoustic neurons was built up, exploring the changing dynamics by adding further neurons.
Above all the Sonapticon invites to interact with the system in all a different forms. An instrument like the singing saw producing
clear sine peaks resulted to be a perfect device to explore the system's resonances.
He has shown the functioning of the Sonapticon in collaboration with the neuromathematician Benjamin Staude as a guest artist of the ZKM Karlsruhe demonstrated. In 2012, the successful premiere took place in the Klangdom there with 43 studio speakers involving three piccolo flutists for the IMATRONIC Festival.
In the ZKM's Klangdom the audio analysis
uses a frame work based on MaxMsp programmed by Holger Stenschke. The adaptation of the neuron model and the composition frame work was
realized in Python by Benjamin Staude. For the visualization Tim Otto Roth used Gem in combination with Puredata.
Already since 2007 Tim Otto Roth has been experimenting with the acoustic translation of self-organization principles. In collaboration with the group ?'XL, he worked in 2008
with a specially created project choir on the translation of so-called cellular automata to the singers of the choir: Music of Life.
In 2011 he transferred the concept of these to strings in a concert at the ZKM Karlsruhe. The idea for the Sonapticon emerged from a conversation with Eugenio Fava,
a neurobiologist then conducting research at the Max-Planck-Institute for Molecular Cell Biology and Genetics (MPI-CBG), after the first choir concert at the German Hygiene Museum in
Dresden. At the invitation of Ludger Brümmer, the director of the Institute for Music and Acoustics, he finally got the opportunity together with the biomathematician
Benjamin Staude and the sound engineer Holger Stenschke in the Klangdom of the ZKM to turn the idea into acoustic reality. The development of the Sonapticon is documented in a blog.
Until now, a presentation has been linked to a large-scale studio like the Klangdom, as the whole activity is still elaborately controlled by a central computer.
The Peri-Sonapticon makes the whole system mobile and flexible, in that each Audioneuron sculpture itself performs the sonic processing on a microprocessor.
The conceptual artist and composer Tim Otto Roth combines art and science in a novel way. With his expansive sound sculptures such as Heaven's Carousel (2014),
consisting of 36 rotating loudspeakers, and the aura calculata water organ from 2016, he has succeeded in opening up new aesthetic experiences and thus new avenues
for art by engaging with contemporary scientific research. In his compositional work, Roth uses space as an (additive) synthesizer in which tones emitted by acoustic sources
distributed throughout a space fuse into site-specific sounds. Apart from his special method for spatializing sound, he focuses his work on microtonal scales whose "harmonics"
can be derived from specific physical processes. In June 2018, his sound installation SMART>SOS celebrated its premiere at IRCAM – Centre Georges Pompidou in Paris.
Since summer 2018, he has been showing the immersive sound sculpture [aiskju:b], which consists of 444 illuminated loudspeakers and is played with data from the IceCube
observatory, at the Kulturkirche St. Elisabeth in Berlin-Mitte, at the Reaktorhalle in Munich and at the Ludwig Forum for International Art in Aachen.
www.imachination.net
October 2021 After intensive months of preparation, on 3 October 2021 the Peri-Sonapticon celebrated a small world premiere at the imachination labs in Oppenau as part of a private view with friends of the studio. Special guest of the evening was the Strasbourg based musician Yérri-Gaspar Hummel, who excited the 41 audio neurons with his saxophone.
Visitors during one of the three presentations at the imachination labs. Credit: imachination projects
Manuel Prugel at work during one of the long nights before the premiere. The audio-neurons are technically based on the microelectronics of the [aiskju:b] project. Nevertheless, the firmware had to be completely redesigned. Credit: imachination projects
The saxophon player Yérri-Gaspar Hummel interacting with the audio neurons.
September 2021 The centerpiece of the new Sonapticon has gone into production: the boards for the audio neurons, which include microelectronics
with a microphone, LED light and sound output.
Special thanks to: Benjamin Piltz (electronics and assembly), Manuel Prugel (programming) and Miriam Seidler (film).
April 2021 Funded by the Neustart Kultur program, a mobile version of the Sonapticon
is currently being developed.
Performance dates will be announced here soon.
27 October 2021, Vernetzt wie Nervenzellen, by Rainer Braxmaier, Offenburger Tageblatt
September 2019, Soundart edited by Peter Weibel shows on three double pages the Heaven's Carousel, the Sonapticon and the water organ aura calculata,
MIT Press 2019.
April 2016, Seizing Attention: Devices and Desires, by Barbara Maria Stafford, Art History, Volume 39, Issue 2,
pp. 422–427.
October 2015 Aura Calculata –
die Klang- und Lichtkunst von Tim Otto Roth siedeln an der Schnittstelle von Kunst und Wissenschaft, by Helga de la Motte-Haber,
Neue Zeitschrift für Musik 05/2015, pp. 34-37
Artist collaborates with neuroscientist to build 'audio-neurons', Interview von Robert Barry, wired.co.uk,
10. Dezember 2012.
Hören, wie Nervenfasern sprechen, Acher-Rench-Zeitung 21. November 2012