/ 
Sonic communication Imagine an orchestra in which the musicians do not play according to notes, but instead play a very individual tone when they have been sufficiently stimulated by the tones of other musicians. This is exactly what happens with the Sonapticon: the dynamically changing, rhythmic tone sequences and cluster-like bursts of sound are the quasi-social product of acoustic communication between loudspeakers distributed throughout the room.
Light as complementary dramaturge When a loudspeaker joins in the communication, not only its characteristic sine tone is emitted, but it also flashes in white. Even if a speaker does not play a sound, the changing colours reveal the complementary play of mutual excitement but also inhibition. The walk-in, immersive sound space thus becomes an electro-acoustic music theatre that gives the impression of being a living organism through the breathing colours.
Playing and composing music with networks Once you have got a feel for the self-organising processes, you can play and compose music with the loudspeaker orchestra in a completely new way. Depending on the composition, the speakers are assigned new microtonal tones and are linked to other speakers - but the actual piece is formed in the spatio-temporal interaction. In this way, music radically shifts from a pre-composed sound sequence to a spatially performative result of an interacting sound network that is anything but random. As a result, composition turns into a process of creating networks and the associated conceptualisation of events, exciting but also inhibiting certain sound interactions through external stimuli, which can also originate from visitors or musicians.
When music & art are getting nervous The Sonapticon is literally nervous, as the way in which sounds stimulate and inhibit each other reflects fundamental electrophysiological processes in nerve cells that make us sentient and thinking beings. This neuronal jam session thus implicitly provides an idea of the dynamic feedback processes that underlie all cognitive acts, including music-making, and whose complexity remains a mystery to us.
3D-Panorama at the Tieranatomisches Theater. Please click for sound and full size.
Innovation From the electronics to the polycarbonate housing, the loudspeaker sculptures were developed in-house by imachination labs. Energy-saving microelectronics are responsible for analysing the frequency (FFT) of the microphone recording. The core of the Sonapticon is a biological neuron model that was specifically implemented in the firmware. The sound laboratory thus becomes a counter-model to AI, which despite its massive resources cannot structurally create anything new: The 70 interacting loudspeakers not only illustrate the neurological foundations of biological intelligence, but also show how these neuromorphic networks create unprecedented music with minimal resources.
Sonaptic mobilization: Theatre of Memory The Sonapticon was realized for the first time in 2012 as a large-scale studio version with 43 speakers in the context of a residency at the Klangdom at the ZKM Karlsruhe. The most recent version makes the whole system mobile, flexible and even more immersive creating custom made audio-neuron sculptures distributed over the whole space. Here, the imachination labs enhanced the loudspeaker sculptures, which were already successfully used for the sound installation [aiskju:b] emitting tones in 360 degrees. The electronics is extended by two special microphones, so the speakers can record sound in real time triggering the neuron model running autonomously on the speakers' micro processors. Above all, the colour illumination indicates the type of neuron and the current state of excitation.
"Theatre of Memory" premieres on January 2024 at the Animal Anatomical Theatre in Berlin.
Accompanying symposium on "neuron aesthetics" at Humboldt University Berlin.
Theatre of Memory at Max-Planck-Institut for Empirical Aesthetics, Frankfurt.
The lunch talk at Max-Planck-Insitut for Empirical Aesthetics in September 2023 was a kind of warming up for the upcoming collaboration with the institutes researchers at the context of the EU funded S+T+Arts ReSilence program. Credit: imachination projects
The basic unit of the Sonapticon is formed by audio neurons, which, in contrast to nerve cells, are not connected by conductive fibres but by sounds moving in space. Credit: imachination projects
Concept study for the Theatre of Memory within the framework of the programme Neustart Kultur.
The studio version Sonapticon premiered in 2012 at the Sounddom of the ZKM Karlsruhe. Credit: imachination projects
Premiere @ TAT Berlin The Theatre of Memory will premiere with a presentation of 70 audio neurons in the Tieranatomische Theater in heart of Berlin from 12 January - 23 March 2024.
The Theatre of Memory is part of the programme of the successful exhibition The Brain in Science and Art at the Berlin Museum of Medical History. The exhibition runs until September 8, 2024.
MPI-EA Frankfurt The Theatre of Memory is co-funded in the context of a ReSilence residency funded by the STARTS program by the European Commission. Part of that residency will be a follow up presentation at the Max-Planck-Institute for Empirical Aesthetics in Frankfurt from 15-28 February 2024.
Peri-Sonapticon A first pilot project with 40 audio-neurons was funded in 2021 by the Beauftragten der Bundesregierung für Kultur und Medien as part of Neustart Kultur
with project funds from the Bundesverbands Bildender Künstlerinnen und Künstler.
Most people do know feedback in acoustic systems as a cascading effect of a microcophone in combination with a loudspeaker as explored artistically by Jimmy Hendrix with his guitar. The Sonapticon uses a similar but different scheme. The basic unit of the Sonapticon are audio neurons interconnected not by wires but by sound transmitted in space. An audio neuron registers impulses spikes) from its acoustic environment by a microphone and fires its own impulse by a loudspeaker. The clue of that system is that every neuron gets assigned a sine tone with an individual frequency.
In the conference paper Sonapticon - space as an acoustic network you will find further reflections on the project idea.
Biological neurons are interconnected by organic wires and communicate by short electric impulses, so-called 'spikes'.
On the left you see a simple scheme of how a neuron works: The incoming electric impulses are summed up and change the membrane
potential of a neuron. There are two different types of incoming spikes: if the spike increases the membrane potential,
the sending neuron is called excitatory, if it decreases the potential, the sender is called inhibitory.
In the absence of external input, the membrane potential decays to a resting potential which is marked by the green ring. A neuron
sends out an electric impulse, it 'fires', when the incoming impulses from other neurons force the potential over a certain threshold,
here indicated by the red ring. After having fired the neuron is not excitable for a certain period of time and the potential goes back
to the resting potential.
This simple interaction scheme is the basis of all nervous activity.
It is still an enigma how these interactions form something like a simple thought in our brain. With latest microscopic techniques you
can observe in vivo in small probes of neuronal tissues how firing neurons self-organize and create synchronized activity patterns.
Unlike in in vivo neural tissue, where neuronal interactions occur on the timescale of milliseconds, the in silico technique of the
Sonapticon allows us to decelerate all biological processes. This generates an environment where spectators can identify and
understand intuitively the causal relationship between integration and initiation of spikes on a single neuron level.
Consequently the Sonapticon is a system of a decelerated synthetic neurons which allows
for a real time interaction with neuronal dynamics and control of the biological parameters. The Sonapticon combines two basic
components:
- digital in silico methods, with latest mathematical models of biological neurons (e. g. the conductivity based model b
Alain Destexhe) and
- an empirical environment (the Klangdom) using acoustics as an analogue space of experimentation and interaction.
Furthermore, using sound waves as the transmitting medium, the quasi-living Sonapticon evaporates the requirement of a well
defined input-structure, like e.g. the sensory system of the human that feeds information into its brain. The borders between inner
world and outer world wear out, as individual neurons in this artificial brain can be directly stimulated by playing the
corresponding frequencies!
Looking to a sound spectrum you see that most sounds do consist of a broad band of frequencies. But sine tones oscillating at a single
frequency show up characteristic peaks in a spectrum. This is how you can simply connect the audio neurons: Every audio neuron
registers a specific set of frequencies assigned to other neurons. These frequencies are marked as yellow and blue lines in the
spectrum. If an audio neuron registers a significant steep peak at one of these lines it interprets this as impulse of a connected
neuron and the line blinks in red. As you see in the change of the audio neuron's potential peaks at the blue lines are interpreted
as excitatory and detected pitches at the yellow lines are interpreted as an inhibition. On the bottom you see a plot with the history
of the changing potential which helps to understand the dynamics of an individual neuron.
Principally every computing device with a microphone and
loudspeaker can function as an audio neuron to be it a laptop, a tablet computer or a smart phone. In August 2012 a first performance
did take place at the Bernstein Centre for Computational Neuroscience on the Campus of the Charité in Berlin. The visitors installed a
little software to be it for PC or Mac and changed their laptops with small external speakers into acoustic neurons. Step by step a
network of twenty acoustic neurons was built up, exploring the changing dynamics by adding further neurons.
Above all the Sonapticon invites to interact with the system in all a different forms. An instrument like the singing saw producing
clear sine peaks resulted to be a perfect device to explore the system's resonances.
He has shown the functioning of the Sonapticon in collaboration with the neuromathematician Benjamin Staude as a guest artist of the ZKM Karlsruhe demonstrated. In 2012, the successful premiere took place in the Klangdom there with 43 studio speakers involving three piccolo flutists for the IMATRONIC Festival.
In the ZKM's Klangdom the audio analysis
uses a frame work based on MaxMsp programmed by Holger Stenschke. The adaptation of the neuron model and the composition frame work was
realized in Python by Benjamin Staude. For the visualization Tim Otto Roth used Gem in combination with Puredata.
Already since 2007 Tim Otto Roth has been experimenting with the acoustic translation of self-organization principles. In collaboration with the group ?'XL, he worked in 2008
with a specially created project choir on the translation of so-called cellular automata to the singers of the choir: Music of Life.
In 2011 he transferred the concept of these to strings in a concert at the ZKM Karlsruhe. The idea for the Sonapticon emerged from a conversation with Eugenio Fava,
a neurobiologist then conducting research at the Max-Planck-Institute for Molecular Cell Biology and Genetics (MPI-CBG), after the first choir concert at the German Hygiene Museum in
Dresden. At the invitation of Ludger Brümmer, the director of the Institute for Music and Acoustics, he finally got the opportunity together with the biomathematician
Benjamin Staude and the sound engineer Holger Stenschke in the Klangdom of the ZKM to turn the idea into acoustic reality. The development of the Sonapticon is documented in a blog.
Until now, a presentation has been linked to a large-scale studio like the Klangdom, as the whole activity is still elaborately controlled by a central computer.
The Peri-Sonapticon makes the whole system mobile and flexible, in that each Audioneuron sculpture itself performs the sonic processing on a microprocessor.
The conceptual artist and composer Tim Otto Roth combines art and science in a novel way. With his expansive sound sculptures such as Heaven's Carousel (2014),
consisting of 36 rotating loudspeakers, and the aura calculata water organ from 2016, he has succeeded in opening up new aesthetic experiences and thus new avenues
for art by engaging with contemporary scientific research. In his compositional work, Roth uses space as an (additive) synthesizer in which tones emitted by acoustic sources
distributed throughout a space fuse into site-specific sounds. Apart from his special method for spatializing sound, he focuses his work on microtonal scales whose "harmonics"
can be derived from specific physical processes. In June 2018, his sound installation SMART>SOS celebrated its premiere at IRCAM – Centre Georges Pompidou in Paris.
Since summer 2018, he has been showing the immersive sound sculpture [aiskju:b], which consists of 444 illuminated loudspeakers and is played with data from the IceCube
observatory, at the Kulturkirche St. Elisabeth in Berlin-Mitte, at the Reaktorhalle in Munich and at the Ludwig Forum for International Art in Aachen.
www.imachination.net
Transdisciplinary symposium on neuron-aesthetics
Since ancient times, the representation of perceptions, thoughts and biographical episodes in memory has been associated with a theatre: consciousness as a stage on which the contents of memory are enacted. With the Theatre of Memory, a corresponding music-theatrical orchestration was created for the Tieranatomisches Theater (TA T) in Berlin.
Transdisciplinary ties will be established by an accompanying symposium on 26./27. January that will bring together scientists and stakeholders from the life sciences and humanities, but also from music. Together, they will explore in dialogue the neuro-theatrical boards that, in Friedrich Schiller's proverbial sense, mean and construct the world. The focus here is on the idea of "neuron-aesthetics", which, unlike neuroaesthetics, does not primarily address cross-brain interaction patterns, but formulates a paradigm shift that goes hand in hand with the microscopic focus on single-neuron research. One of the key topics of the symposium will be memory, the 'Godot' of our neuronal theatre. Here, the music comparisons repeatedly used in neuroscience point to the importance of the spatial and temporal interplay of neurons: "much like a group of musicians in an orchestra who need to be coordinated to create a harmonic symphony, activity of neuronal ensembles must be precisely synchronized to form intact memory." (Kol & Goshen 2021).
The neuron doctrine, which understands neurons as actors in the theatre of memory, will not only be questioned from a physiological perspective, but also contextualised in terms of the history of science and reflected upon from a philosophical perspective: It will be asked whether the world stage we experience and the underlying cognitive processes can be reduced to neurobiological processes. In addition, the interaction of music with neuronal networks and the conception of music as a spatio-temporal sound network will be discussed. The conference thus also responds to the current debate on artificial intelligence by emphasising the fascinating symphonic complexity of living intelligence.
Conception: Prof. Dr. Christoph Ploner, Tim Otto Roth und Miriam Seidler in cooperation with Prof. Dr. Petra Ritter, Prof. Dr. Thomas Schnalke and Felix Sattler.
Program
Freitag, 26. January 2024
13:30
Matthew Larkum, Berlin: Introduction
14:00
Panel 1
Livia de Hoz, Berlin: Making sense of sound up (and down) the auditory circuit hierarchy
Bernhard Seeber, München: From acoustic rooms to neurons and back
Moderatior: Katja Naie
15:30
Panel 2
John-Dylan Haynes, Berlin: Wissenschaft, Kunst, Freiheit
Cornelius Borck, Lübeck: "What you see is what you get"
Moderatior: Felix Sattler
COFFEE BREAK
17:30
Panel 3
Marina Mikhaylova, Berlin: Calcium Symphony: Conducting Neuronal Trafficking for Synaptic Plasticity
Beatrice de Gelder, Maastricht: NN
Moderatior: Christoph Ploner
BREAK with the opportunity to visit the exhibition in the Animal Anatomy Theater
20:00
Abendpanel
Symphonische Komplexität. Tim Otto Roth im Gespräch mit Matthew Larkum und Jan St. Werner (German)
Samstag, 27. Januar 2024
9:30 h
Panel 4
Petra Ritter: Was können digitale Gehirn-Zwillinge?
Sebastian Gießmann: Lebendige Netzwerke: Kulturtechniken, Medien, Wissen
Moderatior: Miriam Seidler
11:00h
Panel 5
David Owald: Wie Netzwerkprinzipien Schlaf und Erinnerungen in der Fliege ermöglichen
Max Stadler: Zelle, Material, Moderne: Modellwelten nervöser Biophysik?
Moderatior: Christoph Ploner
BREAK with the opportunity to visit the exhibition in the Animal Anatomy Theater (TA T)
ABSCHLUSS
Exhibition tour with the curators Thomas Schnalke and John-Dylan Haynes in the exhibition The Brain in Science and Art at Charité's Museum of Medical History.
Thanks
As networking event sponsored by Stiftung Charité .
Venue:
Humboldt-Universität zu Berlin
Campus Nord, Haus 4
Philippstr. 13
10115 Berlin
.
February 24 Theatre of Memory at Max Planck Institute for Empirical Aesthetics in Frankfurt. The vernissage begins on 15 February at 7 p.m.
20 February 2024 How did you do you that?,Artist talk at Max Planck Institute for Empirical Aesthetics, Frankurt.
Symposium On 26/27 January 2024, an accompanying symposium invited to a transdisciplinary exchange. It brought together scientists and stakeholders from the life sciences and humanities, but also from the arts. Together, they explored in dialogue the neuro-theatrical boards that, in Friedrich Schiller's proverbial sense, mean and construct the world. Here, memory functions as the great unknown, the 'Godot'. How cognitive content is represented on the levels of cells and networks still remains a great mystery.
Publication Simulating, exploring and optimizing the spatial sound scene of the Sonapticon. Bernhard U.Seeber, Johannes Kurz, Tim Otto Roth, in: Proceedings of the 10th Convention of the European Acoustics Association Forum Acusticum 2023, January 2024, pages 4321-4325.
Save the date Opening of the Theatre of Memory at the Tieranatomisches Theater in Berlin on Thursday 11 January 2024 at 6:30 p.m. with words of welcome by Prof. Dr. Christoph Ploner (Charité/SFB1315) and Dr. Elke Lange (MPI-EA) and an inaugural speech by Dr. Andreas Beitin (Kunstmuseum Wolfsburg).
October 2021 After intensive months of preparation, on 3 October 2021 the Peri-Sonapticon celebrated a small world premiere at the imachination labs in Oppenau as part of a private view with friends of the studio. Special guest of the evening was the Strasbourg based musician Yérri-Gaspar Hummel, who excited the 41 audio neurons with his saxophone.
Visitors during one of the three presentations at the imachination labs. Credit: imachination projects
Manuel Prugel at work during one of the long nights before the premiere. The audio-neurons are technically based on the microelectronics of the [aiskju:b] project. Nevertheless, the firmware had to be completely redesigned. Credit: imachination projects
September 2021 The centerpiece of the new Sonapticon has gone into production: the boards for the audio neurons, which include microelectronics
with a microphone, LED light and sound output.
Special thanks to: Benjamin Piltz (electronics and assembly), Manuel Prugel (programming) and Miriam Seidler (film).
April 2021 Funded by the Neustart Kultur program, a mobile version of the Sonapticon
is currently being developed.
Performance dates will be announced here soon.
Wenn Lautsprecher hören, by the editiorial team, Offenburger Tageblatt, 20 January 2024.
27 October 2021, Vernetzt wie Nervenzellen, by Rainer Braxmaier, Offenburger Tageblatt
September 2019, Soundart edited by Peter Weibel shows on three double pages the Heaven's Carousel, the Sonapticon and the water organ aura calculata,
MIT Press 2019.
April 2016, Seizing Attention: Devices and Desires, by Barbara Maria Stafford, Art History, Volume 39, Issue 2,
pp. 422–427.
October 2015 Aura Calculata –
die Klang- und Lichtkunst von Tim Otto Roth siedeln an der Schnittstelle von Kunst und Wissenschaft, by Helga de la Motte-Haber,
Neue Zeitschrift für Musik 05/2015, pp. 34-37
Artist collaborates with neuroscientist to build 'audio-neurons', Interview von Robert Barry, wired.co.uk,
10. Dezember 2012.
Hören, wie Nervenfasern sprechen, Acher-Rench-Zeitung 21. November 2012