Printed: 2020-05-27

Institute for Ethics and Emerging Technologies

IEET Link:

Future Emotions: How Synaesthesia, Technology and Experimental Music Dilate Our Internal Geographies

Rory Viner

Ethical Technology

March 03, 2016

What follows is a description of experiments using the medium of sound and emerging sound technologies to destabilize conceptions and reposition ourselves to histories of social trauma and to our own sense of self. Can we interrogate these intersections of meaning and data in new ways? Can we dilate emotional immediacy through such re­imaginings? Does technology allow and help us to reconsider these approaches?

1 ­ Aural Psychotopology and Social Trauma

Statistics. What are these artifacts, buried not in soil but through emotional histories? How are our histories expressed and felt? Would we be able to reposition ourselves to them in unique ways? Are there more human, that is to say, emotional ways to represent historical records? What would happen if we lived in a society that recalled social statistics not by tables of numbers, but through sound. The human mind can understand but not relate emotionally to collections of statistics. Individuals that comprise these statistics lose their agency and I would argue that because of this a certain humanity is lost. To destabilize this emotional distance we need to reposition ourselves to the data. I suggest this can be achieved through making historical records into audible sound. Humans will always have a raw, emotional response to sound over written documents, allowing us to collapse the emotional distance and rethink how we approach historical records. I tried to recapture this emotional relationality through the sonification of data.

I chose to use suicide statistics to create a psychotopological sonic map of trauma in Japan of suicide. By using government statistics from the last full year of available statistics, I pulled the data from each of the 47 prefectures of Japan for each month. I then organized the data into a program called pure data to trigger a synthesizer. Each note would represent the total number of suicides for one month in one specific prefecture. All prefectures would cycle through, playing 47 notes for January then on to the next month for a total of 564 notes for the year.

Not only does the resulting score allows us to approach social trauma in a unique way, collapsing this emotional distance, but we begin also to rethink how physical geographies are represented. Only by image maps? Reminiscent of the songlines of the aborigines of Australia who created geographical maps in their head through secret melodies to waterholes and resources, can other kinds of mental maps be realized through sound? Does representing emotional topologies make us more sensitive to its change? I also think this experiment allowed us to challenge traditional ideas of record keeping and the primacy we give to text and images.

Audio link to suicide sonification

2­ The Dilation of Internal Geographies ­ Sex Sensors and Sound

In the previous experiment, sound data brought us closer to something, but what if it could also alienate and disassociate us from an experience. The next experiment would do just that through the use of biosensors and the sonification of desire and sex. The dilation of our internal geographies within the self can be destabilized through the mere act of data collection, regardless of the end use of such collected data.

In this second experiment, I wished to explore the increasing desire to log data about ourselves and what this means to the positioning or coordinating of self as extended through sonic feedback and music technologies. The corporeal self is constantly being self monitored and other­monitored as our technological self extends. As the intimate becomes public, how does the emotional distance change? What is more intimate than our sex acts? What if the biometric data from our sex were exchanged or collected live? Would we feel more close or distant to ourselves and our partners?

In my this experiment, I transformed sexual kinesthetic movements into sound. To accomplish this, I attached various piezoelectric sensors to myself and my partner at the waist and limbs and streamed and recorded the live output. Each piezoelectric sensor was programmed to trigger a different, specific note within an Arduino sketch after the sensors were connected to a microcontroller. The notes as MIDI data were then sent out of the Arduino program and into a synthesizer. This output was streamed live to be heard during intercourse in order to create a feedback loop where both the sound and our movement would interact with each other­ our bodies instrumental, editing our actions in real time to change our behavior in order to change the musical output. The results were not as predicted. I had previously thought that this feedback loop would create a greater sympatico or feelings of connectedness, but the experience itself was quite different. Rather than collapsing distance, in fact, this created a distance that interfered with the emotional immediacy of the event, creating a sort of depersonalized effect.

Audio link to sex data sonification

Transhumanism and the rethinking of music

The possibilities of and what we think of as music has been constantly shifting. Periodically, we have stabilized periods with established ways of performing and writing music. Through technology, what are entrenched production methods, as well as ways to enjoy music, are shaken up and the possibilities of music are rethought. As technology and humanity begin to merge, one might ask how this will impact the arts. We often consider the political, economic and biological aspects of transhumanism, yet one wonders what art will be like in the future. Big data, pervasive presence extend, speed, complexity, analogy, metaphor, and time will transform the future of the arts. Another important point that will come into conflict is to consider the search for authenticity, agency, authorship when AI emerges.

However, technology will allow us repurpose pre existing fields and unite them in before unthought of ways. Not only does technology allows us to create new things, but allows us to connect old things that were once disparate. Like a form of synesthesia, we will be able to reconsider things unlike before through a re­negotiation of internal psychotopological spaces. Reconsidering relationships to phenomena is key.


Through these experiments, we see how it is possible to destabilize our traditional ways of experiencing data and thus truth, and how positioning ourselves to social trauma and history can be changed in dramatic ways through thus use of unconventional approaches. Dilating emotional immediacy and re­imagining representation and relationally through sound technologies can give us a unique way to connect with statistics that would traditionally feel distant and cold. However, at the same time sonification techniques can also distance us from our physical experience, making it unfamiliar and alienating. Yet, I maintain that granting ourselves greater relationally to the emotions we feel through emerging technologies could increase the humanness in which we make sense of our world. We must not take for granted how our internal geographies mapped.

Not only does technology allows us to create new things, but allows us to connect old things that were once disparate. Like a form of synesthesia, we will be able to reconsider things unlike before through a renegotiation of how we make sense of the world. People often presume transhumanism is about losing our humanity through technology, but rather I think it is equally possible that we can find it through technology. Dealing with the double edge sword that equally grants deeper connections and deeper alienation at the same time will pose a unique existential challenge in the next century.

Rory Viner is a sound artist and experimental composer based in Tokyo, Japan. His works have been featured in Wired Magazine, Vice's Motherboard & The Japan Times as well as various other international news outlets. His current work focuses on synesthesia (the mixing of senses), sensors, data and sound.


Contact: Executive Director, Dr. James J. Hughes,
IEET, 35 Harbor Point Blvd, #404, Boston, MA 02125-3242 USA
phone: 860-428-1837