SYDNEY – A team of Australian researchers is tapping into the nerves and bodies of volunteers to create a raw musical performance driven purely by emotions. In a world first, the data is fed into special software, which will create music that will be played by small robots. The research team comprises a neurophysiologist, a roboticist and an artist.
What sound does a singing body make?
To measure these inner feelings, Sydney researchers are carefully mapping volunteers' blood pressure, breathing and sweat release.
They are also inserting small needles into the nerves to listen into the electrical signals coming from the brain.
"So amongst other things, I study the sympathetic nervous system, which is the branch of the nervous system involved in emotional expression," explains neurophysiologist Vaughan Macefield, professor of integrated physiology at the University of Western Sydney's School of Medicine. "And that is what we are looking at today. So I have a needle inserted into Ben’s knee, the common peroneal nerve at the side of the knee, and as you can hear if I stroke Ben’s foot, we can hear this noise, which actually reflects the activity of sensory endings in the skin.”
The information is fed into a central computer, which decodes the data and translates it into music that fits the emotional state of volunteer, Ben Schulz.
“Currently, I have got all sorts of wires coming in and out of me," Schulz says, "so I have got something to measure my pulse and my blood pressure, and I have also got a small, little needle inserted into my leg, going into a nerve to actually sense the nerve endings in my feet.”
Professor Macefield says the experiment aims to produce unique music.
“The whole idea of this project is to tap into these nerve signals, and with Erin over there, [a] Montreal-based performance artist, we are wanting to use these signals, which are very sensitive markers of the state of emotion, to drive a performance,” he explains.
The experiments at a laboratory at the University of Western Sydney are the start of something far more spectacular. The team aims to put their emotions-driven show on stage, where the inner feelings of two volunteers will generate live music performed by a fleet of small robots.
“This project for me is all about a spirit of wonder and discovery," says Erin Gee, a Canadian sound and new media artist. " I anticipate that people could be as amazed as I am to hear these emotional rhythms.
"I have a prototype software where I am experimenting with that right now" Gee adds as she demonstrates how emotions are being portrayed and how they will sound when inserted into the robot. "I am translating this into glockenspiel instruments because to me I thought that something percussive, something glockenspiel, it reflects the nature of the data because the data is, you know, either on or it is off. That is like a xylophone being hit. Ding.”
The end product is a mix of rhythms.
“Because you will definitely hear natural rhythms of the body being reflected in rhythms of xylophones being hit," Gee notes, "but every once in a while there will be an emotional fluctuation, so it will sound, kind of, like a gamelan orchestra of xylophone, bell sounds.”
The project is supported by the University of Western Sydney and the Concordia University in Montreal, Canada.