Res, a matter.

My paper “Xth Sense: researching muscle sounds for an experimental paradigm of musical performance” was accepted to the Pure Data mini conference, along with the related sound piece Music for Flesh II.
The Pd mini-con took place in Dublin, at the Trinity College. It was my first academic presentation of the Xth Sense outside of my own department; definitely an exciting chance to get new feedbacks and establish a new group of peers.
The experience was great, and the comments of audience and staff were positive. I also had some small technical problem during the performance… some good food for thought!

Performing Music for Flesh II | Pure Data mini conference | 2011

A full report of the event is available on my on-line magazine.

While composing Music for Flesh II, the new piece for the Xth Sense, I found difficult to draw the structure of the performance as the overall live act is mostly an improvisation based on a given set of sound-gestures and a fixed timeline. I define as a sound-gesture a given movement of the performer which can produce one and only sound form; this paradigm implies that the reactive computing system interprets each effective gesture according to a given, but still variable, array of sound behaviours, or sound scenes. The several sound-gestures which compose the piece are still experimental and most of them still have to be defined more precisely.

Being that the design of the performance is only based on this idea of sound-gesture, the use of a graphical score seemed to me the best way to go.
Graphical notation is fairly widespread today; since its first appearance in the 1950’s several experimental and avant-garde composers deployed graphical scores to better understand and describe a sound form, among them John Cage, Christian Wolff, Morton Feldman, Karlheinz Stockhausen, Iannis Xenakis and Anthony Braxton.

Fontana Mix | John Cage | 10 sheets of paper and 12 transparencies | 1958

Christian Wolff | Edges | instruction for improvisers | 1968

Solitude | HC Steiner | data structures in Pure Data | 2007

I’ve been reviewing different approaches to graphical scores, and although there exist digital systems which enable a composer to virtually draw experimental scores, I was still too fascinated by the act of drawing with a pencil on a piece of paper.
Thus, I started with simple exercises, such as drawing a graphical notation of few pieces which had inspired my composition. After some interesting results with Wind Chimes by Denis Smalley, I could draw a first draft of my own piece.
The symbols can be interpreted as an idiosyncratic representation of both the sounds and the kinetic movements to be executed by the performer.

Marco Donnarumma | Music for Flesh II | sound-gesture descriptors | 2011

This was my first attempt to draw a graphical score for a piece based on the Xth Sense; I will keep experimenting with this aspect of composition as the paradigm represented by the “sound-gesture”, which is integral to the Xth Sense technology, offers interesting prospects for experimental notation.

Richness of colour and sophistication of form.

Subgrowth by Kaens | 2007

How to achieve a seamless interaction with a digital framework for DSP which can guarantee richness and sophistication in real time?
Moreover, the interaction I point at is a purely physical one.
Gesture to sound, this is the basic paradigm which I need to explore.

This is a fairly complex issue, as it requires a multi-modal investigation which encompasses interrelated areas of study. The main key points can be summarized as follow:

• developing a musical aesthetic by which appropriate DSP techniques can be conceived;
• coding relevant audio processing algorithms for muscle sounds;
• analysing and understanding the more appropriate gestures to be performed;
• train my muscles in order to learn how to efficiently execute specific gestures;
• developing a mapping system of kinetic energy to control data.

Drawing from this observations a roadmap for a further implementation of the Xth Sense framework, I found myself struggling with a practical issue: the GUI I was developing did not fit my needs, thus slowing down the whole composition process.
In order to comfortably concentrate on composition and design of the performance I needed an immediate and intuitive routing system, which would allow me to map any of the MMG control values available to any of the DSP parameter controls in use.
Such router would satisfy a twofold mapping: one-to-one and one-to-many.

In Pure Data control parameters can be distributed in several ways, however the main methods are by-hand chords connection and chord-less communication exploiting [send] and [receive], or OSC objects.
Obviously, I first excluded the implementation of a by-hand connection method, as I need to freely perform several muscle contractions in order to test the features extracted by the system in real time. A chord-less method would have better fit my needs, but still it did not satisfy me completely, as I wanted to route the MMG control values around my DSP framework fast and intuitively.
The sssad lib by Frank Barknecht coupled with the send/receiver [iem_s] and [iem_r] from IEMlib eventually stand out as a good solution.
sssad is a very handy and simply implemented abstraction which handles presets saving within Pd by means of a clear message system. [iem_s] and [iem_r] are dynamically settable sender/receiver included in the powerful iemlib by Thomas Musil at IEM, Graz Austria. I coupled them into 2 new macro send/receive abstractions which eventually allowed me to route any MMG control value to any DSP control parameter with 4 clicks (in Run mode).

Click to enlarge.

MMG data mapping abstractions | Xth Sense | 2011

Four clicks routing:
1st – click a bang to select which source control value has to be routed.
2nd – activate one of the 8 inputs within the router; it will automagically set that port to receive values from the previously selected source
3rd – touch the slider controlling the DSP parameter control you want to address
4th – activate one of the 8 outputs within the router; it will automagically set that port to send values to the selected parameter.

Main GUI | Xth Sense | 2011

This system works efficiently both for one-to-one and one-to-many data mapping. The implementation of such data mapping system dramatically improved the composition workflow; I can now easily execute gestures and immediately prototype a specific data mapping.
Besides, as you can see from the image above, I included a curve module, which enable the user to change on the fly the curve to be applied to the source value, before it get sent to the DSP framework.
The mapping abstractions are happily embedded in the major sssad preset system, thus all I/O parameters can be safely stored and recalled with a bang.

Heading picture CC licensed by Kaens.

Soundhack GOP and SSSAD wrap object | original Pd object by Tom Erbe

Click to enlarge.

During my exploration of DSP processes for muscle sounds I eventually realized I could make good use of the interesting Soundhack objects, which have been recently ported to Pd by Tom Erbe, William Brent and other developers (see Soundhack Externals).
Among the several modules now available in Pd, I focused my attention on [+bubbler] and [+pitchdelay].

The former is an efficient multi-purpose granulator, while the latter is a powerful pitch-shifting based delay, which allows a functional delay saturation along with octaves control; both work nicely in real time. More interestingly, performance testing showed that the max mean value (here referred to as a continuous event) produced by muscle contractions can be synchronously mapped to the feedback and loop depth of the [+pitchshift~] algorithm in order to obtain an immediately perceivable control over the sound processing.
Sound-gesture obtained through this mapping system are quite effective because the cause and effect interaction between performer’s gesture and sonic outcome is clear and transparent.

Here’s an audio sample. The perceived loudness of the sound and the amount of feedback is directly proportional to the amount of kinetic energy generated by the performer’s contractions.

Moreover, I included in my wrap-up object also sssad capabilities to be able to save presets within the DSP framework I’m developing.

I need to explore further this algorithm and understand further improvements for a real time performance.

Åke Parmerud | Jeu D'ombres | empreintes DIGITALes | 2003

While seeking inspiration and efficient methods to compose a more colorful and sophisticated piece for augmented muscle sounds, I started listening intensively to the work of Swedish composer and media artist Åke Parmerud.
Namely, Jeu D’ombres by Parmerud is an amazing piece of work, I can’t get enough of it. The piece Strings and Shadow, awarded at Prix Ars Electronica 2004, is a beautiful, precise and organic composition which seductively envelope the listener, submerging the hearing with exciting, glassy sonorities, and obscure, introspective moments of void and silence. The structure of the composition flows through one movement to the other with absolute ease, drawing a wide and unconstrained aural space, yet capable of describing a truly human intimacy. A similar, captivating experience can be recalled along the whole album.
Thanks to Martin Parker for pointing me to this music, listening to this sounds is truly inspiring for my own research and composition methods.

• Composer: Åke Parmerud
• Performers: Sofia Asunción Claro,Stockholm Saxophone Quartet
• Cover image: Åke Parmerud

Setting up for the performance | Suoni Sonori, Milan, Italy, January 2011 | Courtesy Ugo Dalla Porta

Here are some critical keypoints which arised from a joint analysis by my supervisor – Martin Parker – and myself regarding the first public performance of Music for Flesh I:

• to design intimacy;
• to create physical spaces with broader body movements;
• to change gesture patterns according to sound patterns;
• to reach a broader frequency spectrum, thus more interestingly exciting viewers’ listening;
• to not let die a single sound scene, but bring it somewhere, build more complex structures around it;
• to be able to switch in real time between different MMG signal functionalities (sonic source or control signal), while keeping safe the possibility to deploy both function of the signal at the same time.

Overall, what I need to focus the performance design on is a richness of color and a sofistication of form.
I look forward to using two sensors on both arms, as this could significantly enhance the richness of the sonic material, while providing the possibility of polyphonic structures.
On another hand, more exciting sonic forms may also be achieved by means of autonomous processes capable of reacting to the performer’s kinetic behaviour, adding machine state-dependent pattern changes, as also suggested by my friend Jaime Oliver during our last email exchange.

Here’s a first demo presentation of Music for Flesh I, a solo sonic piece for the Xth Sense biosensing wearable device.
This piece is the first milestone of the research, many aspects need to be improved, but I’m quite satisfied of this result and above all composing this piece provided many insights about the further development of the research.

Further information here.

Music for Flesh I – solo piece for Xth Sense biosensing wearable device from Marco Donnarumma aka TheSAD on Vimeo.

Recorded in January 2011 at Suonisonori, Milan, Italy.
Camera and editing Gianluca Messina; audio recording Giuseppe Vaciago.

Here’s the Xth Sense biosensing wearable prototype v1.

MMG wearable sensor hardware v1

Keeping up with the hardware design for a couple of weeks, I eventually felt it was the right time to make it wearable. During the Christmas holidays I came back to Italy where Andrea Donnarumma and Marianna Cozzolino gave me an incredibly precious support both during the design process and the technical development.
Before undertaking the development of the Xth Sense sensor hardware, few crucial criteria were defined:

• to develop a wearable, unobtrusive device, allowing a performer to freely move on stage;
• to implement an extremely sensitive hardware device which could efficiently capture in real time and with very low latency diverse muscle sounds;
• to make use of the most inexpensive hardware solutions, assuring a low implementation cost;
• to implement the most accessible and straightforward production methodology in order to foster the future re-distribution and openness of the hardware.

So here’s a loose summary of the making of the Xth Sense biosensing hardware wearable prototype v1.
First, we handled the battery issue. We were not sure whether the microphone would have reacted properly and with the same sensitivity using a standalone battery, so we hacked an old portable minicassetter recorder and extracted the battery case. Next we built the circuit on a copper board, included a trimmer to regulate the voltage feeding the microphone and embedded everything in a plastic DV cassette box along with a 1/2 mono chassis jack socket. Then, we used some common wiring cable to connect the hacked battery case to the circuit and the bracelet to the circuit box.

Hacking a radio battery holder

Building the first open circuit

First testing implementation of the open circuit

Preparing the testing circuit case

First wearable MMG sensor hardware prototype

The resulting device was obviously quite unsophisticated, but we simply wanted to make sure the microphone would have maintained the same capabilities while on battery charge. The experiment was successfull and we started planning a more usable and good-looking design.

First of all we looked for appropriate components: black box (3.15 x 1.57 x 0.67), smaller condenser and resistors, 3V coin lithium batteries and their holders (which were not so easy to find as we expected) and a more flexible wiring cable.

Bits and parts

At this point, circuit needed to be used on a smaller copper board, so we slightly changed its design in order to decrease size and make it nicely fit the box.

Embedding the second prototype sensor hardware in a wearable case

Second sensor hardware prototype in a new wearable case

Device worked like a charm, but in spite of the positive results the microphone shield required further trials. The importance of the shield was manifold; an optimal shield had to fit specific requirements: to bypass the 60Hz electrical interference which can be heard when alternating electric current distribute itself within the skin after a direct contact with the microphone metal case; to narrow the sensitive area of the microphone, filtering out external noises; to keep the microphone static, avoiding external air pressure to affect the signal; to provide a suitable air chamber for the microphone, in order to amplify sonic vibrations of the muscles, allowing to capture also deeper muscle contractions.
First, microphone was insulated by mean of a polyurethane shield, but due to the strong malleability of this material, its initial shape tended to undergo substantial alterations. Eventually, sensor was insulated in a common silicon case that positively satisfied the requirements and further enhanced the Signal-to-noise ratio (SNR).

Stamps for the silicon shield

This detail notably increased the sensitivity of the biosensing device, giving a wider range of interaction.

Such result satisfies the present hardware requirements: the device is very small, it can be put in a pocket or hooked to a belt, and it is enough sturdy to be used with no particular care. I’m going to test the performance of the device during a demo presentation at Suonisonori, Milan Italy in few days.

Before the holidays break I wanted to set a milestone of the inquiry, so I agreed together with my supervisor Martin Parker to host a short presentation reserved to the research staff of our departments (Sound Design and Digital Composition).

MMG biosensing device presentation setup | The University of Edinburgh | 2010

I’ve been setting up the Sound Lab at Alison House since the morning and everything worked fine.
Even though some of the researchers could not make it, I was happy to see Martin, Owen Green and Sean Williams.

MMG biosensing device presentation | The University of Edinburgh | 2010

Although the piece I performed represented more a first presentation than a proper concert for me, it has definitely caught the attention of the listeners and earned some good feedbacks; however what I was most expecting were constructive critics which could allow me to assume a different viewpoint on the present stage of the project.
In fact I did receive several advices which can be roughly summarized as follow:

• harmony, improve the overall harmony of the piece
• silence, including silence in a musical piece demonstrates braveness and coherence
• sonic gesture, theatrical gesture is very important in a musical performance of this kind
• add to the software envelope generators to enhance automation
• try to use a MIDI device
• study a better dynamics compression to be applied on the MMG source signal
• delays and reverbs, if improperly used such effects might destroy the sonic space, instead of creating it
• the distortion effects I first used can be ambiguous

I fully agreed with these critics and I actually realized I could have prepared the presentation much better. However it has been important to listen to my fellows’ feedbacks and that night I came back home and worked until late to improve the piece for the forthcoming concert.
Night time was not enough to work on all the issues raised after the presentation, but I was able to better experiment with silence and subtle processing effects, envelope generators (which eventually have not been used) and a MIDI controller. Results seemed very good.
The day after I worked a couple of hours more, then I went to Alison House to set up the equipment for the concert together with my fellows Matthew and Marcin who performed too on the same night. This time I prepared everything professionally, anxious as I was to present the reviewed piece.

Concert for Biosensing Techs | Edinburgh University | 2010

Concert for Biosensing Techs | Edinburgh University | 2010

The setup consisted of the MMG sensor prototype, a Focusrite Saffire Pro40, a Behringer BCF2000 MIDI controller, a DELL machine running a customized Ubuntu Lucid Lynx with Real Time Kernel, and the Pure Data-based software framework I’m developing.

Concert for Biosensing Techs | Edinburgh University | 2010

Audience feedbacks were very good, and I seemed to understand that what most appealed the listeners was an authentic, neat and natural responsiveness of the system along with a suggestive coupling of sound and gestures. Concerts were supposed to be recorded, but sadly they have not.
Although some harmony issues remained, I was also fairly satisfied of the performance outcome. During the winter break I plan to better implement the prototype, possibly making it portable and to refine the software, coding a better control data mapping and fixing the omnipresent bugs.

What? | Veronique Debord | 2010

Why use only the body to control musical performance?
What about dancers in musical performance?
Who used it already? Why? How?
In which ways it would be different from a traditional musical performance?
To which extents resulting music will be different and authentic?
Will it be actually different?
How does the body relate to space?
How does space relate to sound?
How does sound relate to the body?
How does perception relate to the body?
Who did already give answers to this questions?
Have these questions been answered yet?
If so, have all aspects been properly covered?
Is there a gap? In which area?

Picture CC licensed by Veronique Debord.