Res, a matter.

Chikashi Miyama | Black Vox

[The post has been UPDATED with a live recording of Music for Flesh II]

The Pure Data Convention is one of the best community led event I ever took part in. I’m not sure whether is the laid back attitude shared by the participants, or the excitement of showing and learning all the diverse things Pd can be used for, however the Pure Data meetings are something unique. The cloud of knowledge, techniques, good conversations and ideas that floats around the heads of the delegates is pretty special and cozy.

This year the fourth Pd Convention was organized by our friends and Pd developers Max Neupert and Bjorn Lindig, along with a team of colleagues from the Bauhaus Universitat in Weimar. I was honoured and thrilled to present the Xth Sense interactive system to the community, and eventually I had the chance to teach a workshop, give a lecture on the latest findings and perform the solo piece Music for Flesh II.
The workshop was quite fast, but we had the chance to get our hands dirty with some custom electronics and Pure Data programming for interactive, biophysical music; some pictures below.

The Xth Sense biophysical sensors

Designing sound gestures for the Xth Sense system

A ridiculous amount of speakers at the awesome Seam Studio.

A nerd and its child.

Full immersion, last hours of the course.

Personally it was a great experience as I collected some inspiring feedback and constructive critics about the whole XS project. Namely, I had wonderful conversations with friends Ricardo Palmieri and Richard Graham, that are both working on some highly recommended Pd-based projects. With Ricardo we have been considering a different standard for the production process of the XS sensors, while with Richard we discussed about the use of multiple subwoofers, the more effective impact of whole-body movements to both the sonic outcome and the audience perception, and the mediation between primal muscle sound and its computational mediation.
The latter topic is as tricky as stimulating; recently I’ve been thinking about this quite a lot, and if on one hand there is not an ultimate solution as we deal with the subjectivity of a musical aesthetic, on the other, this aspect of the research is crucial, for a deeper musical analysis could sharpen the timbrical characteristics of the XS as a musical instrument.

It is often very hard for me to be completely satisfied of a performance, but I can say the concert at the Seam Studio in Weimar was perhaps one of the best so far. Probably due to the swift anxiety of performing before the ears of my peers and colleagues, my concentration was fairly strong, and while playing I think I almost entered a trance. For the first time since March I spontaneously moved the whole body, rather than individual limbs; such a somatic behaviour triggered sound forms that I never heard before and playing the piece was far more enjoyable than usual. As simple as it gets, in contrast with single limbs contractions, whole body movements involve the excitation of a greater part of the body; thus many more oscillations are generated and the whole network of muscular tissues is excited. Sonically, this is translated in a faster dynamic and richer texture.

Here you can listen to a live recording of the piece.

Marco Donnarumma | Music for Flesh II

The concerts program was excellent; the fourth PdCon gathered some of my favourite Pd musicians, such as Koray Tahiroğlu, Richard Graham, Chikashi Miyama (pictured at the top of the page), CHDH, Oscar Martin and Alexandre Porres. We also had an amazing exhibition, more workshops and a round table with a virtual Miller Puckette, who participated in streaming from the US. Check the program on-line to get a glimpse of the tight schedule.

Giuliano Obici | Lanhouse Concert

Oscar Martin Noishx

Of course, I’m not forgetting about the papers; being that there were too many lectures, and the sessions were split in two rooms running at the same time I could not attend all of the presentations. However, it is hard to say which ones were the most interesting, because the diversity of the contents presented at the Convention is always so great to be almost overwhelming. If you wish to stay up to date with the latest developments of our community, you are only one click away.

A big thanks to the organization and to all the delegates that made this Convention another unforgettable one. A special thanks to Cyrille Henry, who provided us with some insights on the pmpd, a library for physical modelling and the use of shaders in GEM. Most likely I’ll be working with these tools soon for a new audiovisual performance.

All pictures are courtesy of the 4th PdCon.

The University of Huddersfield, UK

I have been invited to the ICMC 2011 to give a lecture about the state of my current research in biophysical music. A related paper “Xth Sense: a study of muscle sounds for an experimental paradigm of musical performance” will be published in the ICMC proceedings.
“Since 1974 the International Computer Music Conference has been the major international forum for the presentation of the full range of outcomes from technical and musical research, both musical and theoretical, related to the use of computers in music… The theme of the 2011 conference is innovation : interaction : imagination emphasising the importance for our field of innovative technical developments, imaginative responses to new situations and, above all, the creative interaction of the technical and the musical.”
My participation to the ICMC in Huddersfield would have not been possible without a PRE travel grant awarded by The University of Edinburgh.

“A high-tech music weekender, BEAM is a playground of homemade instruments and sonic robots, focused on electronic music being performed or created LIVE.”

Performing Music for Flesh II | Picture by Christos Michalakos

The Beam Festival was an inspiring event; packed with an incredible line-up including Stelarc, Atau Tanaka and Adam Parkinson, Alex Nowitz, Dj Sniff, Ryan Jordan and many others, it offered 3 days of absolute sonic pleasure. Me and colleague Christos Michalakos were awarded a bursary for “outstanding work”, respectively for Music for Flesh II and Frrriction, thus we had the pleasure to perform at the Artaud building during the evening concert sessions.
A complete report of the event is available at my personal blog.

STEIM Crackle Box

In the end of May I’ve been invited to direct the second edition of Host Guest Ghost, an artistic residency in partnership with The Danish National School of Theatre and Contemporary Dance. The project was supported by The Danish Arts Council and Augustinus Fonden.

Still from rehearsal at Dansehallerne, Copenhagen, DK, 2011

The project aimed to foster experimental approaches to dance, theatre and interactive technologies while forming emerging, talented choreographers and creatives. Together with a team of 6 dancers, 3 musicians and a professional chef, we developed a new intermedia piece titled “Raw/Roar”. The piece unveiled a peculiar, unique world of enhanced bodies, biophysical sound, augmented music and reactive videos. Among the different media and technologies explored during the residency, the main focus was set on the Xth Sense.

It was a challenging experience; initially, the dancers found some difficulties to combine their practice with the interactive framework provided by the Xth Sense, but after enough practice and open discussions about the motivation and aesthetic of the system, we defined a common terrain. However, it has been inspiring to realize how differently a musician and a traditionally trained dancer can approach the model of biophysical music. It is perhaps purely logical; whereas the former explores movement as a means towards a transparent externalization of a sound form, the latter is submerged in such a physical self-awareness that the idea of an incarnation through her body of a projected sound can be constraining.
I believe such idea is worth a further analysis on the nature and extendability of the Xth Sense; I would love to collaborate again with dancers to gain more insights about movement, music and performance.
As soon as the documentation will be ready I will post some pictures and videos.

source: pcm.svg, available from http://commons.wikimedia.org/wiki/File:Pcm

The Linux Audio Conference (LAC2011) was held at NUI, National University of Ireland in Maynooth in the beginning of May.
My paper “Xth Sense: researching muscle sounds for an experimental paradigm of musical performance” and a related performance “Music for Flesh II” were both selected for the conference.
So far, the LAC has been one of the best conferences I took part in, both for the diverse and top-notch quality content offered, and for the easiness of social networking. A great place to be, indeed.

Thanks go to Victor Lazzarini, John Lato, Frank Neumann, Robin Gareus, Rory Walsh and all the NUI and LinuxAudio.org team. All presentations were streamed live and recorded. Definitely a great and valuable effort.
If you missed the event, make sure to check the program on-line, you will find an impressive source of ideas, knowledge and technical insights.
Here’s a complete video recording of my talk; an in-depth presentation of earlier methods and findings on the Xth Sense and muscle sounds.
The slides used in this presentation are also available for download at the LAC website.
All material is CC licensed, see the LAC website for detailed information.

Drawing from the stimulating conversations we all had in Maynooth I could extract some more ideas around muscle sounds. Above all some friends made me realized that I still didn’t analyzed in depth muscle sounds as for FFT and detailed frequency responses; although FFT has not proved effective, some statistical documentation about the sonic matter I’m working with could possibly disclose new insights.
Besides, a colleague working in the field of Biomedical Engineering reminded me that human body is composed of two different muscles, namely short and long muscle. Whereas long muscles are used for slow and light contractions, short muscles handle faster and stronger contractions. This observation might be useful while planning the location of four sensors on the performer’s body. I still couldn’t try this myself, but this is one of the next ideas I would like to actualize this year.

This is a long overdue mention of Very Nervous System, a beautiful interactive framework developed over twenty years ago by David Rockeby.
Although VNS is based on video tracking, the approach is very similar to the one behind the Xth Sense biophysical music.
I find his words inspiring (from his website): “Because the computer is purely logical, the language of interaction should strive to be intuitive. Because the computer removes you from your body, the body should be strongly engaged. Because the computer’s activity takes place on the tiny playing fields of integrated circuits, the encounter with the computer should take place in human-scaled physical space. Because the computer is objective and disinterested, the experience should be intimate.”
And he continues referring to the VNS interface, “is unusual because it is invisible and very diffuse, occupying a large volume of space, whereas most interfaces are focussed and definite. Though diffuse, the interface is vital and strongly textured through time and space”

The Xth Sense workshop just ended. It was an exciting experience. We have been hosted by the almighty NK Berlin, an established independent venue for experimental music, DIY electronics and the likes.
During three days we have been building some Xth Sense biosensors from scratch, installing and trying out the Xth software and shortly training on biophysical gestural control of music. Definitely a lot to do for three days only!

Building the Xth Sense biosensor from scratch

Participants multitasking

DIYing biophysical sensors

It was the first time the Xth Sense was unveiled in such detail; on one hand, the sensor hardware proved to be fairly easy to build, also the participants who had not much prior experience with electronics and soldering successfully built their own sensors; on the other, I realized the software needs to be much more portable; we spent too much time dealing with the paths of the required libraries, and I eventually realized that we were struggling with a known bug!
Too bad. Lesson 1: check bug trackers regularly.

Ghost in the GUI

Patching the Xth Sense system

I was pretty satisfied of our workflow, although I realized three days are not enough to achieve the ideal format of such workshop. Eventually everyone built his own sensor and the Xth software was up and running on almost all the machines. The only real problem was to get the software working on two Mac G4.
It seemed that they couldn’t run the software because of too high CPU usage, thus, after some unlucky trials and on-the-run optimization of the software,  we had to work on some other laptops.
Lesson 2:  although sometimes is cool to be able to use a very old machine, make sure beforehand nobody is using one of them during your workshop.

Students playing with the biophysical sensors

More playing, get those sounds out!

It was extremely satisfying to see my students playing around with the system on the last day. It was good to confirm that the Xth Sense have an actual potential to extend the portability of biosensing technologies and gestural control to a wider audience. I wish we had more time for the hands-on training. Observing how each individual differently relate to sound had always fascinated me, and a course based on such bodily paradigm provides a transparent insight on the different expectations and skills of each one.
Lesson 3: DIY is good, but remember to leave enough time for the fun.

Want to thank the guys at NK, they have been extremely helpful and lovely companions. I’ll have to come back soon to Berlin for another workshop session.

I started a small tour with the Xth Sense system.
The first event I was invited to was the ImagineCreate Festival, in Derry, Northern Ireland. ImagineCreate is “a digital arts festival which brings together a world-class line-up of talents and minds from the worlds of art, design, new media and software development.”
I gave a talk about the Xth Sense and biophysical music within a presentation titled “Performance and Pure Data” in collaboration with Richard Graham.

Here’s a short video excerpt of the showcase captured by James Alliban.

It was quite good to take part in the event as I’ve received some very good feedback and suggestions. Among the other invited speakers there were (the above mentioned) Richard Graham, James Alliban, Alex Beim (Tangible Interaction), John Crooks, Gregory Taylor (Cycling ’74) and Brendan McCloskey. A full list of speakers is available on-line.

Many interesting issues were raised after my talk so I added some items to my todo list.
First, I plan to implement a tracking of muscle sounds contour. This way I could deploy the contour as dynamic curve for data mapping, obtaining different results than with a preset curve, such as a logarithmic or exponential one.
Secondly, I could compare the incoming data of each arm passing them to a discrete function; the resulting value would represent the ratio between the energy applied to the arms. It might also be interesting to track rest intervals over long period of time.

Rehearsing Music for Flesh II at Inspace, UK.

I just ended a new work residency at Inspace, home to a joint research partnership between the School of Informatics and New Media Scotland.
During the residency, me and Brendan F Doyle (Production Sound Engineering) worked on a participatory Musicircus which included a complex, reactive sonic system for 8 Xth Sense biosensors and 8 audio channels.
The piece was premiered two days ago on 15th March during a public event which gathered quite a good amount of people from different backgrounds, music, sound design, informatics, hackers and general public.

It was awesome to see how the audience positively reacted to this unconventional paradigm of musical performance, namely generating and controlling electroacoustic sounds and music by muscle’s contraction. After some initial – and expected – disorientation, everybody started realizing how to use the sensors and how to interact with the system; only then the actual Musicircus started.

The event was pretty successful, everything worked out fairly smoothly –  except the few things which always “have to” get broken just before the opening –  and we received several excited and stimulating feedback. Overall a great and satisfying experience.

During the same night, I also performed my new solo sonic piece Music for Flesh II, for augmented muscle sounds and Xth Sense technology. The acoustic capabilities of Inspace were challenging: glass walls all around, an empty floor which acts as a resonant shield, and interesting sonic reflections happening all around the space. I need to thank again my colleague Brendan for his great work with the sound engineering of the venue and his live mixing during my performance.

Picture courtesy by Chris Scott.

My paper “Xth Sense: researching muscle sounds for an experimental paradigm of musical performance” was accepted to the Pure Data mini conference, along with the related sound piece Music for Flesh II.
The Pd mini-con took place in Dublin, at the Trinity College. It was my first academic presentation of the Xth Sense outside of my own department; definitely an exciting chance to get new feedbacks and establish a new group of peers.
The experience was great, and the comments of audience and staff were positive. I also had some small technical problem during the performance… some good food for thought!

Performing Music for Flesh II | Pure Data mini conference | 2011

A full report of the event is available on my on-line magazine.