This is a long overdue mention of Very Nervous System, a beautiful interactive framework developed over twenty years ago by David Rockeby.
Although VNS is based on video tracking, the approach is very similar to the one behind the Xth Sense biophysical music.
I find his words inspiring (from his website): “Because the computer is purely logical, the language of interaction should strive to be intuitive. Because the computer removes you from your body, the body should be strongly engaged. Because the computer’s activity takes place on the tiny playing fields of integrated circuits, the encounter with the computer should take place in human-scaled physical space. Because the computer is objective and disinterested, the experience should be intimate.”
And he continues referring to the VNS interface, “is unusual because it is invisible and very diffuse, occupying a large volume of space, whereas most interfaces are focussed and definite. Though diffuse, the interface is vital and strongly textured through time and space”