Tonal Harmony is widely considered to be the most technical and complex part of music theory, and harmonic skills can be hard to acquire. Experience of the precise and flexible manipulation of harmony in real time generally requires hard-won instrumental skill. Even with instrumental skills, it can be hard to gain clear insight into harmonic abstractions. The above state of affairs gives rise to substantial barriers not only for beginners but also for many musicians. To address these problems, Harmony Space [Holland et al, 2009] is an interactive digital music system designed to give insight into a wide range of musical tasks in tonal harmony ranging from performance and composition to analysis. Harmony Space employs a principled set of spatial mappings to offer fluid, precise, intuitive control of harmony. These mappings give rise to sensory-motor, music-theoretic and information-theoretic affordances that are not readily obtainable in any other way. The result is that a wide range of harmonic abstractions are rendered amenable to concrete, visible manipulation by simple spatial means. In the language of conceptual metaphor theory, most relationships in tonal harmony become accessible, to rapid, universal, low-level, robust human inference mechanisms using image schema, such as containment, contact, centre-periphery, and source-path-goal, in place of slow, abstract symbolic reasoning. While keeping the above principles invariant, different versions of Harmony Space have been designed to exploit different detailed interaction styles for different purposes. We note some key variants, such as the desktop version [Holland, 1994], the camera tracked version [Holland et al., 2009], and the most recent whole body version, Song Walker [Holland et al., 2011]. Preliminary results from a recent study of the Song Walker system are outlined, in which both beginners and expert musicians undertook a range of solo and collaborative musical tasks involving the performance, composition and analysis of music. Finally, we offer a discussion of the limitations of the current system, and outline directions for future work.

Recent experiments showed that the use of haptic vibrotactile devices can support the learning of multi-limb rhythms [Holland et al., 2010]. These experiments centred on a tool called the Haptic Drum Kit, which uses vibrotactiles attached to wrists and ankles, together with a computer system that controls them, and a midi drum kit. The system uses haptic signals in real time, relying on human entrainment mechanisms [Clayton, Sager and Will, 2004] rather than stimulus response, to support the user in playing multi-limbed rhythms. In the present paper, we give a preliminary report on a new experiment, that aims to examine whether passive learning of multi-limb rhythms can occur through the silent playback of rhythmic stimuli via haptics when the subject is focusing on other tasks. The prototype system used for this new experiment is referred to as the Haptic iPod.

Whole Body Interaction appears to be a good fit of interaction style for some categories of application domain, such as the motion capture of gestures for computer games and virtual physical sports. However, the suitability of whole body interaction for more abstract application domains is less apparent, and the creation of appropriate whole body interaction designs for complex abstract areas such as mathematics, programming and musical harmony remains challenging. We argue, illustrated by a detailed case study, that conceptual metaphor theory and sensory motor contingency theory offer analytic and synthetic tools whereby whole body interaction can in principle be applied usefully to arbitrary abstract application domains. We present the case study of a whole body interaction system for a highly abstract application area, tonal harmony in music. We demonstrate ways in which whole body interaction offers strong affordances for action and insight in this domain when appropriate conceptual metaphors are harnessed in the design. We outline how this approach can be applied to abstract domains in general, and discuss its limitations.