Sonic Move - Creative and Expressive Sonification of Human Movement

Sonic Move – “Creative and Expressive Sonification of Human Movement” is a co-research project.

First case of the project with Dance Theater Minimi – Biodata Sonata.

nimi.fi/ohjelmisto/biodatasonaatti/The research team of the project consists of VTT Human Sensing solutions, Aalto University Sound and Physical Interaction Group and HUMEA Lab at UEF. Industry partners and co-funders are Genelec Oy and Taustamarkkinat BGMT Oy. The project is funded by Business Finland in 2023-2025.

The project investigates the use of human body movements in the synthesis of sound and tactile feedback and its use in dance and rehabilitation applications. Body movements can be measured very precisely with, for example, multi-camera systems. When movement data is combined with sound synthesis, sound environments can be created where, for example, the speed of hand movement can be combined with the pitch of the sound. In this way, the movement of the body can be sensed with both the senses of sight and hearing. The project aims to apply this multi-sensory solution as a means of expressing dance art and possibly as part of neurological rehabilitation.

The project involves collaboration with professional dancers and choreographers. The goal is to produce dance performances where sound and tactile feedback are an integral part of the experience. A key part of the project is sound design, where together with choreographers and dancers, the aim is to find different sound colours and nuances for the movements. These can be, for example, the sounds of nature: water, wind, and animal sound that responds dynamically to body movements. The tactile feedback is also aimed at adapting to both movements and sound. Tactile feedback can also be presented to the audience using a seat or pillow: these are provided for the project by an in-kind partner.

The sound and tactile feedback solutions produced by the project can be used in the work of dance choreographers in the future and in the product development of business partners. The solutions can increase the immersiveness of dance, attract more spectators to dance performances and make the products of business partners more competitive. In addition, the project brings together an interdisciplinary group of experts who benefit from each other.