The Legos project was exploratory and lead to several types of outcomes
New knowledge on the effect of auditory feedback in gesture-sound interactive systems. See the page Publications for the complete list. We summarize below important cases we studied:
- Influence of movement sonification on perception (A. Tajadura-Jimenez, IEEE Multimedia 2015) ou dans le cas de toucher sur une surface (A. Tajadura-Jimenez, Proc NorCHI’14)
- Sonification of ocular movement (Collaboration with Jean Lorenceau, ENS Paris, see .E.O. Boyer PhD thesis)
- Sensory substitution: audio virtual surfaces (E. O.Boyer et al, Proc. SIVE 2015) and audio-only gaming (Hanneton et al., International Journal of Adaptive Control and Signal Processing, 2015)
- “Sound tasks” (E. O. Boyer et al, LNCS 8905, 2014 and J. Francoise et al Sigggraph’14)
- Visuo-manual tracking task with continuous sound feedback (E.O. Boyer et al. submitted)
- Pointing to an auditory source (E. O. Boyer et al, Front. Computat. Neurosci. 2013)
New methodology and technology for designing the sound feedback.
- Participatory workshops were found effective (see O.Houix et al, LNCS, 2014), the method was further used in the HC2 Summer School and in others workshops/courses. The research also leads to the development of the Stonic object that will be used for further studies in interactive sound design.
- Designing and programing movement to sound relationship using “mapping by demonstration“. This method was demonstrated at Siggraph’14 in the Emerging Technologies and published in more than 6 conferences proceedings (J.Francoise et al.).
Towards potential applications
- Rehabilitation aided by auditory feedback. This leads to a new project in the labex SMART
- Sensory subsition: see videos
- Interactive Sound Design: see videos
- Learning sport movement
and finally it allowed us to establish new collaborations and contribute to the field of movement sonification. See the collaborations pages and the Legos Days.