Category Archives: sonic interaction design

Goals

We developed a family of interactive objects integrating dynamic sound feedback, called Stonic. They are designed to provide users with different affordances, i.e. action possibilities, which are linked to different sound feedbacks. These objects are used for experimental studies to better understand and model augmented auditory feedback in object manipulations.

 Mechanical interactions producing sounds

slides_workshop_legos_2015.017

Solid and liquid interactions producing sounds, from Houix et al. 2012 and Guyot P et al. 2O14.

We used the taxonomy of basic physical interactions, (discrete, like  impacts, & multiple impacts, or continuous like  tearing, shaking, rubbing, …)  to define a lexicon of basic gestural interactions. We also looked at the different taxonomies on how people grasp objects to study the relationships between the shape and the type of grasping (for example precision vs power grasping).

 

 

First prototype

Stonic, first prototype.

First prototype and the different gestural interactions (Left). Different embedded sensors (Right). 3D modeled by Arnaud Ouin (Beaux Arts du Mans).

The different interactions producing sounds are the basis of our requirements for the object creation, because in our case we are interested in manipulations of objects that are related to the production of sounds. We developed a first prototype. The object is instrumented with different sensors and the data is processed in order to recognized the different actions. The different actions are sonified with recorded sounds played interactively with MUBU framework.

 

 

An object family

Julien Groboz, independent designer, created different 26 shapes meeting our specifications.

new_figures_low

Formal propositions by Julien Groboz. Scale 1, made with acrylic resin and / or Polysterene and their 3D print.

Object(s) should allow: manual or bimanual gestural interactions; with or without contact with a surface; precision or power grasping and different elementary interactions: to hit, to rub, to turn, to swing, to tilt, to place, to shake, to press, to crush. All the information about the different sensors were provided.

 

 

We selected 6 forms to test their affordances and Julien Groboz modeled the objects for 3D printing.

Video demonstration:

Sonic affordances

Fabien Bourlier created two sets of  different sonic affordances. A first one corresponds to concrete sounds related to physical interactions and a second one more abstract keeping morphological aspects of the physical interactions producing sounds. The different created sounds will be dynamically and interactively played using MUBU.

DSCN3671

Recording session. (c) Fabien Bourlier.

Different physical materials were used in order to embodied the object with two contrasted materials: wood that evokes the warmth and the intimacy;  metal / glass with an evocation more colder et less resonant. Several interactions producing rich timbre were tested and recorded: breaking, twisting, trampling on, throwing, hitting, rolling, ….

A second set two family of abstract sounds have been created. The first one is based on an organic metaphor, but without identification, that produces rich timbres by contrast with “aerial” sounds with more noisy and uncluttered timbres.

New videos with sound design will be online soon.

The Virtual Bath Tub

The virtual bath tub is an interactive virtual surface in the air the user can interact with, as if there were a bath tub filled will water in front of him.

Inspired from foley artists issues to manipulate water, this scenario allows playing with splash and underwater sound sequences related to the energy of the movement.

Conception: Eric O. Boyer.

Motion capture with Leap Motion Device.
Sound synthesis made with MuBu objects (www.ismm.ircam.fr).
Sound materials by Roland Cahen and Diemo Schwarz (Topophonie Project).

Workshop HC2

Summer School 2013 Human Computer Confluence, http://hcsquared.eu

Workshop From everyday objects to sonic interaction design
July 17-19 2013, Ircam, PDS and IMTR research teams (collaboration with Goldsmiths-London)

Lauren Hayes – PhD student in creative music practice
Emmanouil Giannisakis – Master student in digital media engineering
Jaime Arias Almeida – PhD student in informatics
Alberto Betella – PhD student in communication, information and audiovisual media
David Hofmann Phd student in theoretical neuroscience

HC2-students

HC2-scissorsgame HC2-flashmob

Workshop Participatory Design

We held a two-day workshop at IRCAM, May 30th and June 1st 2012, to brainstorm on possible sonification of every-day objects. Every participants brought two objects, that were analyzed and discussed. On the second workshop day, experiments using sensors and real-time synthesis were carried-on on a selection of objects.

More information can be found in the publication:

Houix, O., Gutierrez, F., Susini, P., & Misdariis, N. (2013). Participatory Workshops: Everyday Objects and Sound Metaphors. In Proc. of the 10th International Symposium on Computer Music Multidisciplinary Research (p. 41‑53). Marseille, France: PUBLICATIONS OF L.M.A.