VISUALSONIC.EU



The EGGS system

EGGS (Elementary Gestalts for Gesture Sonification) is a gesture sonification system, developed by Maurizio Goina and Pietro Polotti, School of Music and New Tecnologies, Conservatorio G. Tartini, Trieste, Italy. A crucial contribution as performer and user of the system is provided by Sarah Taylor.



Journal article:

---- Polotti, P., and Goina, M.: 'Gesture Sonification. An Interaction Design Approach to an Artistic Research Case'. In EAI Endorsed Transactions on Creative Technologies, ct 18: e1, 2021-03-19, ISSN 2409-9708.

Abstract:
INTRODUCTION: The subject of this paper is a phenomenological study of cognitively plausible relationships between gesture and sound mediated by technologies. In this discussion, the term ‘gesture’ indicates a movement of the body consciously performed and able to express or communicate something. The expression ‘cognitively plausible’ refers to an interactive sound response that is immediate, continuously varying and enactively coherent with the generating (sonified) gesture.
OBJECTIVES: The objectives of this paper are two. The first is an artistic exploration of gesture sonifications in various contexts. The second is rather methodological, i.e., to provide a possible general paradigm for Artistic Research (AR). METHODS: More in detail, the AR phenomenological approach is modeled according to an Interaction Design (IxD) research paradigm.
RESULTS: A number of case studies of gesture sonification are presented and discussed according to the above methodological framework.
CONCLUSION: We claim that the introduction of such methodological framework was successful in terms of providing robust guidelines for our research and for a clear and structured presentation of its results.

 


Conference presentations:

---- Elblaus, L., Goina, M., Robitaille, M. A., and Bresin, R.: 'Modes of sonic interaction in circus: Three proofs of concept'. In Proceedings of Sound and Music Computing Conference, 2014, Athens.

Abstract:
The art of circus is a vibrant and competitive culture that embraces new tools and technology. In this paper, a series of exploratory design processes resulting in proofs of concepts are presented, showing strategies for effective use of three different modes of sonic interaction in contemporary circus. Each design process is based on participatory studio work, involving professional circus artists. All of the proofs of concepts have been evaluated, both with studio studies and public circus performances, taking the work beyond theoretical laboratory projects and properly engaging the practice and culture of contemporary circus. The first exploration uses a contortionist's extreme bodily manipulation as inspiration for sonic manipulations in an accompanying piece of music. The second exploration uses electric amplification of acoustic sounds as a transformative enhancement of existing elements of circus performance. Finally, a sensor based system of real-time sonification of body gestures is explored and ideas from the sonification of dance are translated into the realm of circus.

 

---- Polotti, P. and Goina, M.: 'EGGS in Action'. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2011). Oslo, Norway, 2011.

Abstract:
In this paper, we discuss the results obtained by means of the EGGS (Elementary Gestalts for Gesture Sonification) system in terms of artistic realizations. EGGS was introduced in a previous edition of this conference. The works presented include interactive installations in the form of public art and interactive onstage performances. In all of the works, the EGGS principles of simplicity based on the correspondence between elementary sonic and movement units, and of organicity between sound and gesture are applied. Indeed, we study both sound as a means for gesture representation and gesture as embodiment of sound. These principles constitute our guidelines for the investigation of the bidirectional relationship between sound and body expression with various strategies involving both educated and non-educated executors.

 

---- Polotti, P. and Goina, M.: 'Per una rappresentazione audiovisiva del gesto'. In Atti del XVIII Colloquio di Informatica Musicale (XVIII CIM). Torino-Cuneo, Italy, 2010.

Abstract:
In questo articolo vengono presentati alcuni risultati dell'indagine che stiamo conducendo sulla rappresentazione audiovisiva del gesto in chiave artistica. Ci concentreremo in particolare sul lavoro Visual Sonic Enaction (VSE), un'installazione interattiva in forma di public art che permette ai visitatori di scoprire ed esplorare l'espressività; del proprio gesto. A differenza di un normale sistema di pittura interattiva, VSE vuole essere uno stimolo a creare una rappresentazione audio-visiva dei propri movimenti fisici. Come di fronte a una sorta di "specchio" interattivo, ognuno può "guardare" ed "ascoltare" il proprio gesto. Il suono e l'immagine agiscono da feedback, stimolando e guidando il movimento in modo enattivo. I risultati audiovisivi generati dal pubblico durante una sessione dell'installazione VSE vengono registrati. Dopo l'evento, questa sorta di autoritratti multi-modali sono resi disponibili su internet in modo che chiunque possa dare una "occhiata audiovisiva" all'espressività; corporea delle persone che hanno sperimentato il sistema. Questo lavoro è una tra le diverse realizzazioni inerenti lo studio della rappresentazione audiovisiva del gesto sviluppate nel corso degli ultimi tre anni ed è parte di un progetto più ampio di carattere metodologico, mirante a definire nuove strategie di indagine artistica tramite l'adozione di tecniche e modalità operative tipiche del design.

 

---- Goina, M. and Polotti, P.: 'Elementary Gestalts for Gesture Sonification'. In Proceedings of 2008 Conference on New Interfaces for Musical Expression (NIME '08), A. Camurri, S. Serafin and G. Volpe (eds), pp. 150–153. Genova, Italy, 2008.

Abstract:
In this paper, we investigate the relationships between gesture and sound by means of an elementary gesture sonification. This work takes inspiration from Bauhaus’ ideals and Paul Klee’s investigation into forms and pictorial representation. In line with these ideas, the main aim of this work is to reduce gesture to a combination of a small number of elementary components (gestalts) used to control a corresponding small set of sounds. By means of a demonstrative tool, we introduce here a line of research that is at its initial stage. The envisaged goal of future developments is a novel system that could be a composing/improvising tool as well as an interface for interactive dance and performance.

 

---- Goina, M. and Polotti, P.: 'Principi elementari per una sonificazione del gesto'. in Atti del XVII Colloquio di Informatica Musicale (XVII CIM), D. Rocchesso, A. Orvieto, A. Vidolin, and M. G. Ballerano editors, pp. 47-52. Venezia, Italy, 2008.

Abstract:
In questo lavoro viene condotta un’indagine sulle relazioni tra gesto e suono per mezzo di una sonificazione elementare del gesto. L’ispirazione per questo lavoro viene dai principi del Bauhaus ed in particolare dalla ricerca di Paul Klee nel campo delle forme e della rappresentazione pittorica. In linea con queste idee, l’obbiettivo principale di questo lavoro è quello di ridurre il gesto alla combinazione di un numero essenziale di componenti elementari (gestalt) usati per controllare un insieme di suoni corrispondentemente piccolo. Sulla base di una realizzazione software dimostrativa introduciamo qui una linea di ricerca che è ancora in corso di sviluppo. L’obbiettivo futuro è quello di realizzare un nuovo sistema che sia uno strumento atto alla composizione e/o all’improvvisazione, come pure un’interfaccia per la danza interattiva.



© 2008-2021 Maurizio Goina & Pietro Polotti