The EGGS system
EGGS (Elementary Gestalts for Gesture
Sonification) is a gesture sonification system, developed by Maurizio
Goina and Pietro Polotti, School of Music and New Tecnologies,
Conservatorio G. Tartini, Trieste, Italy. A crucial contribution as
performer and user of the system is provided
by Sarah Taylor.
Journal article:
---- Polotti, P., and Goina, M.: 'Gesture Sonification.
An Interaction Design Approach to an Artistic Research Case'. In EAI Endorsed Transactions
on Creative Technologies, ct 18: e1, 2021-03-19, ISSN 2409-9708.
Abstract:
INTRODUCTION: The subject of this paper is a phenomenological study of cognitively plausible relationships between gesture and sound mediated by technologies. In this discussion, the term ‘gesture’ indicates a movement of the body consciously performed and able to express or communicate something. The expression ‘cognitively plausible’ refers to an interactive sound response that is immediate, continuously varying and enactively coherent with the generating (sonified) gesture.
OBJECTIVES: The objectives of this paper are two. The first is an artistic exploration of gesture sonifications in various contexts. The second is rather methodological, i.e., to provide a possible general paradigm for Artistic Research (AR). METHODS: More in detail, the AR phenomenological approach is modeled according to an Interaction Design (IxD) research paradigm.
RESULTS: A number of case studies of gesture sonification are presented and discussed according to the above methodological framework.
CONCLUSION: We claim that the introduction of such methodological framework was successful in terms of providing robust guidelines for our research and for a clear and structured presentation of its results.
Conference presentations:
---- Elblaus, L., Goina, M., Robitaille, M. A., and Bresin, R.: 'Modes of sonic interaction in circus: Three proofs of concept'. In Proceedings of Sound and Music Computing Conference, 2014, Athens.
Abstract:
The art of circus is a vibrant and competitive culture that embraces new tools and technology. In this paper, a series of exploratory design processes resulting in proofs of concepts are presented, showing strategies for effective use of three different modes of sonic interaction in contemporary circus. Each design process is based on participatory studio work, involving professional circus artists. All of the proofs of concepts have been evaluated, both with studio studies and public circus performances, taking the work beyond theoretical laboratory projects and properly engaging the practice and culture of contemporary circus.
The first exploration uses a contortionist's extreme bodily manipulation as inspiration for sonic manipulations in an accompanying piece of music. The second exploration uses electric amplification of acoustic sounds as a transformative enhancement of existing elements of circus performance. Finally, a sensor based system of real-time sonification of body gestures is explored and ideas from the sonification of dance are translated into the realm of circus.
---- Polotti, P. and Goina,
M.: 'EGGS in Action'. In Proceedings of the International
Conference on New Interfaces for Musical Expression (NIME 2011).
Oslo, Norway,
2011.
Abstract:
In this paper, we discuss the results obtained by means of the EGGS
(Elementary Gestalts for Gesture Sonification) system in terms of
artistic realizations. EGGS was introduced in a previous edition of
this conference. The works presented include interactive installations
in the form of public art and interactive onstage performances. In all
of the works, the EGGS principles of simplicity based on the
correspondence between elementary sonic and movement units, and of
organicity between sound and gesture are applied. Indeed, we study both
sound as a means for gesture representation and gesture as embodiment
of sound. These principles constitute our guidelines for the
investigation of the bidirectional relationship between sound and body
expression with various strategies involving both educated and
non-educated executors.
---- Polotti,
P. and Goina,
M.: 'Per una rappresentazione audiovisiva del gesto'. In Atti del
XVIII
Colloquio di Informatica Musicale (XVIII CIM). Torino-Cuneo,
Italy,
2010.
Abstract:
In questo articolo vengono presentati alcuni risultati
dell'indagine che stiamo conducendo sulla
rappresentazione audiovisiva del gesto in chiave
artistica. Ci concentreremo in particolare sul lavoro
Visual Sonic Enaction (VSE), un'installazione
interattiva in forma di public art che permette ai
visitatori di scoprire ed esplorare l'espressività; del
proprio gesto. A differenza di un normale sistema di
pittura interattiva, VSE vuole essere uno stimolo a
creare una rappresentazione audio-visiva dei propri
movimenti fisici. Come di fronte a una sorta di
"specchio" interattivo, ognuno può "guardare" ed
"ascoltare" il proprio gesto. Il suono e l'immagine
agiscono da feedback, stimolando e guidando il
movimento in modo enattivo. I risultati audiovisivi
generati dal pubblico durante una sessione
dell'installazione VSE vengono registrati. Dopo
l'evento, questa sorta di autoritratti multi-modali sono
resi disponibili su internet in modo che chiunque possa
dare una "occhiata audiovisiva" all'espressività;
corporea delle persone che hanno sperimentato il
sistema.
Questo lavoro è una tra le diverse realizzazioni inerenti
lo studio della rappresentazione audiovisiva del gesto
sviluppate nel corso degli ultimi tre anni ed è parte di
un progetto più ampio di carattere metodologico,
mirante a definire nuove strategie di indagine artistica
tramite l'adozione di tecniche e modalità operative
tipiche del design.
---- Goina,
M. and Polotti, P.: 'Elementary Gestalts for Gesture Sonification'. In Proceedings
of 2008 Conference on New Interfaces for Musical Expression (NIME '08),
A. Camurri, S. Serafin and G. Volpe (eds), pp. 150–153. Genova,
Italy,
2008.
Abstract:
In this paper, we investigate the relationships between gesture and
sound by means of an elementary gesture sonification. This work takes
inspiration from Bauhaus’ ideals and Paul Klee’s
investigation into
forms and pictorial representation. In line with these ideas, the main
aim of this work is to reduce gesture to a combination of a small
number of elementary components (gestalts) used to control a
corresponding small set of sounds. By means of a demonstrative tool, we
introduce here a line of research that is at its initial stage. The
envisaged goal of future developments is a novel system that could be a
composing/improvising tool as well as an interface for interactive
dance and performance.
---- Goina,
M. and Polotti, P.: 'Principi
elementari per una sonificazione del gesto'. in Atti del XVII
Colloquio di Informatica Musicale (XVII CIM), D. Rocchesso, A.
Orvieto, A. Vidolin, and M. G. Ballerano editors, pp. 47-52. Venezia,
Italy, 2008.
Abstract:
In questo lavoro viene condotta un’indagine sulle relazioni tra
gesto e
suono per mezzo di una sonificazione elementare del gesto.
L’ispirazione per questo lavoro viene dai principi del Bauhaus ed
in
particolare dalla ricerca di Paul Klee nel campo delle forme e della
rappresentazione pittorica. In linea con queste idee,
l’obbiettivo
principale di questo lavoro è quello di ridurre il gesto alla
combinazione di un numero essenziale di componenti elementari (gestalt)
usati per controllare un insieme di suoni corrispondentemente piccolo.
Sulla base di una realizzazione software dimostrativa introduciamo qui
una linea di ricerca che è ancora in corso di sviluppo.
L’obbiettivo futuro è quello di realizzare un nuovo
sistema che
sia uno strumento atto alla composizione e/o all’improvvisazione,
come
pure un’interfaccia per la danza interattiva.
|