Aims

The aim was to test the applicability and intelligibility of expressive gesture in a new setting. First, could expressive gesture, e.g. music and dance, be used as a basis for (audience) collaboration and participation? And second, could it be used as an input control / means of communication in an interactive game environment?

Main Features

Non-verbal communication:
-Both interaction between the computer and between team and players
-Expression in vocal sounds and body movements

Collaboration:
-Team work and audience participation

The game involves participants in an activity using non-verbal emotional expressions. Two teams use expressive gestures in either voice or body movements to compete. Each team has an avatar controlled either by singing into a microphone or by moving in front of a video camera. Players control their avatars by using acoustical or motion cues. The avatar is navigated / moving around in a  3D distributed virtual environment using the Octagon server and player system. The voice input is processed using a musical cue analysis module yielding performance variables such as tempo, sound level and articulation as well as an emotional prediction. Similarly, movements captured from the video camera are analysed in terms of different movement cues.

Presentations

Interactive evening at Stockholm Music Acoustic Conference 2003 (SMAC03), August 6,

Niornas vecka, October 21-23, 2003.

Stockholm Arts and Science, Konserthuset, Sept 24-26.

Oslo, 2004

Tour of Norwegian schools 2006

Unga Klara, Stockholm, 2006

Development
The original version of Ghost in the Cave was developed as a collaboration including four partners within the MEGA project: Speech, Music and Hearing, KTH, Stockholm; Octaga/ Telenor, Oslo; DIST InfoMus Lab, University of Genoa; DEI, University of Padua; and with CID, KTH, Stockholm.

The developers were Marie-Louise Rinman (design), Anders Friberg (audio/video interaction), Bendik Bendiksen, Ivar Kjellmo (3Dworld), Damien Cirotteau, Hugh McCarthy (audience detection and music generation) Barbara Mazzarino, Sofia Dahl (video input).Thanks to all the rest of you at the different labs that helped the realization in various ways.

The version 2004 had technical and design improvements made by KTH and Octaga as well as new music by the composers Mattias Sköld and Anders Gross.

In 2005 a third version was developed coordinated by Octaga. The aim was to simplify the technichal setup in order to facilitate touring.

Publications
Rinman, M-L, Friberg, A, Bendiksen, B, Cirotteau, D, Dahl, S, Kjellmo, I, Mazzarino, B, and Camurri, A (2004). Ghost in the Cave - an interactive collaborative game using non-verbal communication. in A. Camurri, G. Volpe (Eds.), Gesture-based Communication in Human-Computer Interaction, LNAI 2915, (pp. 549-556), Berlin: Springer Verlag.

Rinman M-L, Friberg A, Kjellmo I, Camurri, A, Cirotteau D, Dahl S, Mazzarino B, Bendiksen B  McCarthy H (2003) EPS - an interactive collaborative  game using non-verbal communication. in R. Bresin (Ed.) Proceedings of SMAC03, Vol II pp. 561- 563.

Luthander, P (2003) Datorspelet som styrs av känslor. Dagens Nyheter fredagen den 24 oktober.

Straumsheim Grønli, K (2004) Uttrykk som spillkontroll. Forskning.no

Videos

Example of movement navigation and music control avi file (4.7M only PC), avi file (12.5M PC/Mac)

From SMAC03
navigation avi file (8.8M PC/Mac)
in the "angry" cave avi file (10.6M PC/Mac)