[an error occurred while processing this directive]

Ghost in the Cave 

An interactive collaborative game using non-verbal communication


A collaborative game in which two teams of players use the voice or body gestures for input control. Players not directly involved in  controlling the avatars are participating by influencing the music through movements. The task is to navigate in the 3D world and to find three caves. Within each cave the players do an "emotional" task.


The aim was to test the applicability and intelligibility of expressive gesture in a new setting. First, could expressive gesture, e.g. music and dance, be used as a basis for (audience) collaboration and participation? And second, could it be used as an input control / means of communication in an interactive game environment?


Main Features

Non-verbal communication:
-Both interaction between the computer and between team and players
-Expression in vocal sounds and body movements

-Team work and audience participation

The game involves participants in an activity using non-verbal emotional expressions. Two teams use expressive gestures in either voice or body movements to compete. Each team has an avatar controlled either by singing into a microphone or by moving in front of a video camera. Players control their avatars by using acoustical or motion cues. The avatar is navigated / moving around in a  3D distributed virtual environment using the Octagon server and player system. The voice input is processed using a musical cue analysis module yielding performance variables such as tempo, sound level and articulation as well as an emotional prediction. Similarly, movements captured from the video camera are analysed in terms of different movement cues.



Interactive evening at Stockholm Music Acoustic Conference 2003 (SMAC03), August 6,

Niornas vecka, October 21-23, 2003.

Stockholm Arts and Science, Konserthuset, Sept 24-26.

Oslo, 2004

Tour of Norwegian schools 2006

Unga Klara, Stockholm, 2006

The original version of Ghost in the Cave was developed as a collaboration including four partners within the MEGA project: Speech, Music and Hearing, KTH, Stockholm; Octaga/ Telenor, Oslo; DIST InfoMus Lab, University of Genoa; DEI, University of Padua; and with CID, KTH, Stockholm.

The developers were Marie-Louise Rinman (design), Anders Friberg (audio/video interaction), Bendik Bendiksen, Ivar Kjellmo (3Dworld), Damien Cirotteau, Hugh McCarthy (audience detection and music generation) Barbara Mazzarino, Sofia Dahl (video input).Thanks to all the rest of you at the different labs that helped the realization in various ways.

The version 2004 had technical and design improvements made by KTH and Octaga as well as new music by the composers Mattias Sköld and Anders Gross.

In 2005 a third version was developed coordinated by Octaga. The aim was to simplify the technichal setup in order to facilitate touring.

Rinman, M-L, Friberg, A, Bendiksen, B, Cirotteau, D, Dahl, S, Kjellmo, I, Mazzarino, B, and Camurri, A (2004). Ghost in the Cave - an interactive collaborative game using non-verbal communication. in A. Camurri, G. Volpe (Eds.), Gesture-based Communication in Human-Computer Interaction, LNAI 2915, (pp. 549-556), Berlin: Springer Verlag.

Rinman M-L, Friberg A, Kjellmo I, Camurri, A, Cirotteau D, Dahl S, Mazzarino B, Bendiksen B  McCarthy H (2003) EPS - an interactive collaborative  game using non-verbal communication. in R. Bresin (Ed.) Proceedings of SMAC03, Vol II pp. 561- 563.

Luthander, P (2003) Datorspelet som styrs av känslor. Dagens Nyheter fredagen den 24 oktober.

Straumsheim Grønli, K (2004) Uttrykk som spillkontroll. Forskning.no


Example of movement navigation and music control avi file (4.7M only PC), avi file (12.5M PC/Mac)

From SMAC03
navigation avi file (8.8M PC/Mac)
in the "angry" cave avi file (10.6M PC/Mac)

[an error occurred while processing this directive]