Virtual Virtuosity
Studies in automatic music performance

Click here to download the dissertation (.PDF file)
Click here to download the dissertation errata (.PDF file)


This dissertation presents research in the field of automatic music performance with a special focus on piano.
A system is proposed for automatic music performance, based on artificial neural networks (ANNs). A complex, ecological-predictive ANN was designed that listens to the last played note, predicts the performance of the next note, looks three notes ahead in the score, and plays the current tone. This system was able to learn a professional pianist's performance style at the structural micro-level. In a listening test, performances by the ANN were judged clearly better than deadpan performances and slightly better than performances obtained with generative rules.
The behavior of an ANN was compared with that of a symbolic rule system with respect to musical punctuation at the micro-level. The rule system mostly gave better results, but some segmentation principles of an expert musician were only generalized by the ANN.
Measurements of professional pianists' performances revealed interesting properties in the articulation of notes marked staccato and legato in the score. Performances were recorded on a grand piano connected to a computer. Staccato was realized by a micropause of about 60% of the inter-onset-interval (IOI) while legato was realized by keeping two keys depressed simultaneously; the relative key overlap time was dependent of IOI: the larger the IOI, the shorter the relative overlap. The magnitudes of these effects changed with the pianists' coloring of their performances and with the pitch contour. These regularities were modeled in a set of rules for articulation in automatic piano music performance.
Emotional coloring of performances was realized by means of macro-rules implemented in the Director Musices performance system. These macro-rules are groups of rules that were combined such that they reflected previous observations on musical expression of specific emotions. Six emotions were simulated. A listening test revealed that listeners were able to recognize the intended emotional colorings.
In addition, some possible future applications are discussed in the fields of automatic music performance, music education, automatic music analysis, virtual reality and sound synthesis.

Keywords: music, performance, expression, interpretation, piano, automatic, artificial neural networks, rules, articulation, legato, staccato, emotion, virtual reality, human computer interaction, perception, music education, Director Musices, JAPER, PANN, computer music, MIDI, MidiShare, Disklavier, Bösendorfer, cellular phone, mobile phone, MPEG-7, Java, Lisp