|
Vetenskapsfestivalen, Göteborg, 1999.05.07
Synthesis of Emotional Expression in Piano Music
(Abstracts in English and Swedish)
Roberto Bresin
Department of Speech Music & Hearing (TMH)
Royal Institute of Technology (KTH) - Stockholm
Listening test
During the seminar at the Röhsska Design Museum, Göteborg, a forced choice listening test was carried
out where listeners of the public were asked to classify the performances of a melody according to seven emotional
alternatives. 19 people did the test. Some days later (99.05.10) the same test was done during a seminar for electro-acoustic
music students, at the Department of Speech Music & Hearing, KTH, Stockholm.
The piece of music used for the test was the melody line of a Swedish
nursery tune (Ekorrn satt i granen, henceforth Ekorrn,
"The squirrel sat on the fir-tree", composed by Tegner) written in major tonality.
Twentysix listeners, 15-57 years old, average age 34, 8 female and 18 male, were subjects during the two seminars.
Subjects were instructed to identify the emotional expression of each example of Ekorrn
as one out of seven alternative substantives: fear, anger, happiness,
sadness, solemnity, tenderness, no-expression. The responses were automatically written on a form that was previuosly
given to all the people in the public (click here to download a copy of the form in Word format)
The subjects listened to the stimuli over the loudspeakers in the seminar room.
The order of the performances was the following:
- No-expression (MIDI file, Sound file)
- Sadness (MIDI file, Sound
file)
- Happiness (MIDI file, Sound file)
- Tenderness (MIDI file, Sound file)
- Anger (MIDI file, Sound file)
- Fear (MIDI file , Sound file)
- Solemnity (MIDI file, Sound file)
- Sadness (MIDI file, Sound file)
- Happiness (MIDI file, Sound file)
- No-expression (MIDI file, Sound file)
The average duration of the experiment was approximately 5 minutes.
Data obtained from the listening tests were analized and the results are partly
summerized in the following confusion matrixes:
Confusion
matrixes (%) for the classification test of seven synthesized performances of Ekorrn by 26 subjects.
- Table 1 All
the listeners: 26 subjects.
- Table 2 Listeners at the Göteborg seminar: 19 subjects.
- Table 3 Electro-acoustic music students: 7 subjects.
- Table 4 Male listeners: 18 subjects.
- Table 5 Female listeners: 8 subjects.
- Table 6 Listeners who play an instrument: 11 subjects.
- Table 7 Listeners who used to play an instrument: 6 subjects.
- Table 8 Singers: 5 subjects.
- Table 9 Non musically-trained listeners: 8 subjects.
- Table 10 All musically-trained listeners: 18 subjects.
- Table 11 All listeners who play an instrument and /or sing: 16 subjects.
Table 1 All
the listeners: 26 subjects. (back to table
menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
19
|
0
|
4
|
15
|
27
|
0
|
35
|
|
ANGER
|
0
|
85
|
12
|
0
|
0
|
3
|
0
|
|
FEAR
|
54
|
0
|
19
|
4
|
23
|
0
|
0
|
|
SOLEMNITY
|
0
|
8
|
12
|
42
|
0
|
3
|
35
|
|
SADNESS
|
31
|
0
|
4
|
3
|
62
|
0
|
0
|
|
HAPPINESS
|
0
|
12
|
0
|
0
|
0
|
81
|
8
|
|
NO EXPRESSION
|
3
|
0
|
0
|
12
|
0
|
4
|
81
|
Table 2 Listeners
at the Göteborg seminar: 19 subjects. (back
to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
21
|
0
|
0
|
11
|
21
|
0
|
47
|
|
ANGER
|
0
|
79
|
16
|
0
|
0
|
5
|
0
|
|
FEAR
|
53
|
0
|
21
|
0
|
26
|
0
|
0
|
|
SOLEMNITY
|
0
|
11
|
16
|
42
|
0
|
5
|
26
|
|
SADNESS
|
26
|
0
|
0
|
6
|
68
|
0
|
0
|
|
HAPPINESS
|
0
|
16
|
0
|
0
|
0
|
74
|
10
|
|
NO EXPRESSION
|
5
|
0
|
0
|
16
|
0
|
5
|
74
|
Table 3 Electro-acoustic
music students: 7 subjects. (back to table
menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
14
|
0
|
14
|
29
|
43
|
0
|
0
|
|
ANGER
|
0
|
100
|
0
|
0
|
0
|
0
|
0
|
|
FEAR
|
57
|
0
|
14
|
14
|
14
|
0
|
0
|
|
SOLEMNITY
|
0
|
0
|
0
|
43
|
0
|
0
|
57
|
|
SADNESS
|
43
|
0
|
14
|
0
|
43
|
0
|
0
|
|
HAPPINESS
|
0
|
0
|
0
|
0
|
0
|
100
|
0
|
|
NO EXPRESSION
|
0
|
0
|
0
|
0
|
0
|
0
|
100
|
Table 4 Male
listeners: 18 subjects. (back to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
22
|
0
|
5
|
17
|
28
|
0
|
28
|
|
ANGER
|
0
|
89
|
11
|
0
|
0
|
0
|
0
|
|
FEAR
|
67
|
0
|
17
|
6
|
11
|
0
|
0
|
|
SOLEMNITY
|
0
|
5
|
11
|
39
|
0
|
6
|
39
|
|
SADNESS
|
28
|
0
|
6
|
0
|
67
|
0
|
0
|
|
HAPPINESS
|
0
|
5
|
0
|
0
|
0
|
89
|
6
|
|
NO EXPRESSION
|
5
|
0
|
0
|
6
|
0
|
6
|
83
|
Table 5 Female
listeners: 8 subjects. (back to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
13
|
0
|
0
|
12
|
25
|
0
|
50
|
|
ANGER
|
0
|
75
|
12
|
0
|
0
|
13
|
0
|
|
FEAR
|
25
|
0
|
25
|
0
|
50
|
0
|
0
|
|
SOLEMNITY
|
0
|
13
|
12
|
50
|
0
|
0
|
25
|
|
SADNESS
|
38
|
0
|
0
|
12
|
50
|
0
|
0
|
|
HAPPINESS
|
0
|
25
|
0
|
0
|
0
|
63
|
12
|
|
NO EXPRESSION
|
0
|
0
|
0
|
25
|
0
|
0
|
75
|
Table 6 Listeners
who play an instrument: 11 subjects. (back
to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS |
ANGER |
FEAR |
SOLEMNITY |
SADNESS |
HAPPINESS |
NO EXPRESSION |
|
TENDERNESS
|
18
|
0
|
10
|
18
|
18
|
0
|
36
|
|
ANGER
|
0
|
91
|
9
|
0
|
0
|
0
|
0
|
|
FEAR
|
64
|
0
|
18
|
0
|
18
|
0
|
0
|
|
SOLEMNITY
|
0
|
9
|
9
|
55
|
0
|
0
|
27
|
|
SADNESS
|
27
|
0
|
9
|
9
|
55
|
0
|
0
|
|
HAPPINESS
|
0
|
9
|
0
|
0
|
0
|
91
|
0
|
|
NO EXPRESSION
|
0
|
0
|
0
|
0
|
0
|
0
|
100
|
Table 7 Listeners
who used to play an instrument: 6 subjects. (back
to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
17
|
0
|
0
|
0
|
16
|
0
|
67
|
|
ANGER
|
0
|
100
|
0
|
0
|
0
|
0
|
0
|
|
FEAR
|
50
|
0
|
17
|
0
|
33
|
0
|
0
|
|
SOLEMNITY
|
0
|
0
|
33
|
33
|
0
|
17
|
17
|
|
SADNESS
|
17
|
0
|
0
|
0
|
83
|
0
|
0
|
|
HAPPINESS
|
0
|
33
|
0
|
0
|
0
|
67
|
0
|
|
NO EXPRESSION
|
0
|
0
|
0
|
16
|
0
|
17
|
67
|
Table 8 Singers:
5 subjects. (back to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
0
|
0
|
0
|
20
|
40
|
0
|
40
|
|
ANGER
|
0
|
100
|
0
|
0
|
0
|
0
|
0
|
|
FEAR
|
40
|
0
|
40
|
0
|
20
|
0
|
0
|
|
SOLEMNITY
|
0
|
0
|
0
|
80
|
0
|
0
|
20
|
|
SADNESS
|
60
|
0
|
0
|
0
|
40
|
0
|
0
|
|
HAPPINESS
|
0
|
20
|
0
|
0
|
0
|
60
|
20
|
|
NO EXPRESSION
|
0
|
0
|
0
|
0
|
0
|
0
|
100
|
Table 9 Non
musically-trained listeners: 8 subjects. (back
to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
25
|
0
|
0
|
12
|
38
|
0
|
25
|
|
ANGER
|
0
|
75
|
12
|
0
|
0
|
13
|
0
|
|
FEAR
|
50
|
0
|
25
|
12
|
13
|
0
|
0
|
|
SOLEMNITY
|
0
|
13
|
12
|
25
|
0
|
0
|
50
|
|
SADNESS
|
37
|
0
|
0
|
0
|
63
|
0
|
0
|
|
HAPPINESS
|
0
|
0
|
0
|
0
|
0
|
100
|
0
|
|
NO EXPRESSION
|
12
|
0
|
0
|
25
|
0
|
0
|
63
|
Table 10 All
musically-trained listeners: 18 subjects. (back
to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
17
|
0
|
6
|
17
|
22
|
0
|
39
|
|
ANGER
|
0
|
89
|
11
|
0
|
0
|
0
|
0
|
|
FEAR
|
56
|
0
|
17
|
0
|
28
|
0
|
0
|
|
SOLEMNITY
|
0
|
6
|
11
|
50
|
0
|
6
|
28
|
|
SADNESS
|
28
|
0
|
6
|
6
|
61
|
0
|
0
|
|
HAPPINESS
|
0
|
17
|
0
|
0
|
0
|
72
|
11
|
|
NO EXPRESSION
|
0
|
0
|
0
|
6
|
0
|
6
|
89
|
Table 11 All
listeners who play an instrument and /or sing: 16 subjects. (back
to table menu)
| |
Classified Emotion
|
|
Intended Emotion
|
TENDERNESS
|
ANGER
|
FEAR
|
SOLEMNITY
|
SADNESS
|
HAPPINESS
|
NO EXPRESSION
|
|
TENDERNESS
|
9
|
0
|
5
|
19
|
29
|
0
|
38
|
|
ANGER
|
0
|
95
|
5
|
0
|
0
|
0
|
0
|
|
FEAR
|
52
|
0
|
29
|
0
|
19
|
0
|
0
|
|
SOLEMNITY
|
0
|
4
|
5
|
67
|
0
|
0
|
24
|
|
SADNESS
|
44
|
0
|
4
|
5
|
47
|
0
|
0
|
|
HAPPINESS
|
0
|
15
|
0
|
0
|
0
|
75
|
10
|
|
NO EXPRESSION
|
0
|
0
|
0
|
0
|
0
|
0
|
100
|
© Roberto
Bresin - Last update: 1999.07.12
|