©2003 Kåre Sjölander and Jonas Beskow
Introduction Download Documentation Links Forum
WaveSurfer user forum

FAQ  FAQ
Search  Search
Memberlist  Memberlist
Register  Register
Profile  Profile
Login  Login

Speak.plug: An Mbrola plugin to speak up labels with prosody

 
Post new topic   Reply to topic    www.speech.kth.se/wavesurfer Forum Index -> WaveSurfer
View previous topic :: View next topic  
Author Message
dzach



Joined: 03 Jul 2003
Posts: 41
Location: Athens, Greece

PostPosted: Sat Jan 17, 2004 11:17 pm    Post subject: Speak.plug: An Mbrola plugin to speak up labels with prosody Reply with quote

Kare

I expose here one a plug-in I wrote for wavesurfer, speak.plug v0.9. It uses the Mbrola speech engine and databases to speak up phoneme labels from a transcription pane.

If "Speak" is clicked in the popup menu while on a trasncription pane, it speaks the phoneme labels in that pane. Otherwise, i.e if clicked in a non transcription pane, it speaks the phoneme labels in the last label pane found (if one exists).

For prosody generation, it uses the first pitch contour pane available (if one exists), otherwise it generates its own pitch data from scratch.

I am a lousy tcl programmer. I only got into this because I love wavesurfer. In order to make it to version 1.0, I need to use the canvas drawn pitch curve (which could be drawn by the user) instead of the calculated pitch. Then, it will be easy: draw and play icon_smile.gif. Hand made prosody for wavesurfer.

I pledge for help. Canvas's are not my strong point, in fact, they are not my point at all. I've searched for days in the wavesurfer code, but I'm stuck. All I need is a statement that fetches consecutive canvas pitch points, so I replace it in the code.

I hope you, or the wavesurfer community, find speak.plug useful. If indeed this happens, I'll be glad to have added a stroke to your wanderful wavesurfer.

Cheers
Dimitrios

The Speak.plug code follows. It can be Copy-pasted it in a file named "speak.plug" and put together with the other plugins.

Code:
# -----------------------------------------------------------------------------
# Speak.plug version 0.9
# Mbrola player plugin for wavesurfer
#
# By Dimitrios Zachariadis, 16.1.2004
# Portions of code copied off the "meanpitch.plug"-in of the wavesurfer distribution
# -----------------------------------------------------------------------------

wsurf::RegisterPlugin speak \
    -description "Plug-in that speaks the phonemes from the current or the last label pane found, \
    using the Mbrola engine and voices. It uses the first pitch contour pane available \
    (if one exists) for prosody generation, otherwise it generates its own from scratch. \
    Needs the mbrola executable and the speech database paths" \
    -addmenuentriesproc speak::addMenuEntries

namespace eval speak {
   variable Info
}
# ----------------------------------------------------------------------------
proc speak::addMenuEntries {w pane m hook x y} {
   if {![string match create $hook]} {
      $m add command -label "Speak" -command [namespace code [list SpeakUp $w $pane]]
   }
}
# ----------------------------------------------------------------------------
proc speak::getPitch {w} {
   variable Info

   # Use pitch values from a pitch pane if one exists, otherwise
   # compute them from scratch and remember for future reference

   foreach otherpane [$w _getPanes] {
      upvar wsurf::analysis::${otherpane}::var v
      if {$v(drawPitch)} {
         set Info($w,pitchList)     $v(pitchList)
         set Info($w,frameInterval) $v(frameInterval)
         return
      }
   }
   set s [$w cget -sound]
   set Info($w,pitchList) [$s pitch -method esps]
   set Info($w,frameInterval) 0.01
}
# ----------------------------------------------------------------------------
proc speak::SpeakUp {w pane} {
variable Info
upvar ::wsurf::trans::${pane}::var p

   if {[info exists p(nLabels)]} {
   } else {
      foreach otherpane [$w _getPanes] {
         upvar wsurf::trans::${otherpane}::var v
         if {$v(drawTranscription)} {
            upvar ::wsurf::trans::${otherpane}::var p
            echo "fount drawTrans"
            set ft 1
         }
      }
      if {!$ft} return
   }
   
   if {[info exists p(nLabels)]} {
      getPitch $w
      
        if {[catch {open "ws.pho" w} out]} {
         return $out
        } else {
            if {[info command encoding] != ""} {
               fconfigure $out -encoding $p(encoding)
         } 
         fconfigure $out -translation {auto lf}
         set start 0.0
         set tp 0.000
         set tick [expr int($Info($w,frameInterval)*1000)]
         set j 0
         set lastHz 0
         for {set i 0} {$i < $p(nLabels)} {incr i} {
            set sout ""
            set ind [lindex $p(map) $i]
            set end [expr $p(t1,$ind,end)*1000]
            set label $p(t1,$ind,label)
            set dur [expr int([expr {$end - $start}]) ]
            while {$tp < $end} {
               set hz 0.0
               set per 0
               set per [expr int(100 * [expr $tp - $start] / $dur)]
# --------------------------------------------------------------------------------------------
# The next statement needs to have the expression that reads the values replaced, so that it uses
# values drawn on the canvas (probably corrected by the user) and not from the pitchList
# In this way, corrections in the pitch curve made by the user will be played in Mbrola
# --------------------------------------------------------------------------------------------
               set hz [expr int([lindex [lindex $Info($w,pitchList) $j] 0])]
               if {$hz != 0.0} {
                    set sout "$sout $per $hz"
                    set lastHz $hz
               }
               set tp [expr $tp + $tick]
               incr j
            }
            puts $out "$label $dur $sout"
            set start $end
         }
         puts $out "_ 0 0 $lastHz"
         puts $out "#"
         close $out
         ::setMsg "Speaking file --> ws.pho"
#---------------------------------------------------------------------------------------
#         Change next string to the proper path of the mbrola executable and  database
# --------------------------------------------------------------------------------------
         eval exec "e:/speech/mbrola.exe e:/speech/databases/gr2 ws.pho ws.wav"
         snack::sound ss -load ws.wav
         ss play
        }
   }
}
Back to top
View users profile Send private message Send email Visit posters website
kare



Joined: 16 Aug 2002
Posts: 371
Location: KTH, Centre for Speech Technology

PostPosted: Mon Jan 19, 2004 11:29 pm    Post subject: Reply with quote

This is a great plug-in! I'm sure we're going to use it ourselves and it's definitely going to be part of WaveSurfer. I'll see what we can do about making the pitch curver drawable.
We'll get back to you.

Kåre
Back to top
View users profile Send private message Visit posters website
dzach



Joined: 03 Jul 2003
Posts: 41
Location: Athens, Greece

PostPosted: Tue Jan 20, 2004 12:33 am    Post subject: Reply with quote

Kåre

I'm very glad you like it!

There are two ways to use the pitch curve: By hand (or auto) drawing a simplified piecewise line on top of the f0 curve, and by manipulating the dots that form the pitch contour in a "Pitch contour" pane. I believe both are necessary, since they serve different tasks, but as it stands now, the second method is easier to implement since the dots already exist on the pitch contour pane canvas, and are indeed movable. But I haven't figured out how to access their values sequentially. Its because of my slow learning curve...

Two text boxes in a preferences page, would also eliminate the need for the casual user to hard code the Mbrola paths in the plugin.

Dimitrios
Back to top
View users profile Send private message Send email Visit posters website
kare



Joined: 16 Aug 2002
Posts: 371
Location: KTH, Centre for Speech Technology

PostPosted: Wed Jan 21, 2004 11:38 am    Post subject: Reply with quote

A collegue of mine has already made some progress regarding pitch contour editing. We'll send you a copy soon. Naturally, the plug-in should be available for installation from within WaveSurfer.

Kåre
Back to top
View users profile Send private message Visit posters website
dzach



Joined: 03 Jul 2003
Posts: 41
Location: Athens, Greece

PostPosted: Wed Jan 21, 2004 11:47 am    Post subject: Reply with quote

I'm eager to see it.

Following your help with the problem I had with version 1.6.0, I repost the corrected code for the speak.plug plugin.
Code:
# -----------------------------------------------------------------------------
# Speak.plug version 0.9.1
# Mbrola player plugin for wavesurfer
#
# By Dimitrios Zachariadis, 21.1.2004
# Portions of code copied off the "meanpitch.plug"-in of the wavesurfer distribution
# -----------------------------------------------------------------------------

wsurf::RegisterPlugin speak \
    -description "Plug-in that speaks the phonemes from the current or the last label pane found, \
    using the Mbrola engine and voices. It uses the first pitch contour pane available \
    (if one exists) for prosody generation, otherwise it generates its own from scratch. \
    Needs the mbrola executable and the speech database paths" \
    -addmenuentriesproc speak::addMenuEntries

namespace eval speak {
   variable Info
}
# ----------------------------------------------------------------------------
proc speak::addMenuEntries {w pane m hook x y} {
   if {[string match query $hook]} {
      return 0
   }    
   if {[string length $hook]==0} {
      $m add command -label "Speak" -command [namespace code [list SpeakUp $w $pane]]
   }
}
# ----------------------------------------------------------------------------
proc speak::getPitch {w} {
   variable Info

   # Use pitch values from a pitch pane if one exists, otherwise
   # compute them from scratch and remember for future reference

   foreach otherpane [$w _getPanes] {
      upvar wsurf::analysis::${otherpane}::var v
      if {$v(drawPitch)} {
         set Info($w,pitchList)     $v(pitchList)
         set Info($w,frameInterval) $v(frameInterval)
         return
      }
   }
   set s [$w cget -sound]
   set Info($w,pitchList) [$s pitch -method esps]
   set Info($w,frameInterval) 0.01
}
# ----------------------------------------------------------------------------
proc speak::SpeakUp {w pane} {
variable Info
upvar ::wsurf::trans::${pane}::var p

   if {[info exists p(nLabels)]} {
   } else {
      foreach otherpane [$w _getPanes] {
         upvar wsurf::trans::${otherpane}::var v
         if {$v(drawTranscription)} {
            upvar ::wsurf::trans::${otherpane}::var p
            set ft 1
         }
      }
   }
   
   if {[info exists p(nLabels)]} {
      getPitch $w
      
        if {[catch {open "ws.pho" w} out]} {
         return $out
        } else {
            if {[info command encoding] != ""} {
               fconfigure $out -encoding $p(encoding)
         } 
         fconfigure $out -translation {auto lf}
         set start 0.0
         set tp 0.000
         set tick [expr int($Info($w,frameInterval)*1000)]
         set j 0
         set lastHz 0
         for {set i 0} {$i < $p(nLabels)} {incr i} {
            set sout ""
            set ind [lindex $p(map) $i]
            set end [expr $p(t1,$ind,end)*1000]
            set label $p(t1,$ind,label)
            set dur [expr int([expr {$end - $start}]) ]
            while {$tp < $end} {
               set hz 0.0
               set per 0
               set per [expr int(100 * [expr $tp - $start] / $dur)]
# --------------------------------------------------------------------------------------------
# The next statement needs to have the expression that reads the values replaced, so that it uses
# values drawn on the canvas (probably corrected by the user) and not from the pitchList
# In this way, corrections in the pitch curve made by the user will be played in Mbrola
# --------------------------------------------------------------------------------------------
               set hz [expr int([lindex [lindex $Info($w,pitchList) $j] 0])]
               if {$hz != 0.0} {
                    set sout "$sout $per $hz"
                    set lastHz $hz
               }
               set tp [expr $tp + $tick]
               incr j
            }
            puts $out "$label $dur $sout"
            set start $end
         }
         puts $out "_ 0 0 $lastHz"
         puts $out "#"
         close $out
#---------------------------------------------------------------------------------------
#         Change next string to the proper path of the mbrola executable and  database
# --------------------------------------------------------------------------------------
         eval exec "e:/speech/mbrola.exe e:/speech/databases/gr2 ws.pho ws.wav"
         snack::sound ss -load ws.wav
         ss play
        }
   }
}

_________________
Dimitrios
Back to top
View users profile Send private message Send email Visit posters website
Display posts from previous:   
Post new topic   Reply to topic    www.speech.kth.se/wavesurfer Forum Index -> WaveSurfer All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB 2.0.1 © 2001, 2002 phpBB Group