TTS - Talking Head

[jitterjames]

In VC2 version 1.955 we added an experimental feature.  A "talking head" that can show an animation when using the TTS.Speak action.

This feature is experimental, and we may decide not to keep it as a standard part of VoxCommando if it causes too many problems, but I will try at least to keep it as an option in an alternate version of the TTS plugin that you can use.  Maybe it will be OK and we can keep it.

There are a few limitations.  The first is that the talking head will only be animated when using TTS.Speak.  It does not work with TTS.SpeakSync (or speak2wav) because SpeakSync essentially freezes the program until speaking is finished.  I am trying to find a work around for this problem but so far I am not having too much luck.

There is a new TTS action for showing and hiding the window containing the talking head.  TTS.ShowHead takes one parameter.  True to show the head and False to hide the head.  In an upcoming version I may add an optional parameter to "keep on top" or something similar.  This will not allow you to show the head in front of certain "screen stealing" programs such as WMC or XBMC in "true fullscreen" mode, but will hopefully allow us to show the head in front of all other "normal" windows.

If you look in the plugin folder ..yourVcFolder\Plugins\TTS\visemes you will see all the still images used for the animation of the head when speaking.  You can use the following link as a reference for the mouth positions

http://aidreams.co.uk/forum/index.php?page=Visemes_-_for_Character_Animation#VISEME_1_.28silence.29

Image 01 is the face at rest, Image 23 is the face when blinking.  You can use any file names you want but they should be in the correct alphabetical order so be sure to use the same name for all files, and pad the numbers with a '0'

eg:  file01.jpg, file02.jpg, file03.jpg

You can try to create your own images and place them in this "visemes" folder.  We are hoping that if you come up with something good you will share it with the rest of us by posting a link to the files you have created!  You can use 3D software, photography, pull stills from video, illustration, clay scultpure, or just edit images in your favourite drawing program.  Anything goes, just try to keep the main "head" in a still position to avoid jerking motions.

[jitterjames]

Here is an example of an alternate set of viseme images.  I did not quite get the lip positions right on this try though.  It is a subtle art!

https://www.dropbox.com/s/i897d2g8becf22e/visemes%20blue%20woman.rar

[nime5ter]

A few more features/fixes have been added to the talking head as of version 1.9.5.6:

http://voxcommando.com/mediawiki/index.php?title=ChangeLog#Version_1.9.5.6

[bp_pbs]

YES!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[jank]

i use the talking head and it works great   does anyone have any other images?

[jitterjames]

You can make your own!

[jank]

yes i know.
I'm working on it

[Kalle]

Maybe it would help if I record Spock from guile3d for each of the 23 visemes when he speak? What do you think?

[jitterjames]

That sounds like a pretty clear copyright infringement.  Users are free to do what they want with the software in private but I don't think I could allow this  material to remain as an attachment on the public forum.

[jank]

That's cool. But I would not post it in the forum

[Kalle]

@James

Yes sorry, I have not thought about it.

[rio14]

hi jitterjames and all forum
is it possible add the option to the talking head, it appear and comes when we call him and disappear when there is no activity.thank you

[nime5ter]

Welcome to the forum.

There are a variety of TTS actions for controlling the talking head. http://voxcommando.com/mediawiki/index.php?title=Plugin_TTS

It depends what you mean by "when there is no activity".

The only way to trigger a command is with a voice command or an event trigger. If there is "no activity" there will not be any events to trigger the hide talking head command. The only option I can think of is to set a timer.

If you want to issue a voice command to "show talking head" and then have the talking head hide after a period of time, one method is:

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.2.1.4-->
<commandGroup open="True" name="talking head simple show hide" enabled="True" prefix="" priority="0" requiredProcess="" description="">
  <command id="149" name="Show Talking Head" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
    <action>
      <cmdType>TTS.ShowHead</cmdType>
      <params>
        <param>True</param>
        <param>False</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>TTS.MoveHead</cmdType>
      <params>
        <param>10</param>
        <param>10</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>VC.SetEventTimer</cmdType>
      <params>
        <param>4m</param>
        <param>hidehead</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <phrase>Show Talking Head</phrase>
  </command>
  <command id="150" name="Hide Talking Head" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
    <action>
      <cmdType>TTS.ShowHead</cmdType>
      <params>
        <param>False</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <phrase>Hide Talking Head</phrase>
    <event>hidehead</event>
  </command>
</commandGroup>
In the above example, you give a voice command to show the talking head. That command will set an event timer to trigger the "Hide Talking Head" command in four minutes (you can change the time to x seconds, x minutes, whatever).

You can also experiment with using the two automatic events that occur any time there is a TTS.Speak action: tts.start and/or tts.end

But that might get messy.

[rio14]

thank you for your answer, yes what i wante is automatic appear of the head and automatic hide i did an other face is for denese images takes from youtube it can replace the original in VC, i dont like attache it, i dont know if is legal or illigal, but if some one interested i send it by email, its my first attempt, im beginner in VC
im looking for a good xml codes, the new file is visemes just keep copy of the original and put the new in .yourVcFolder\Plugins\TTS\visemes, thank you

[jitterjames]

You can show and hide the TTS talking head quite easily using the events generated when speech starts and stops and the actions for showing and hiding the head:

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.2.1.5-->
<commandGroup open="True" name="TTS show / hide automatically" enabled="True" prefix="" priority="0" requiredProcess="" description="">
  <command id="294" name="auto show head" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
    <action>
      <cmdType>TTS.ShowHead</cmdType>
      <params>
        <param>True</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <event>tts.start</event>
  </command>
  <command id="328" name="auto hide head" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
    <action>
      <cmdType>TTS.ShowHead</cmdType>
      <params>
        <param>False</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <event>tts.end</event>
  </command>
</commandGroup>
Note that in the current version of VC the head will disappear and reappear again immediately (flicker) if you have two or more TTS.Speak actions in your macro.  In the next release the tts.end event will be slightly delayed to avoid this situation.