A Way For Using The Echo Or Echo Dot Microphones For Vox Commando Voice Input

[Simeon Tuitt]

Hi Everyone,

I need some help so I can test out a possible solution to us being able to use the Amazon Echo and Echo Dot Microphones for use with Vox.

It will basically work by scraping the voice recording history page where everything said to an echo device you own is stored in text.

So the idea is to scrape the text and then have it passed to vox as a command, the only problem for now is I will need to name my vox Alexa or have one of it's activation words be Alexa at least since the history text contains the ACTIVATION WORD and then what was said after it.

The url that the information needs pulling from is: http://alexa.amazon.co.uk/spa/index.html?#settings/dialogs

It updates in real time after saying anything to the Echo device, so if someone can whip up some code to test it that would be great.

I will just have my robo browser login info saved so I can go straight to http://alexa.amazon.co.uk/spa/index.html?#settings/dialogs

The thing I can't work out from the code I scraped to find where the text I say appears is how to get vox to find the most recent version / command I have said as the span number in code will change as new commands come in and there will be multiple commands on a page, so how to get vox to pull just the most recent history event text.

Maybe have the robo b page maximise and then use the coordinates of the variable "dd-title d-dialog-title" to scan just that bit of text.

If you have any ideas or suggestions or can help with some code to pull the spoken command text and then have it input to vox that would be great.

Thanks in advance for any help.
Sim
P.S: Span Class= is "dd-title d-dialog-title" >whatever my command I said was goes here</span>

[Kalle]

Hi Simeon,
can you please post the whole scrape of the page (maybe as txt file)? Use the Regex tool in VC.
I'm not sure if the robo browser is really needed - a scrape command with a bit of regex may also work - only a guess.


A other question: The echo app stores the recorded audio - which format has the audio file?

[Simeon Tuitt]

Hi Kalle, I just scraped the page with the regex tool but it can't find what I am seeing on the page, attached is a text file of the scrape return, it is the same when I right click and try and view page source, it's not actually showing the page content.

In robo b however the elements and everything is all there to see as included above.

The audios are available for listening to of commands said to the echo but not downloadable, only gives the option to delete them.

Is there anyway to pull all the code from robo b?

I think you are probably right about a scrape command with regex, peg leg did something with regex for someone who wanted to know the temperature of their thermostat and it pulled the numbers from the variable and then gave them to vox.

The temperature example would probably work for this with slight modification so instead of TTS the output of my span variable "dd-title d-dialog-title" from the scraped page, instead it just inputs it as a command to vox.

http://voxcommando.com/forum/index.php?topic=2263.0

I just need to pull the inner html from the variable "dd-title d-dialog-title" and get it sent as an input to vox, then I should be able to use my current commands for vox through the echo.

Echo won't understand the voice input but everything I say will be added to the history for vox to act on.

[Simeon Tuitt]

I've noticed the DIV tag goes up in 2s and the lowest DIV tag is the latest command spoken and recorded by the echo, images attached for where I highlighted and copy of the inner HTML code from robo b..

So a rule to use lowest DIV tag variable "dd-title d-dialog-title" maybe to get vox only using the latest history entry.

[jitterjames]

I don't think you'll be able to use scrape since you probably need to log in to authenticate. Stick with robob.

Did you try using x y coordinates?

[Simeon Tuitt]

Hi James, yeah I need to be logged in, so with robo b I will just save my login so always logged in, I can have the page go full screen with a hidden window and use coordinates to grab the latest DIV / SPAN "dd-title d-dialog-title" / inner text which is the command I have spoken.

It's just some xml I need to grab the spoken command text and send it to vox, I can input the x y coordinates variables if a place is left for those.

Would regex option not work if I was always logged into the browser with my password saved?

[jitterjames]

http://voxcommando.com/mediawiki/index.php?title=Plugin_RoboB

You can use RegEx, just not scrape. Two completely different things.

[nime5ter]

We don't have access to the Amazon interface so it's something you'll need to figure out on your own.

Here is an example of how one would "scrape" (get html or text from a web page) and then evaluate the text using regular expressions.

Here I use RoboB to get recent post information from the VC forum main page.

I set the "current element"to the tag with ID "ic_recentposts".

I get (or otherwise known as scrape) the html nested within that element only. So now that information is stored in our {LastResult}.

Then I use a regular expression on that result to retrieve text that matches a particular pattern. (I used the Regex Tool to figure out the right pattern).

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.2.2.6-->
<command id="454" name="Show recent forum posts (RoboB)" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <action>
    <cmdType>RoboB.Select</cmdType>
    <params>
      <param>VCF</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.Navigate</cmdType>
    <params>
      <param>http://voxcommando.com/forum/index.php</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.Wait</cmdType>
    <params />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.Show</cmdType>
    <params />
    <cmdRepeat>0</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.ElementByID</cmdType>
    <params>
      <param>ic_recentposts</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.GetHTML</cmdType>
    <params />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RegExTool.Open</cmdType>
    <params>
      <param>True</param>
    </params>
    <cmdRepeat>0</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegEx</cmdType>
    <params>
      <param>&lt;dt.*?href="(.*?)".*?nofollow"&gt;(.*?)&lt;/</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>OSD.ShowText</cmdType>
    <params>
      <param>Recent forum posts:</param>
      <param>8000</param>
      <param>-5</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>OSD.AddText</cmdType>
    <params>
      <param>{i}. {Match.{i}.2}{CR}link: {Match.{i}.1}</param>
    </params>
    <cmdRepeat>{#M}</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.Dispose</cmdType>
    <params />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <phrase>Show recent forum posts</phrase>
</command>

This isn't really the method I'd use ordinarily because it's not the most efficient, but it's an example that anyone can test and play with.

[Dave]

Hi Simeon,

did you get it to work?
I got my dot a few days ago and found an interesting script for a custom skill on the Eventghost forum, which I use with small changes to directly send commands from Alexa to the TCP plugin of VC.
I'm still testing it, but at the moment it looks quite promising. I will post a tutorial how to set it up (hopefully) soon, because getting it to work is a bit tricky.
In the meantime, here is a short video demonstration. Sorry for the bad quality..

[vulcanjedi]

Awesome dave, super excited about your pending tutorial.

[Simeon Tuitt]

Thanks nime5ter for the input, gives me something to play around with to see if I can get things working, just been mega busy to give it a go just yet, will keep you posted on how I get on with this approach and the others.

We don't have access to the Amazon interface so it's something you'll need to figure out on your own...

[Simeon Tuitt]

Hi Dave, fantastic to see you have got something working, can you drop the steps for how you have done this and are you having to say an INVOKE SKILL activation phrase like ALEXA TELL Skill Name to... or are you saying Alexa COMMAND?

Thanks in advance and look forward to the steps.

Cheers.
Sim

Hi Simeon,

did you get it to work?
I got my dot a few days ago and found an interesting script for a custom skill on the Eventghost forum, which I use with small changes to directly send commands ...

[jitterjames]

Hi Dave, fantastic to see you have got something working, can you drop the steps for how you have done this and are you having to say an INVOKE SKILL activation phrase like ALEXA TELL Skill Name to... or are you saying Alexa COMMAND?

I think you always need to have some sort of prefix command to let Alexa know you are talking to a sub-system.

If you listen carefully to Dave when he issues his command he says "Alexa sage VOX..."  which is German for "Alexa tell VOX..."

[Dave]

Here are the instruction for installing the "EchoToVC"-Skill:

http://voxcommando.com/forum/index.php?topic=2761

[Simeon Tuitt]

Hi Dave and James, thanks for putting the Vox Alexa Skill together, it's just what I needed to use all my echo dots as microphones for vox.