« previous next »
Pages: [1]

Author Topic: Request - Streetcar ETA Updates + Reminder  (Read 6285 times)

0 Members and 1 Guest are viewing this topic.

h7329

  • Jr. Member
  • **
  • Posts: 6
  • Karma: 0
    • View Profile
Request - Streetcar ETA Updates + Reminder
« on: April 15, 2015, 09:53:49 PM »
Hi everyone:

I'm new to VoxCommando, so apologies if this request comes across as basic. But I need your help. I've not been able to find an implementation that I can adapt, so I'm posting here for help. Any kind of assistance would be appreciated.

I'm trying to voxcommando to provide me with ETA updates of the streetcar and if possible provide two reminders based on the results returned. So, for example, I'd like to be able to say something like "Westbound Queen Streetcar" or "Eastbound Dundas Streetcar" and voxcommando retrieves the estimated arrival times of the next two street cars from http://www.nextbus.com.

From www.nextubs.com, each streetcar and stop has a custom url. For example, in "http://www.nextbus.com/#!/ttc/501/501_1_501Sun/1958" it's "http://www.nextbus.com/#!/ttc/"STREETCAR LINE"/"DIRECTION"/"STOP". I noticed that the nextbus embeds the next two street car ETAs in the page title. Therefore, after watching a few tutorials online, I was able to scrape for ETAs and get voxcommando to read them out loud. I then made 4 separate commands (each with a unique url for the 4 possible streetcar lines and directions I needed ETAs for). Unfortunately, I noticed that the results returned didn't correspond with the updates for the desired streetcar lines and corresponding stops (i.e., voxcommando seemed to be scrapping from the same url, even though for each direction, I entered unique urls). I don't know what streetcar results it's returning. Should I have scraped elsewhere other than the title? I'm attaching the xml that I created.

1. Is there a streamlined way of doing this? I want to be able to have at least 4 unique results (2 street car lines, headed in 2 directions). I experimented with a payload list but didn't know what I was doing. For example, I thought I could just ask "Give me a street car status for" and then have voxcommando for for me to dictate a line and direction like "Westbound Queen" or "Eastbound Queen" or "Westbound Dundas" etc.

2. It takes me about 7 minutes to get from my house to the stops I've requested. At the end of every "streetcar status" inquiry where voxcommando returns at least one value greater than ten minutes, can voxcommando ask if I would like a reminder? If I respond "yes" then, it will set and announce reminders 10 and 7 minutes before the streetcar arrives.

For example, I ask voxcommando for the street car status and tell it for which line and direction. It tells me 12 and 15 minutes. Since at least one number is higher than 10 minutes, it then asks me if I would like a reminder. If I confirm "yes" for reminder, it sets the reminders, using the lower number of results returned that is greater than 10. (For instance, if the results returned were 3 minutes and 11 minutes the reminder would be based on 11 minutes). At 10 minutes away (i.e., 2 minutes after the result of 12 minutes was retrieved), voxcommando would speak "Westbound Queen Streetcar in 10 minutes" and at 7 minutes away (i.e., 5 minutes after the result of 12 minutes was retrieved) voxcommando would speak "Westbound Queen Streetcar in 7 minutes. Leave Now. Leave Now."

The general format is something like "2 min" and "9 min" and also "Due" or "Arriving" along with the direction.

Here are the directions and stops I'm interested in:
http://www.nextbus.com/#!/ttc/501/501_1_501Sun/1958 - Queen Westbound
http://www.nextbus.com/#!/ttc/501/501_0_501Sun/2666 - Queen Eastbound
http://www.nextbus.com/#!/ttc/505/505_1_505Sun/8606 - Dundas Westbound
http://www.nextbus.com/#!/ttc/505/505_0_505Sun/8159 - Dundas Eastboud

Any assistance would be appreciated.
Logged

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 1999
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: Request - Streetcar ETA Updates + Reminder
« Reply #1 on: April 15, 2015, 10:50:09 PM »
Welcome to the forum.

For the basic scraping, you'll want to use payload xml -- associate each URL with the ttc line/direction as the phrase.

The command below -- and the attached payload XML file, which you can put in your payloads folder -- are the streamlined way to ask for all four streetcar routes in one command.

Code: [Select]
OLD CODE REMOVED -- new code in post below

For the timer setting ... it's past my bedtime to process that, so let's save that for tomorrow. :) But in the meantime I have changed your regular expression to capture each arrival time separately. {Match.1.1} represents the next time and {Match.1.2} the following time.

If you want to do math, you'll need to use the Python plugin. But for now you could use a logic block in the LCB to make some basic choices if one of the results is >10 min.
« Last Edit: April 16, 2015, 07:34:33 AM by nime5ter »
Logged
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

h7329

  • Jr. Member
  • **
  • Posts: 6
  • Karma: 0
    • View Profile
Re: Request - Streetcar ETA Updates + Reminder
« Reply #2 on: April 16, 2015, 12:54:55 AM »
Thanks, nime5ter for looking at this and I'm excited by the prospect of math operations.

I did some testing and unfortunately the same problem persists. Regardless of the streetcar line that is said (e.g., "Dundas Westbound" or "Queen Westbound") VC returns the same prediction. The prediction also doesn't seem to be coming from any of the urls in the payload. I compared the returned prediction with predictions coming from a browser (see attached image). They were all very different. Any ideas what could be going on?
Logged

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 1999
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: Request - Streetcar ETA Updates + Reminder
« Reply #3 on: April 16, 2015, 07:23:43 AM »
The scrape is working correctly. The data being scraped is incorrect. (i.e., if you look at the page source for the URLs in question you'll see that they contain the outdated info in the page element you/we were scraping).

So that means you'll need to find another source of data, whether on this page or elsewhere. ... Or we may need to use the RoboBrowser plugin. I have some time this morning so I'll look into it.
Logged
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 1999
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: Request - Streetcar ETA Updates + Reminder
« Reply #4 on: April 16, 2015, 10:15:45 AM »
As expected, RoboBrowser is the solution. You'll need to enable the RoboBrowser plugin, if you haven't yet.

This is because the standard scrape action gets the web page's source text before the javascript has run. As mentioned, this is also what you'll see if you look at the source code yourself in your browser.

RoboBrowser gives us more sophisticated means to interact with web pages. In the commands below, we navigate to the correct page, wait for it to load, then isolate the text within specific page elements to return the 2 next times as {Match.1} and {Match.2}. You could also grab the third time if you wanted, as {Match.3}.

Code: [Select]
<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.1.4.5-->
<commandGroup open="True" name="ttc" enabled="True" prefix="" priority="0" requiredProcess="" description="">
  <command id="85" name="test streetcar" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="" loopMax="" description="">
    <action>
      <cmdType>RoboB.Select</cmdType>
      <params>
        <param>test</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>Results.SetVar</cmdType>
      <params>
        <param>car</param>
        <param>{PF.1}</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Navigate</cmdType>
      <params>
        <param>{1}</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Show</cmdType>
      <params>
        <param>False</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <phrase>Give me streetcar status for</phrase>
    <payloadFromXML phraseOnly="False" use2partPhrase="False" phraseConnector="by" Phrase2wildcard="anyone" optional="False">payloads\ttc.xml</payloadFromXML>
  </command>
  <command id="86" name="++announce streetcar" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="" loopMax="" description="">
    <action>
      <cmdType>TTS.Speak</cmdType>
      <params>
        <param>{var.car} Streetcar.</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Select</cmdType>
      <params>
        <param>test</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>VC.Pause</cmdType>
      <params>
        <param>3000</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.ElementByID</cmdType>
      <params>
        <param>prediction</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.GetHTML</cmdType>
      <params />
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RegExTool.Open</cmdType>
      <params>
        <param>True</param>
      </params>
      <cmdRepeat>0</cmdRepeat>
    </action>
    <action>
      <cmdType>Results.RegEx</cmdType>
      <params>
        <param>&lt;td&gt;(\d+)&lt;/td&gt;</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>OSD.ShowText</cmdType>
      <params>
        <param>{var.car} Streetcar. First car: {Match.1} minutes.  Next car in {Match.2} minutes.</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>TTS.Speak</cmdType>
      <params>
        <param>First car in: {Match.1} minutes.  Next car in {Match.2} minutes.</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>VC.Pause</cmdType>
      <params>
        <param>2000</param>
      </params>
      <cmdRepeat>0</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Select</cmdType>
      <params>
        <param>test</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Dispose</cmdType>
      <params />
      <cmdRepeat>1</cmdRepeat>
    </action>
    <event>RoboB.DocComplete.test</event>
  </command>
</commandGroup>
Logged
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 1999
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: Request - Streetcar ETA Updates + Reminder
« Reply #5 on: April 16, 2015, 12:25:59 PM »
Here's a reminder solution that comes close to what you were asking for, for the streetcars over 10 minutes away. It probably requires more stress testing.

Plugins required: RoboBrowser & Python

Code: [Select]
Replaced by new code below
If you're comfortable scripting in Python, you could implement more sophisticated logic than the above. However, that would likely be overkill for this purpose -- particularly given that the site's predicted arrival times change at a moment's notice.
« Last Edit: April 27, 2015, 11:27:44 AM by nime5ter »
Logged
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

h7329

  • Jr. Member
  • **
  • Posts: 6
  • Karma: 0
    • View Profile
Re: Request - Streetcar ETA Updates + Reminder
« Reply #6 on: April 16, 2015, 02:04:01 PM »
Thank you! I--and more importantly, my employer  :) --appreciate all your help!

I've run a few tests and it seems to be working! I was able to successfully get updates and set a reminders for an arriving streetcar. I'll run a few more tests and share anything else I find out. Once again, thank you. This is very useful.
Logged

h7329

  • Jr. Member
  • **
  • Posts: 6
  • Karma: 0
    • View Profile
Re: Request - Streetcar ETA Updates + Reminder
« Reply #7 on: April 21, 2015, 01:12:14 PM »
After a bit of testing/use I'm finding out that from time to time, the commands crash VC. Most times (probably 3 out of 4), it's successfully but the crash seems to be happening in between returning a prediction and asking if I want I want to set a reminder. (So, I can see that the streetcars are more than 10 mins away, and then it crashes.) Any assistance would be appreciated.
Logged

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 1999
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: Request - Streetcar ETA Updates + Reminder
« Reply #8 on: April 21, 2015, 01:22:09 PM »
I can do a bit more testing here. I may have made some mistakes when I made those commands.

It would be helpful if you could enable logging in VC and then post a log after the next crash.

http://voxcommando.com/mediawiki/index.php?title=Log
Logged
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

h7329

  • Jr. Member
  • **
  • Posts: 6
  • Karma: 0
    • View Profile
Re: Request - Streetcar ETA Updates + Reminder
« Reply #9 on: April 21, 2015, 03:59:56 PM »
Thanks! I was wondering how I could generate a log file. :) I'll report back as soon as I replicate the behaviour.
Logged

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 1999
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: Request - Streetcar ETA Updates + Reminder
« Reply #10 on: April 21, 2015, 05:25:14 PM »
Yes, logging is always helpful. :)

In the meantime I was able to duplicate the problem. I couldn't quite figure out the culprit, but my quick attempt at a timer method wasn't very thought through so it's not that surprising that it failed. (My bad!)

I'd recommend using a python script to handle all the logic instead.

You can replace the whole TTC command group in your tree with the following version, and you should also download the attached ttc.py script and put it in the PY folder of your VC directory. The python script will load when your first start VoxCommando so you'll need to restart VC to test this command group.

Code: [Select]
<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.1.4.5-->
<commandGroup open="True" name="ttc" enabled="True" prefix="" priority="0" requiredProcess="" description="">
  <command id="85" name="get streetcar" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="" loopMax="" description="">
    <action>
      <cmdType>RoboB.Select</cmdType>
      <params>
        <param>test</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Navigate</cmdType>
      <params>
        <param>{1}</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Show</cmdType>
      <params>
        <param>False</param>
      </params>
      <cmdRepeat>0</cmdRepeat>
    </action>
    <if ifBlockDisabled="False" ifNot="False">
      <ifType>(A)Contains(B)</ifType>
      <ifParams>{PF.1}&amp;&amp;http</ifParams>
      <then>
        <action>
          <cmdType>Results.SetVar</cmdType>
          <params>
            <param>car</param>
            <param>{2}</param>
          </params>
          <cmdRepeat>1</cmdRepeat>
        </action>
      </then>
      <else>
        <action>
          <cmdType>Results.SetVar</cmdType>
          <params>
            <param>car</param>
            <param>{PF.1}</param>
          </params>
          <cmdRepeat>1</cmdRepeat>
        </action>
      </else>
    </if>
    <phrase>Give me streetcar status for</phrase>
    <payloadFromXML phraseOnly="False" use2partPhrase="False" phraseConnector="by" Phrase2wildcard="anyone" optional="False">payloads\ttc.xml</payloadFromXML>
    <event>ttc.reminder</event>
  </command>
  <command id="86" name="++announce streetcar" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="" loopMax="" description="">
    <action>
      <cmdType>TTS.Speak</cmdType>
      <params>
        <param>Checking schedule.</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Select</cmdType>
      <params>
        <param>test</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>VC.Pause</cmdType>
      <params>
        <param>3000</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.ElementByID</cmdType>
      <params>
        <param>prediction</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.GetHTML</cmdType>
      <params />
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>Results.RegEx</cmdType>
      <params>
        <param>&lt;td&gt;(\d+)&lt;/td&gt;</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Select</cmdType>
      <params>
        <param>test</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>RoboB.Dispose</cmdType>
      <params />
      <cmdRepeat>1</cmdRepeat>
    </action>
    <if ifBlockDisabled="False" ifNot="False">
      <ifType>(A)Contains(B)</ifType>
      <ifParams>{Match.3}&amp;&amp;Match</ifParams>
      <then>
        <action>
          <cmdType>PY.ExecString</cmdType>
          <params>
            <param>ttc_alert('{var.car}',{Match.1},{Match.2},0)</param>
          </params>
          <cmdRepeat>1</cmdRepeat>
        </action>
      </then>
      <else>
        <action>
          <cmdType>PY.ExecString</cmdType>
          <params>
            <param>ttc_alert('{var.car}',{Match.1},{Match.2},{Match.3})</param>
          </params>
          <cmdRepeat>1</cmdRepeat>
        </action>
      </else>
    </if>
    <event>RoboB.DocComplete.test</event>
  </command>
  <command id="647" name="set ttc reminder" enabled="true" alwaysOn="False" confirm="True" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
    <action>
      <cmdType>TTS.Speak</cmdType>
      <params>
        <param>Will do.</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>VC.SetEventTimer</cmdType>
      <params>
        <param>{2}m</param>
        <param>ttc.reminder</param>
        <param>{1}</param>
        <param>{PF.1}</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <action>
      <cmdType>VC.SetEventTimer</cmdType>
      <params>
        <param>{3}m</param>
        <param>ttc.reminder</param>
        <param>{1}</param>
        <param>{PF.1}</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <phrase>set ttc reminder for</phrase>
    <payloadFromXML phraseOnly="False" use2partPhrase="False" phraseConnector="by" Phrase2wildcard="anyone" optional="False">payloads\ttc.xml</payloadFromXML>
    <phrase>in</phrase>
    <payloadRange>1,20</payloadRange>
    <phrase>and</phrase>
    <payloadRange>1,10</payloadRange>
    <phrase>minutes</phrase>
  </command>
  <command id="655" name="load ttc.py" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
    <action>
      <cmdType>PY.ExecFile</cmdType>
      <params>
        <param>PY\ttc.py</param>
      </params>
      <cmdRepeat>1</cmdRepeat>
    </action>
    <event>VC.Loaded</event>
  </command>
</commandGroup>

Hopefully this will work better for you.
Logged
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

h7329

  • Jr. Member
  • **
  • Posts: 6
  • Karma: 0
    • View Profile
Re: Request - Streetcar ETA Updates + Reminder
« Reply #11 on: April 21, 2015, 05:29:26 PM »
Thanks a lot, nime5ter. I'll replace the command tree, try it out, and report back. I appreciate your help.
Logged

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 1999
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: Request - Streetcar ETA Updates + Reminder
« Reply #12 on: April 21, 2015, 06:00:11 PM »
One thing:

If you want to check a streetcar line only once, and then have VC give you alerts at 10 minutes and the 7 minutes before the originally predicted arrival time *without bothering to verify that arrival time*, the python script can be much simpler.

I sort of forgot your original request in the process of working with the actual reported times, which did not seem very reliable. :)
Logged
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)
Pages: [1]
« previous next »