Author Topic: XBMC + Live TV + VoxCommando  (Read 26061 times)

0 Members and 1 Guest are viewing this topic.

maxwel

  • Jr. Member
  • **
  • Posts: 39
  • Karma: 1
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #30 on: February 10, 2014, 01:43:53 PM »
Ok maxwel, I've translate the phrases and also the TTS.speak actions to english. I've disable in both commands the last action RoboB.Dispose, so the RoboB browser stay open after the command. In this way you can see/play in the browser window by your self when you enable the "tracking" function at the top of this windows, which will show you the data behind the elements on this website when you move the mouse over it.

copy the VcGrp_TV guideEN.xml in your command tree. Save the tvguide.xml and minutestoGgCal.xml in the VC payload folder.

1. You need find a website with a TV guide for your country and then figure out which elements you need.
2. You must edit the payload tvguide.xml for your channels.

Good luck  ;D
Kalle

Hi Kalle,

I have followed your instruction well but I have a small problem.. I have found the TV guide for my country but the right page is not showing up! so I still need your help Kalle! can you fix that for me please? here is the website with a TV guide ( http://www.tvgids.tv/nu-en-straks )

Thanks you

Grtz Max

Dave

  • $upporter
  • Sr. Member
  • *****
  • Posts: 139
  • Karma: 31
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #31 on: February 10, 2014, 03:52:38 PM »
Hey Max,
I played a little with Kalles xml and your Website.
The problem is, that the tags we need to find the right text on the site, seem to change during the (not only on your website, but also on every german tv guide website i tried...).
so you can get it work, but only for a few minutes or maybe hours.  >:(
but here is an example how you could do it with your site:

Code: [Select]
<?xml version="1.0" encoding="utf-16"?>
<command id="224" name="TV guide jetzt - current" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <action>
    <cmdType>RoboB.Select</cmdType>
    <cmdString>tvguide</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.SetWinSize</cmdType>
    <cmdString>1920&amp;&amp;1080</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.Show</cmdType>
    <cmdString />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.Navigate</cmdType>
    <cmdString>http://www.tvgids.tv/nu-en-straks</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.Wait</cmdType>
    <cmdString />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>PY.ExecString</cmdType>
    <cmdString>result = 2*{1}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.ElementByTag</cmdType>
    <cmdString>STRONG&amp;&amp;{LastResult}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.GetText</cmdType>
    <cmdString />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegExReplace</cmdType>
    <cmdString>(0\B)</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString>:0&amp;&amp; Uhr </cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString>:&amp;&amp; Uhr</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.SetVar</cmdType>
    <cmdString>time1</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>PY.ExecString</cmdType>
    <cmdString>result = {1}*2+1</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.ElementByTag</cmdType>
    <cmdString>STRONG&amp;&amp;{LastResult}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.GetText</cmdType>
    <cmdString />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegExReplace</cmdType>
    <cmdString>(0\B)</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString>:0&amp;&amp; Uhr </cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString>:&amp;&amp; Uhr</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.SetVar</cmdType>
    <cmdString>time2</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>PY.ExecString</cmdType>
    <cmdString>result = 7*{1}+42</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.ElementByTag</cmdType>
    <cmdString>DIV&amp;&amp;{LastResult}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.GetText</cmdType>
    <cmdString />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString>&lt;STRONG&gt; </cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString> &lt;/STRONG&gt;</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegExReplace</cmdType>
    <cmdString>(..:..)</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.SetVar</cmdType>
    <cmdString>title1</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>PY.ExecString</cmdType>
    <cmdString>result = 7*{1}+45</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.ElementByTag</cmdType>
    <cmdString>DIV&amp;&amp;{LastResult}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>RoboB.GetText</cmdType>
    <cmdString />
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString>&lt;STRONG&gt; &lt;/STRONG&gt;</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegExReplace</cmdType>
    <cmdString>(..:..)</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>OSD.ShowText</cmdType>
    <cmdString>{Var.title1}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>TTS.SpeakSync</cmdType>
    <cmdString>seit {Var.time1} läuft {Var.title1}. Im Anschluss folgt ab {Var.time2} {LastResult}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <phrase>was läuft jetzt, was läuft, was kommt gerade, current program guide for</phrase>
  <phrase optional="true">auf, im, in</phrase>
  <payloadFromXML phraseOnly="False" use2partPhrase="False" phraseConnector="by" Phrase2wildcard="anyone">payloads\tvguide.xml</payloadFromXML>
</command>

you need to activate the python plugin for this.
i only added the first 4 programs to the tvguide.xml
sorry, but the tts and phrases are in german again, so you have to translate it.

while the tags on the website change, the position of the elements maybe remains the same. perhaps you can change selection by tag to a selction by coordinates...

jitterjames

  • Administrator
  • Hero Member
  • *****
  • Posts: 7715
  • Karma: 116
    • View Profile
    • VoxCommando
Re: XBMC + Live TV + VoxCommando
« Reply #32 on: February 10, 2014, 06:21:27 PM »
Hi Dave,

Can you please tell me which tags (specifically) are changing?  Do you think they are doing this on purpose to prevent people from scraping their sites?  If that is the case then it is my duty to find a way around it ;)

If you could show me a chunk of html and specify which tags change that would be very helpful.  Thanks.

Dave

  • $upporter
  • Sr. Member
  • *****
  • Posts: 139
  • Karma: 31
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #33 on: February 11, 2014, 04:15:03 AM »
Hey James, thank you for taking a look!

Code: [Select]
<div class="section-item now-and-later">
<div class="section-item-content">

<div class="channel">
<a class="channel-icon sprite-channel-1" href="/zenders/nederland-1" title="TV Gids Nederland 1"></a>
</div>

<a href="/tv/nos-olympische-spelen-sotsji-live" title="NOS Olympische Spelen Sotsji Live"></a>
<a class="now" href="/tv/nos-olympische-spelen-sotsji-live/10662767" title="NOS Olympische Spelen Sotsji Live" rel="nofollow">
<div class="content ellipsis">
<strong>08:00</strong>
NOS Olympische Spelen Sotsji Live
</div>
<div class="progress">
<div class="bar" style="width: 0%;" data-start-time="1392102000000" data-end-time="1392105600000"></div>
</div>
</a>

<a href="/tv/nos-olympische-spelen-sotsji-live" title="NOS Olympische Spelen Sotsji Live"></a>
<a class="later" href="/tv/nos-olympische-spelen-sotsji-live/10662768" title="NOS Olympische Spelen Sotsji Live" rel="nofollow">
<div class="content ellipsis">
<strong>09:00</strong>
NOS Olympische Spelen Sotsji Live
</div>
</a>
</div>
</div>


<div class="section-item now-and-later">
<div class="section-item-content">

<div class="channel">
<a class="channel-icon sprite-channel-2" href="/zenders/nederland-2" title="TV Gids Nederland 2"></a>
</div>

<a href="/tv/nos-jeugdjournaal-met-gebarentolk" title="NOS Jeugdjournaal met gebarentolk"></a>
<a class="now" href="/tv/nos-jeugdjournaal-met-gebarentolk/10662802" title="NOS Jeugdjournaal met gebarentolk" rel="nofollow">
<div class="content ellipsis">
<strong>08:45</strong>
NOS Jeugdjournaal met gebarentolk
</div>
<div class="progress">
<div class="bar" style="width: 0%;" data-start-time="1392104700000" data-end-time="1392105600000"></div>
</div>
</a>

<a href="/tv/nos-journaal" title="NOS-Journaal"></a>
<a class="later" href="/tv/nos-journaal/10662805" title="NOS-Journaal" rel="nofollow">
<div class="content ellipsis">
<strong>09:00</strong>
NOS-Journaal
</div>
</a>
</div>
</div>

this is a piece of the html from the website maxwel found. but its basically the same problem with all the german sites.
i don't know if they do it on purpose. at least i don't see a reason why changing the tags would be necessary, but i'm not a programmer...
the "<div>"-tags, which i use to get the title change, while the "<strong>"-tags, which i use to get the time, remain.
i hope this is what you wanted me to show  ???

a workaround could be to get the tag of one object by coordinates (if thats possible). on some sites you could than calculate the other tags by using the first tag as a reference.

jitterjames

  • Administrator
  • Hero Member
  • *****
  • Posts: 7715
  • Karma: 116
    • View Profile
    • VoxCommando
Re: XBMC + Live TV + VoxCommando
« Reply #34 on: February 11, 2014, 11:22:26 AM »
Yes it is what I was asking, but I'm still not sure what it is about those div tags that is changing.  In any case...

This site is a good example of a basic principle.  While I love RoboBrowser for the things that I allows me to do, I prefer to use scrape whenever it is possible.  Scrape is faster and sometimes more reliable.  I will use RoboBrowser when I really have no choice, such as, when a site only loads content dynamically after the initial page load, via javascript calls etc.  In this case scrape is useless.  Or if a site requires to you log in and it maintains session variables with your login info.  For this you need RoboBrowser.

This site though, loads all the content we need in the initial call so scrape is really perfect, fast, and probably less likely to be broken (although that really depends on how you use it).

Here is my command for getting the "What 's on now and then later for a given channel name" using the site: http://www.tvgids.tv/nu-en-straks

Maybe mine will break too, but I tested it last night, and then again this morning and it seems to work.  This command is designed to work with a payloadXML file and you must ask for a channel by name.  It is not designed to work from a channel number, so if you were going to try to detect what channel number you were already watching and then use that to see what was playing, it would need some modification.  As it is I can ask "what is playing on HBO" and it will respond: "Now:Thing Ice, Later: True Detective"

I have attached the payloadXML file with (I think) all the channels.  I recommend that you edit the list to remove any channels that you know you will never ever use, and to edit the phrases for the ones you do use to make sure they can be easily understood by the engine, and to create comma separated aliases for them if you like.  I started at the top and edited a few.  In particular, phrases for any shows that are made up of letters should be separated by spaces (H B O instead of HBO) so that the engine expects to hear them spelled out.  Fox can be left as Fox since we say it like a word.  In some cases, if you are not sure you could use an alias and do it both ways.  On the value column, you should not edit it unless you notice that i have made a mistake.  I have used . in the values for spaces because . matches any character but if you were to enter a space it would just ignore it.  This is needed for the regex to work, but you could use the more accurate \s (which stands for whitespace character), I just find that using \s makes it harder to read (and is more work to type:))

Here is the command:
Code: [Select]
<?xml version="1.0" encoding="utf-16"?>
<command id="208" name="tv guide scrape" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <action>
    <cmdType>Scrape</cmdType>
    <cmdString>http://www.tvgids.tv/nu-en-straks</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>System.SetClipboardText</cmdType>
    <cmdString>{LastResult}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegExSingle</cmdType>
    <cmdString>title="TV.Gids.{1}".*?title="(.*?)".*?rel="nofollow".*?title="(.*?)".*?rel="nofollow"&amp;&amp;, Later: </cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>OSD.ShowText</cmdType>
    <cmdString>Now: {Match.1}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>TTS.Speak</cmdType>
    <cmdString>Now: {Match.1}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <phrase>what is playing on</phrase>
  <payloadFromXML phraseOnly="False" use2partPhrase="False" phraseConnector="by" Phrase2wildcard="anyone" optional="False">payloads\nustraks.xml</payloadFromXML>
</command>

Dave: See if you can use this technique with your other site but let me know if you need help.  I enjoy these puzzles.
« Last Edit: February 11, 2014, 11:36:29 AM by jitterjames »

jitterjames

  • Administrator
  • Hero Member
  • *****
  • Posts: 7715
  • Karma: 116
    • View Profile
    • VoxCommando
Re: XBMC + Live TV + VoxCommando
« Reply #35 on: February 11, 2014, 12:45:57 PM »
I also noticed that we can scrape more information by using the url for a particular channel:

Note that you must use this new payloadXML (attached) in order for the URL to work.

Code: [Select]
<?xml version="1.0" encoding="utf-16"?>
<command id="230" name="What is next on {1}" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <action>
    <cmdType>Scrape</cmdType>
    <cmdString>http://www.tvgids.tv/zenders/{1}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegExSingle</cmdType>
    <cmdString>&lt;div.class=\"section-content\"&gt;(.*)</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.SetLastResult</cmdType>
    <cmdString>{Match.1}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.RegExSingle</cmdType>
    <cmdString>class=\"section-item-title\"&gt;\s+(..:..)\s+(.*?)\s+&lt;/span&gt;&amp;&amp; : </cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.MatchConcat</cmdType>
    <cmdString>{CR}then at &amp;&amp;6</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>Results.Replace</cmdType>
    <cmdString>#039;&amp;&amp;'</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <action>
    <cmdType>OSD.ShowText</cmdType>
    <cmdString>IN progress, started at {LastResult}</cmdString>
    <cmdRepeat>1</cmdRepeat>
  </action>
  <phrase>what is next on</phrase>
  <payloadFromXML phraseOnly="False" use2partPhrase="False" phraseConnector="by" Phrase2wildcard="anyone" optional="False">payloads\nustaksnext.xml</payloadFromXML>
</command>

Kalle

  • $upporter
  • Hero Member
  • *****
  • Posts: 2319
  • Karma: 47
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #36 on: February 11, 2014, 12:46:48 PM »
I've  did not test it yet, but that is why we ask you for help - great stuff!  You are the "Voxmaster"  ::bow
***********  get excited and make things  **********

maxwel

  • Jr. Member
  • **
  • Posts: 39
  • Karma: 1
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #37 on: February 11, 2014, 02:52:49 PM »


Hey,

It works like a charm! Thank you so much for your good efforts guys - I really appreciate it - fantastic service as ever!

Thanks guys,

Ps: I hope you're also helped dave

Max
« Last Edit: February 11, 2014, 02:58:43 PM by jitterjames »

Kalle

  • $upporter
  • Hero Member
  • *****
  • Posts: 2319
  • Karma: 47
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #38 on: February 12, 2014, 05:55:54 AM »
Great job James - it works perfect and really fast (much faster as with RoboB) the result comes after milliseconds  :clap

As next I will figured out, how we can have a result by time - like "what is running at 1 pm on channel xyz"

Kalle
***********  get excited and make things  **********

Dave

  • $upporter
  • Sr. Member
  • *****
  • Posts: 139
  • Karma: 31
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #39 on: February 12, 2014, 07:00:27 AM »
Thank you James! ::bow

After learning to use robob, regex and python to do it my (or Kalles) way, I guess now I have to learn how the scrape command works  ::hmm ;)

@Kalle: hast du James Methode schon mit einer deutschen Seite probiert?

Kalle

  • $upporter
  • Hero Member
  • *****
  • Posts: 2319
  • Karma: 47
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #40 on: February 12, 2014, 08:40:13 AM »
@Dave

Nein, nur mit tvgids.tv. Wenn Du schon was erstellt hast, lass es mich wissen :D
***********  get excited and make things  **********

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 2012
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: XBMC + Live TV + VoxCommando
« Reply #41 on: February 12, 2014, 10:11:25 PM »
@Dave & Kalle:

I made some tv guide commands for you to try. These scrape the website http://www.tvinfo.de/, which Kalle used in his original RoboB commands.

1. What's playing now on [channel name]?
2. What's showing next on [channel name]?
3. What's playing on [channel name] at [time]? (This currently only works for today's date, but you guys can adapt it for other days.)

Of course, you will need to translate the phrases into German and adjust the payload xml phrases as well.

Unfortunately, that site uses the iso-8859-1 character set instead of utf-8 (who does that nowadays??), and my attempts to re-encode the text efficiently have thus far failed, so I just used "results.replace" to fix the character set.  :( It's not pretty, but it works.
« Last Edit: February 12, 2014, 10:29:36 PM by nime5ter »
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

Kalle

  • $upporter
  • Hero Member
  • *****
  • Posts: 2319
  • Karma: 47
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #42 on: February 13, 2014, 02:54:53 AM »
Big Thanks to nime5ter  :hugs x 1.000.000.000.000.000.000...

I've tried this and now it works perfect!  ::bow

I'm really surprised how easy looks the command in VC. When I look at the website tvinfo.de and at the HTML code, I would never find what I'm looking for to create the command.

Kalle
« Last Edit: February 13, 2014, 03:52:37 AM by Kalle »
***********  get excited and make things  **********

nime5ter

  • Administrator
  • Hero Member
  • *****
  • Posts: 2012
  • Karma: 61
    • View Profile
    • Getting Started with VoxCommando
Re: XBMC + Live TV + VoxCommando
« Reply #43 on: February 13, 2014, 08:58:03 AM »
Thanks. I'm glad it works for you.

Actually, you can clean up the regex for the 2nd, "show next" command. I was a bit messy while playing with the regex last night.

You are really just trying to find the second "floatLeft" tag after the channel name, so you can use:

Code: [Select]
<span>{1}</span>.*?floatLeft.>.*?floatLeft.>(.*?)<.*?class="bold">(.*?)<
This looks for the channel you ask for between <span> tags and then captures two separate patterns. It hunts for the second instance of a tag that has floatLeft"> in it. It captures the text directly after that (the time), then continues scanning until it encounters class="bold">and captures the text after that (the show title).

Just in case others look at this thread, I want to explain that this regex catches two separate patterns, but these are interpreted as a single match {Match.1}.

So, if it finds the first pattern after floatLeft"> but no class="bold"> after that, it will not find any matches.

I'm then defining how those two strings are concatenated together by including a delimiter - {CR} as the second, optional Results.Regex parameter. This delimiter formats the two-part match more neatly in my OSD message. Otherwise it just squishes the two patterns together into one string.

A {Match.2} would be found if this entire regular expression were found more than once on the scraped page.
« Last Edit: February 13, 2014, 09:20:52 AM by nime5ter »
TIPS: POST VC VERSION #. Explain what you want VC to do. Say what you've tried & what happened, or post a video demo. Attach VC log. Link to instructions followed.  Post your command (xml)

Kalle

  • $upporter
  • Hero Member
  • *****
  • Posts: 2319
  • Karma: 47
    • View Profile
Re: XBMC + Live TV + VoxCommando
« Reply #44 on: February 13, 2014, 09:20:11 AM »
Thanks for explanation  :bignod
***********  get excited and make things  **********