Large Payload XML Warning

[macrho]

Each time I generate XML from xSQL and restart VC, I get the warning:

Payload XML file XbmcPayload\xbmcSongs.xml contains 28,677 items.
Do you wish to load this command?

Is there a way I can stop this from being asked?

[jitterjames]

 You really should see if there is a way to filter out some songs to reduce the size of that list.

...or if you are not using the play song command then you should just delete the command.

But yeah that was a new feature I was experimenting with and I forgot to turn it off.  I will fix it in the next release.

Sorry about that!

[nime5ter]

As one example of ways to filter the songs payloadXML to reduce the number of items, see: http://voxcommando.com/forum/index.php?topic=1318.msg11351#msg11351

[Kalle]

But yeah that was a new feature I was experimenting with and I forgot to turn it off.  I will fix it in the next release.

This is a nice feature if VC will generate the xml by accident, maybe it is possible to have a check mark in the options menu to disable this feature (VoxCommando 2.0)   

[macrho]

For me, one of the absolute coolest features of VC is that I can say "play song..."
I'd hate to have to resort to using only a subset of my songs and then have to rank them all
There are no optimizations available to make large libraries easier to handle in VC?

[nime5ter]

It sounds like you're just getting an advisory message (which James says he'll remove). If the "play song" command is working for you and you use it a lot, the options are to either cope with the loading time or find a filtering method that's appropriate for your situation. It doesn't necessarily have to be by rating.

Coming up with a filtering method that's appropriate for your situation *is* the best method of optimizing the loading. (You can also turn off subset matching for that particular payloadXML, if you haven't yet. You'd then have to say the complete song title for the play song command to work. That is also a form of optimization.)

I just noticed that James provided another possible way for you to filter your songs.xml the other day, based on play count. http://voxcommando.com/forum/index.php?topic=1325.msg11411#msg11411

Why not try that for a week or so, and see how much it impacts things before deciding it's not an option?

If you're regularly asking to play songs that have never been played before, it will quickly become obvious and you'll know it's not the right solution for you. But if it ends up happening only rarely, in those cases you ask for the artist or album in order to find/play the desired song that one time, and then it will be added to your played songs list in short order -- problems solved.

[jitterjames]

For me, one of the absolute coolest features of VC is that I can say "play song..."
I'd hate to have to resort to using only a subset of my songs and then have to rank them all

That is your choice.

There are no optimizations available to make large libraries easier to handle in VC?

Sure you can make them smaller, or you can use dictation.  The fact that you can have 30 thousand songs and ask for any one of them using only part of the song name is not impressive enough?

I  tried to add the line

Code: [Select]

System.Window.MakeitallFasterAndBetterAndUseLessMemoryToAchieveTheImpossible(true); but sadly it threw an error. 

[macrho]

James, you got it all wrong, this is what you need to add:

Code: [Select]

System.Window.SatisfyCustomerNeeds(true);

See, all fixed! I can have my cake and eat it too 

[nime5ter]

[img width= height= alt=sunglasses shop]http://sunglasses.name/gif/michaelmoonwalking.gif[/img]

[jitterjames]

Try this version:

https://www.dropbox.com/s/l6vvknfzwds3ph8/VC%20Setup%201.186.exe

The limit is 90K items, which is not to say that it will be able to load a library of that size, but it will not warn you.

[macrho]

thank you, reloaded without issue

Try this version:

https://www.dropbox.com/s/l6vvknfzwds3ph8/VC%20Setup%201.186.exe

The limit is 90K items, which is not to say that it will be able to load a library of that size, but it will not warn you.

[macrho]

I'm now on version 2.1.3.8 and still receiving the large payloadXML warning, is there perhaps a flag to suppress the message?

[jitterjames]

What message are you receiving exactly?

And what version of the plugin do you have?

[jitterjames]

Never mind.  I found it.  I was looking in xSql for this message but it's actually for any payloadXML.

I think I must have applied the 90K limit in version 1 only during the time that I was working on both versions simultaneously.

Anyway, I'm going to add an option so you can set the warning limit manually in options.

[macrho]

Thanks much 

[vulcanjedi]

My actor payload is throwing this warning, any suggestions on filtering for this category?
Not sure I just updated to Kodi finally and latest VC after being lazy.
Love the new updates., Thanks James / Nimst3r!

[jitterjames]

Sounds like you need to delete some of those "B Movies".

Maybe you can filter it down to actors that appear in 2 or more movies?

[nime5ter]

If you want to try the filter that James suggests above, I *think* this query will work:

Code: [Select]

SELECT distinct idActor, strActor from actors WHERE idActor IN (select idActor from actorlinkmovie GROUP BY idActor HAVING COUNT(*)>1) ORDER BY strActor

[vulcanjedi]

Thanks Nime5ter, I have been trying to attempt James' suggestion however my repeated attempts this wknd have show my sql querying is apparently not a particular strength of mine.
Running the sql command directly from mysql  dropped the results by nearly 2/3rds!
I pasted your query but it fails in VC with the following for me?
...

---------------------------
Fatal error encountered during command execution.

SCANNING ABORTED

[jitterjames]

That sql was tested using the xSql plugin (sqlite) so you may need to adapt it to work in MySql.  We don't have a working MySQL setup at the moment so we can't do much in the way of testing over here.

Maybe someone else using MySql can pitch in... nudge nudge wink wink pegleg... 

[vulcanjedi]

That sql was tested using the xSql plugin (sqlite) so you may need to adapt it to work in MySql.  We don't have a working MySQL setup at the moment so we can't do much in the way of testing over here.

Maybe someone else using MySql can pitch in... nudge nudge wink wink pegleg... 

You guys continue to amaze. Thanks for the headsup about it being xSql vs Mysql.  Odd that I pasted directly in mysql prompt and it ran fine.  Took a while on the first attempt in mysql but when i looked back it did finish. In VC however it errors out  not sure if the plugin is timing out or something or query syntax could be optimized for mysql

[vulcanjedi]

You guys continue to amaze. Thanks for the headsup about it being xSql vs Mysql.  Odd that I pasted directly in mysql prompt and it ran fine.  Took a while on the first attempt in mysql but when i looked back it did finish. In VC however it errors out  not sure if the plugin is timing out or something or query syntax could be optimized for mysql

Not sure if the MusicVids is error, cause its' not selected, but if not then I may be right on the 'timing out'

[nime5ter]

Yes, it looks like the server is timing out during the query. It's not the music videos.

By the way, do you actually use the commands in VC that require the actors payload XML? Because obviously if you don't use those commands, you can just delete the "offending" commands.

I'm not saying you should do that. Just thought I'd mention it, in case it's relevant.

Maybe James can extend the timeout values in a future release of the plugin (he's not here right now, so I don't know). This is not my area. I can't tell from the error whether that's a timeout on the server side that's beyond VC's control or not.

[jitterjames]

I think you just have an unusually large number of movies.

How long does it take to execute the query directly on MySql?  I don't want to create a super long timeout (assuming that is even an option) because it could potentially cause freezing for people when they have other issues.

[PegLegTV]

I just tested nime5ter's Sql query and it worked for me, I have 227 movies and it said it found 2259 actors so at least we know the Sql query is correct, and works with MySql 

[jitterjames]

Thanks.

I guess I didn't realize how many actors Kodi indexes per movie.  It's quite a high ratio of actors to movies.

[vulcanjedi]

Yes I thought the actors volume was a lot more than it used to be as well. I typically prescrape my media with MediaCompanion to create NFOs and am sure to deselect actor scraping with that tool.
I guess I shouldn't wait too long to update my Windows // XBMC/Kodi // VoxCommando all at once, my roommate was out of town and I felt ambitious!
Hmm I just reran the query directly in mysql again and again its taking a long time. I wouldn't suggest altering the timeout too drastically as you suggest, perhaps optimizing this query will be more efficient, maybe I can ask some dba colleagues at work tomorrow.
I had to kill the query just now I would say it was taking over 2-3mins if it wasn't hung.
I just ran your original query and per mysql supposedly it took just 0.13 secs and I still cant reproduce how i managed to get the suggested query to return but when it did I reran it several times and after that it was much more akin to the response time of yours...odd.
UPDATE it just finished.... (6 min 29.32 sec) lol

[jitterjames]

Yes I thought the actors volume was a lot more than it used to be as well. I typically prescrape my media with MediaCompanion to create NFOs and am sure to deselect actor scraping with that tool.

It sounds like you are not interested in actors so I suggest you just delete the actor query row from your xMySql settings. Problem solved.

[nime5ter]

James came up with a good query this morning that may work better for you:

Code: [Select]

SELECT distinct idActor, strActor from actors WHERE idActor IN (select idActor from actorlinkmovie WHERE iOrder < 2)
This will create a list with the top 2 billed actors in your movies. We've also removed the "sort by", which really isn't necessary at all and may have contributed to your problem.

Try with this, and if it works well, you could consider increasing the iOrder number to include, say, the top 4 billed actors or so.

[jitterjames]

I just found this thread... 

http://voxcommando.com/forum/index.php?topic=1143.msg9590#msg9590

Looks like all of this has been covered including a 3rd option for an SQL query.   

[vulcanjedi]

Cheers guys,
I used this mornings suggestion as it performed better than my attempts from the previous thread.
I tweaked the billing order around a bit but the query is all coming back in the 0.1# secs range directly on mysql and in the plugin its successful as well.
I do think I'll need to try something else for the SongsArtists one as well since thats large like the other thread also suggests.

Thanks for the support as always.

http://voxcommando.com/forum/index.php?topic=1143.msg9590#msg9590

James came up with a good query this morning that may work better for you:

Code: [Select]

SELECT distinct idActor, strActor from actors WHERE idActor IN (select idActor from actorlinkmovie WHERE iOrder < 2)
This will create a list with the top 2 billed actors in your movies. We've also removed the "sort by", which really isn't necessary at all and may have contributed to your problem.

Try with this, and if it works well, you could consider increasing the iOrder number to include, say, the top 4 billed actors or so.

[nime5ter]

There are various threads discussing song payload issues with sample queries. I think the wiki page on this links to at least one of those, along with making some other suggestions: http://voxcommando.com/mediawiki/index.php?title=Large_music_libraries

On a side note, if it's not the loading issue that bothers you, but mainly the warning you're getting, you have some control over when that warning message appears in Options > Advanced: http://voxcommando.com/mediawiki/index.php?title=Options#Advanced