Author Topic: Windows 10 Cortana Integration (Read 10428 times)

xzener · « **on:** April 28, 2015, 12:09:30 AM »

Hello JitterJames,

Cortana is great on the Windows Phone... Rumer has it, Windows 10 will have Cortana built in. This would be the perfect addition to Vox Commando. Ive seen videos of Dom Controller, even with an asian guy with "not so good" engrish. Cortana seems to responds very well to his voice.

http://www.domcontroller.com/en/

Any plans for Cortana integration with Vox Commando??

Xzener

xzener · « **Reply #1 on:** May 04, 2015, 05:15:51 AM »

Or at least VoxWav for Windows Phones perhaps?

jitterjames · « **Reply #2 on:** May 04, 2015, 08:16:55 AM »

Perhaps. It's a pretty small demographic.

But can't you run VoxCommando directly on a Windows 10 phone?

xzener · « **Reply #3 on:** May 04, 2015, 11:53:53 PM »

Why would I want to run VC on a phone? I wouldn't mind using Cortana to voice commands.

jitterjames · « **Reply #4 on:** May 05, 2015, 11:02:13 AM »

Why wouldn't you?

According to our FAQ, VC will make you popular with the ladies. Now imagine what would happen if you had it running on your W10 phone... You'll have to beat them off with a stick.

RickyD333 · « **Reply #5 on:** June 06, 2015, 03:11:35 PM »

Funny enough, there are women that WANT to come to my place to see my Voxcommando home automation. LOL, thanks James!

I found this video today. It's from last year, but has some nice information on Cortana integration for developers. I really think integrating Cortana into Voxcommando would be really fantastic. The fun developer information really hits at around 13min in the video.

http://channel9.msdn.com/Blogs/Dev-Inspiration/What-Developers-Need-To-Know-About-the-Cortana-API

And wait... VC can run on Windows Phone? Cool! If I can find some decent apps that let me make my own phone UI (like Nova Launcher and UCCW on Android) I might switch over.

jitterjames · « **Reply #6 on:** June 06, 2015, 03:18:50 PM »

Quote from: RickyD333 on June 06, 2015, 03:11:35 PM

And wait... VC can run on Windows Phone? Cool! If I can find some decent apps that let me make my own phone UI (like Nova Launcher and UCCW on Android) I might switch over.

I really don't know if it can run on a Windows 10 phone or not. I know it can run on some tablets but they are running a full Windows OS.

I don't have a Windows phone and don't have much interest in getting one. I'm still OK with my 4 year old Android phone that Kalle gave me.

sniperkiller · « **Reply #7 on:** August 01, 2015, 08:01:01 PM »

I wanted to change cortana name and add some extra scripts to its workings, so I found several things about programming using cortana API, which could be really usefull for you, this is for pc, not for mobile

http://talkitbr.com/2015/07/13/integrando-a-cortana-em-seu-aplicativo-windows-10/

http://stackoverflow.com/questions/30431688/how-to-connect-cortana-commands-to-custom-scripts

http://stackoverflow.com/questions/27670696/how-to-modify-cortana-voice-activation-commands-xml-in-c

lenovo already made a "plugin" for cortana
http://www.getreachit.com/

I am a programmer and I am thinking about creating some simple program that can run a script alike to TellVoxCommando.exe but I would asume you would be far better at doing this than me =P

jitterjames · « **Reply #8 on:** August 03, 2015, 06:29:15 PM »

I think you should go for it.

Sending commands to VC is very simple. Let me know if you need help with that part.

For my part, I guess I need someone to explain to me how they actually see this working and what the benefits would be.

If you want to use Cortana, why not just run both side by side and use a prefix for VC?

Haddood · « **Reply #9 on:** August 03, 2015, 08:38:25 PM »

Scenario 1:
Cortana replaces MS VR engine... so Cortana will do the VR and send the text to VC ... it even has a prefix mode which is "Hey Cortana" ... for always on scenario
this way alot of the commands we did ...will be readily available ... like alarms, reminders, time date ...etc. and VC will be mainly an automation engine with interface to introduce commands to Cortana ...

sin a way Cortana to VC will be what VC used to be for EG

Scenario 2:
VC calls function programed in Cortana ... like when we say "VC remind me to call my dad in 5 minutes" it will send the text to Cortana ... so basically Cortana will be an extension to VC ...
this will require building all cortana commands in VC ... maybe read Cortana xml for built in commands (if there is one)

Scenario 3:
I do not know if Cortana needs internet to function like Siri (if it uses massive computing power on a server to improve the accuracy), or it depends on MS VR in windows ... if it uses MS VR ... then maybe all it need is simple communication: finding a way that put Cortana to listen the moment VC start listening ... ex. after prefix.start event ... then if Cortana recognize the command, it generate an event in VC. with info about it ... so VC can give feedback ...
like this each one of VC and Cortana handles its own commands

jitterjames · « **Reply #10 on:** August 04, 2015, 07:16:51 PM »

I've just been playing around with Cortana a bit since my free upgrade to Windows 10 came through on my HTPC.

Sadly it seems that running Cortana kills Windows speech recognition completely, so I can't use VC at the same time. The only solution I have found is to disable Cortana and then to do a full restart. Sometimes it works, but I think only if VC is already running first or something. Anyway, for now it is totally unusable on my system so I have disabled Cortana.

I found a lot of other problems with Cortana anyway. Hopefully they will improve it over time.
* You need to set your OS location to the USA for it to work at all.
* You need to sign into Windows using a MS account (*yuck*) and not a local account for it to work at all.
* It only works in English.
* It uses your IP for your location which you can't override and in my case the location is completely wrong (by about 700 km).

Cortana worked extremely well for setting up a reminder, and the recognition accuracy for doing a search using open dictation seems very good. I could not get it to do anything else really, it just kept opening up a Bing search page for everything I said. Maybe I didn't read the instructions properly.

sniperkiller · « **Reply #11 on:** August 04, 2015, 08:38:03 PM »

I found this also:

Cortana + AutoVoice - Voice Automation from your PC:

http://joaoapps.com/autovoice/cortana/

I live in Colombia and have cortana configured in Spanish, it works perfectly and can even play any song or video by title, artist or playlist, can launch any program (including batch scripts that I create) but you have to say "open" for it to work so saying "open shutdown" is not pretty XD

and with the autovoice option you can do anything now, but it requires an android device to be the "brain", here is how it works:

1. you tell cortana a command that she won't recognize by default
2. cortana searches that command in google chrome (you have to set chrome as default)
3. autoremote intercepts the search and send it to your android
4. in android you receive the message by autoremote, it is sent to autovoice
5. autovoice triggers depending on the command
6. tasker get the trigger
7. you can do anything in tasker
8. ? ? ? ?
9. Done!

in tasker you can send commands to any home automation device, or can send a command to eventghost (which can then send a command do VC)

Still VC is more practical, potent, customizable and have more plugins and integrations so I would rather keep using VC than Cortana, but cortana uses less resources and have a lot of things by default that would take me some time to implement on VC like launching all apps or all songs by tittle, artist or playlist, I have this already on VC but my DB is huge and VC can easily consume about 300MB of ram =S

marcusvdt · « **Reply #12 on:** August 12, 2015, 10:12:02 AM »

By reading this thread, I don't see a reason to use Cortana.
300MB of RAM for what VC can do... I think it's nothing nowadays.

All the examples that you guys mentioned add complexity for the automation engine, and I'm not seeing the real gain with it. Is the VR in Cortana so better than the one in VC that this alone makes it a big advantage over VC?
VC can understand me very well with the correct setting and I'm using VC SP.

jitterjames · « **Reply #13 on:** August 12, 2015, 11:08:01 AM »

For doing predetermined commands where all possible phrases and payloads are known there is no advantage, but for doing open dictation, web searches etc. it is very good. Cortana is cloud based which means that it can analyze your audio and compare it to Bing results in order to make a better guess. It is similar to how Google SR works.

Bing is also good at doing certain things like web searches, texting and calling people and creating reminders etc, but I don't really see how this relates to VC. It seems like it should just be run beside VC if you want to use it for this.

I don't believe there is any way for VC to talk to Cortana, only for Cortana to talk to VC. There is no way to use the Cortana TTS voice either as far as I can tell, outside of Cortana itself, so unless Cortana is the one recognizing something there is no way to make her speak.

RickyD333 · « **Reply #14 on:** August 12, 2015, 12:12:06 PM »

I just posted this for Cortana web searches.

http://voxcommando.com/forum/index.php?topic=2311.0

Like James said, VR engines like Cortana are great for open dictation and web searches. There is a feature in Cortana that allows you to search for files on your system, however I can't seem to 'exactly' find them with voice search -- works great with typing in a file name though. I jumped on the Microsoft community page and asked if this was a bug, if I'm doing it wrong, or if Cortana can't yet do this with voice search. Some Microsoft dude sent me a reply, but for some reason I don't think it will solve this issue. Hopefully we can eventually use Cortana voice search for finding files.

marcusvdt · « **Reply #15 on:** August 12, 2015, 12:48:06 PM »

Ok, I see an advantage then, open dictation for web searches since VC SP does not allow for open dictation. But wait, I think Cortana is not available in Portuguese too. So I agree this is big advantage and I really would like to have this working. But I still think it is weird to have to talk to two different virtual "assistants" for different requests.

Unfortunately I think my only option is to try to find the most trustable way to integrate Chrome's Ok Google with VC, right?
Btw, integrating Ok Google with VC was already on my list so I can allow for some open dictation in portuguese. The Ok Google works preety good in my cell phone, so I'm wondering I can have something similar integrating it with VC.
I wonder I can send my voice (recorded audio) to Google's engine so it can recognize it. I think there are some examples in the forum about it... I'll check all of this once I finally start trying to actually do that.
One thing that would be preety cool is to not have to pre determine what I can search in my XBMC library. I really would like to be able to perform a real open search on my library without necessarily using the payload xmls. The reason is that I have a DLNA media server running on another PC and Kodi does not add remote items to its media library.
Maybe I can do such media searches once I can find a way to have Ok Google to return back to VC what I have said to it. Anyway it's another subject.

For the thing of searching files on local PC, I don't think I would have a use for it. I'm wondering a full automated platform where I should not have a reason to search for a file at normal day to day use. Suppose I could search for a file in my HTPC, what could I do with it then?

This is only my point of view and taking my own needs into account, and maybe my imagination is not following what you guys are seeing because simple lack of imagination. I'm not trying to be a smart ass, but maybe kindly playing the devil's advocate role so I can perhaps get the whole idea.

Thanks!

jitterjames · « **Reply #16 on:** August 12, 2015, 01:30:24 PM »

I have no idea if this site can be trusted or not, but they claim it is coming...

http://www.techtudo.com.br/noticias/noticia/2015/07/cortana-vai-ganhar-versao-em-portugues-ate-o-fim-de-2015-no-windows-10.html

jitterjames · « **Reply #17 on:** August 12, 2015, 01:33:25 PM »

Quote from: marcusvdt on August 12, 2015, 12:48:06 PM

This is only my point of view and taking my own needs into account, and maybe my imagination is not following what you guys are seeing because simple lack of imagination. I'm not trying to be a smart ass, but maybe kindly playing the devil's advocate role so I can perhaps get the whole idea.

Home automation is just one reason for using VC. Some people use it for gaming, some might use it as a productivity tool or because they are disabled or have trouble typing or using a mouse for some reason. I have often used it to solve problems that would otherwise require me to perform repetitive tasks, and using VC was much faster than trying to write a program from scratch.

So if you use VC on your work computer you might want to use it to find files. On the other hand, I think many people try to solve problems with VC just to see if they can, and then once they solve the problem, they don't actually use the solution because it's still easier to do it manually. There is nothing wrong with this... just saying it happens.

RickyD333 · « **Reply #18 on:** August 12, 2015, 02:18:06 PM »

Quote from: marcusvdt on August 12, 2015, 12:48:06 PM

For the thing of searching files on local PC, I don't think I would have a use for it.

Well for someone like me I occasionally use Photoshop for little project's and do some writing (especially in my college days... had about 6 different essays I was working on per week). So if I am sitting down to work on one of those project's I can just ask Cortana (at least that's what I would like to do) to open up the particular file I need (or maybe if I forgot where a file is, it can show me the file location... maybe I want to email it/upload it to Google Drive and I forgot where I put the file).

For me, I'm basically trying to create the Star Trek experience because it's what I've dreamed of since I was a wee dork-ling. Now I just need my holodeck... Computer, play simulation Dixon Hill...

marcusvdt · « **Reply #19 on:** August 12, 2015, 03:37:39 PM »

Thanks for the responses.
Yes, it's lack of imagination on my part. This is because I tend to think majority of other things that I do in my life are easier or faster to do without VC. I'm so focused in making VC to perfectly fit my immediate needs as I initially wondered, that I have no room in the brain to match it to other tasks, like work for example.

Sometime in the future I'll probably want to expand VC usage to other areas of my life, or even creating something for disabled people. The last is actually a very good idea.

So taking into this boat, I think I would try to have both Cortana and VC (which I currently call Doris) hearing me all the time. If that could be possible somehow, I could then ask the tasks respectively to Cortana and Doris according to what is easier for each of them to execute. For example (just thinking, not sure what is actually possible):
Cortana, search web for the future is here.
Doris, read search results.
Doris, new email attach search result 1.
Cortana, someoneone@gmail.com
Doris send the email.
Doris, turn all the lights on.
Cortana, search hard drive for photos named Orlando 2015.
Doris, slideshow current photo search.

etc...

RickyD333 · « **Reply #20 on:** August 12, 2015, 03:50:35 PM »

You don't have to switch between saying "Doris" and "Cortana". You can just stick with "Doris." You can actually just use the xml I linked to earlier and adjust the phrases to your liking. If you go into the Cortana settings (click in the Cortana search box, select the notebook icon on the left, then select settings), you can turn on and off the "Hey, Cortana" feature. I have mine set to off because I use Voxcommando to activate Cortana.

jitterjames · « **Reply #21 on:** August 12, 2015, 04:23:14 PM »

Quote from: RickyD333 on August 12, 2015, 03:50:35 PM

You don't have to switch between saying "Doris" and "Cortana". You can just stick with "Doris."

I think you are missing his point. When he says "Doris" he is talking to VC and when he says "Cortana" he is talking to Cortana.

RickyD333 · « **Reply #22 on:** August 12, 2015, 04:53:09 PM »

Oh right, I had to think about that for a moment. My bad. If I was going to use Cortana for things like that I would have certain phrases for each of those occasions. For example, "computer search" brings up Cortana and my intention is to perform an internet search and then "computer search system" would perform the same action, but I would instead be intending to search my computer files. I would just do that to make things consistent.