Multiple Kinect devices support (by adding kinect ID option)

[Tebura]

Hi,

I recently found that using Kinect for speech recognition works for my test setup way way better than any of the microphones I tried so far! As I am thinking of trying a multiroom setup, I was wondering if it would be possible to support multiple Kinect devices?

I am thinking of a really trivial modification: just making VoxCommando pass Kinect device ID during Kinect library initialization. May not even have to be exposed in the GUI and remain in the config file, as a less commonly used feature.

Would it be possible to add this?
Michal

[jitterjames]

Are you using the streaming feature in VoxCommando's options menu or just using the Kinect as a Windows microphone?

Are you talking about using two microphones at once or switching between them?

[Tebura]

I'm sorry for not making that clear. Yes, I am using the Kinect streaming feature of VoxCommandoSP. And I meant just being able to simply start two independent instances of VoxCommandoSP (from different folders, with different command sets) on the same machine.

[Tebura]

OK, let me throw a more specific question, since perhaps the problem can be solved another way.

Is there any difference between using the Kinect streaming mode of VoxCommandoSP and grabbing audio through Kinect API with the same settings (automatic beam, noise suppression etc.) and outputting it to a virtual recording device (e.g. with VAC) to which then VoxCommandoSP listens?*

So the question is whether VoxCommando explicitly grabs the Kinect-processed audio stream that later goes through the usual speech recognition pipeline or does it rely on the Kinect API to return recognized words?

*I have actually tried that, but I am not getting recognition rates as good as when using Kinect streaming mode of VoxCommandoSP

[jitterjames]

I don't think it should make any difference. Kinect is not used for recognition AFAIK but who knows what happens behind the scenes.  Maybe the difference is because your two Kinects are not the same quality or maybe VAC is modifying something. Even if it is just a change in volume this can have a profound effect.

[jitterjames]

I'm sorry for not making that clear. Yes, I am using the Kinect streaming feature of VoxCommandoSP. And I meant just being able to simply start two independent instances of VoxCommandoSP (from different folders, with different command sets) on the same machine.

I only have 1 kinect so I'll need your help with testing.

[Tebura]

Naturally, I'm more then happy to test if it works with 2 Kinects. If you were adding KinectID as a parameter for the configuration file, I would recommend also adding EchoCancellationSpeakerIndex along with it, as that would become important in a multiroom setup.

And thanks for the comments. I'm checking the code for where things could be different. Can I ask which EchoCancellationMode is VoxCommando using when AEC is on ( https://msdn.microsoft.com/en-us/library/microsoft.kinect.echocancellationmode.aspx )? I have hardcoded it to "CancellationOnly"perhaps that makes some difference?

In any case, I am testing it on exactly the same Kinect device with the same volume levels and my program is just pushing out the audio samples it receives from Kinect object.

[jitterjames]

I'm using CancellationAndSuppression

[Tebura]

Thanks. That could be it. I will test a bit more and if I can confirm that there is no more difference in recognition rate, I'll post my simple app. This would effectively solve the issue of supporting more than 1 Kinect and add some extra flexibility. E.g., having an external app outputting Kinect-processed audio stream I can have two instances of VoxCommandoSP, each performing recognition in a different language (English for me, Japanese for my wife).

[Haddood]

Guys
I have 2 Kinects if that will help in testing

[Tebura]

Thanks Haddood, would you then check how the little attached app I made works for you?

The app simply outputs audio stream that comes out after processing with Kinect library algorithms. So it should be the same stream that is passed to voice recognition in Kinect Streaming mode of VoxCommando.

To use it, you will need to have Virtual Audio Cable/VB-Audio Virtual Cable installed to create a virtual output device that loopbacks to a virtual microphone. Then you need to put the virtual output device index (as OutputDeviceID field) in the config file and that is pretty much it. Just use VoxCommandoSP without KinectStreaming and make it listen to the virtual microphone.

You will notice a few other options in the config file to reproduce the same functionality as present in VoxCommando streaming tab, as well as KinectID field to support more than 1 Kinect.

https://drive.google.com/open?id=0B0PpnlX_bqmwRHV3bUhrQnBuNzg

I would be interested in knowing whether the app indeed fully reproduces VoxCommandoSP Kinect mode functionality and we are getting the same recognition rates.

[jitterjames]

There is a new version of VC on the download page.  I have not announced it yet but it includes some updates to the Kinect streaming.

http://voxcommando.com/mediawiki/index.php?title=ChangeLog#Version_2.2.1.6

If you get a chance please try it out and let me know.

[Tebura]

Thanks, I'll take it for a spin!

Also, sorry, seems I have pasted a private Google Drive link to the KinectMic app in the previous post, here's the correct one:

https://drive.google.com/file/d/0B0PpnlX_bqmwRHV3bUhrQnBuNzg/view?usp=sharing

[Haddood]

Thanks, I'll take it for a spin!

Also, sorry, seems I have pasted a private Google Drive link to the KinectMic app in the previous post, here's the correct one:

https://drive.google.com/file/d/0B0PpnlX_bqmwRHV3bUhrQnBuNzg/view?usp=sharing

tried to download it and Microsoft edge said it has a virus and was deleted!!!
never mind ...downloaded it with chrome will test and post my findings

[Tebura]

Yeah, it's an unsigned exe which is not in any database of commonly downloaded files, so it could trigger some warnings. Actually just trying to get the most recent beta of VoxCommando makes Edge block the download.