Kinect on E3 and the future

[Auri]

Hey guys,

Just dropping in for a quicky
Anyone seen the E3 press release from Microsoft and how they showed the future of XBOX360 and XBOX live ?

Well ... I could not care less for the XBOX360 and XBOX Live but what they showed with Kinect was AMAZING !!
They're are also going to officially launch it for PC and since TODAY the Kinect SDK for Windows is in public beta and available freely from the microsoft site.

Now what I saw in the E3 microsoft demo was controlling your xbox by voice and body movement, again ... CARE about the xbox ... but if they can do it on the XBOX then surely we can do it on the PC and collaborate in great effect with other software such as Vox, XMBC, EG, etc ...

Now I know that Kinemote or whatever the windows hack driver is/was called and that you already played with it james but I remember back then that no1 had the microphones working from Kinect.

Could you tell me what the status currently is with the use of Kinect in collaboration with Vox ?  Have you continued testing more with it James, has the kinect community gotten the microphone working, cause I think it has the potential to become THE microphone for software such as Vox.

I'm just curious since I'm thinking of getting a Kinect and it would be nice to start using vox with it then.
Currently as you all know I don't have much time to spend on my PC lately and my wife could go into labor as we speak, ... but I do use my XBMC daily atm with my Harmony one, and it saddens me everytime that I cannot use vox since I have not yet had the time to properly test and shop for an open-air mic with the needed noise-cancelation to actually function decently.

So a Kinect update from your point of view James would be greatly appreciated and helpfull in my choice to buy a Kinect or not (yet ... )

Hope I can return in full force soon !!

Peace,
Auri

[Wanilton]

Great news Auri, I import mine from USA and receive my Kinect today, wow, big day for me, i installed sdk, and my first test was use microphone with voxcommand, and work very fine, equal use amulet remote, but i need more tests, for one conclusive opinion.

About Kinemot don´t work yet with this sdk from microsoft of course, and installing SDK from microsoft, remove all previous drivers, I hope developers from Kinemot launch new version with full support.

Well, there are only 2 examples in SDK pack install, and work fine...

I am happy for share my experience with others users...

Link for Official WebSite Kinect for Windows SDK
http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/

Download, forum, documentation etc..

Link for download page:
http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/download.aspx

System requirements

Hardware
Kinect for Xbox 360 sensor
Computer with a dual-core, 2.66-GHz or faster processor
Windows 7–compatible graphics card that supports DirectX® 9.0c capabilities
2-GB RAM (4-GB RAM recommended)

Software
Windows 7 (x86 or x64)
Visual Studio 2010 Express (or other 2010 edition)
Microsoft .NET Framework 4.0
For additional requirements for samples, see the Readme file

Installation instructions

To install this SDK beta:
1.On this page, click Download to start the Kinect for Windows SDK beta download.
2.Click Run to start Setup, and follow the instructions in the Setup Wizard.

Or, to save the download on your computer so that you can install it later, click Save.

[jitterjames]

I would not rush out to buy a Kinect at this point if I were you.  I just installed the sdk (which includes an audio driver to be able to use the kinect array microphone in any program, including VC) and in terms of just using the microphone without any special software running (I need to look into this) I am not very impressed.  I am sitting 5-6 feet away and it doesn't seem to understand a lot of simple commands.  As soon as music starts playing it understands nothing!  Maybe I am doing something wrong, but I was hoping to be impressed and so far it's a big let down.

I should say that I did create a new speech profile and I configured it to use the microphone as type "other" which is what we should use for a microphone array.  I did not do any training, but I generally can create a profile and go with other microphones because my accent is pretty standard for US english.  It doesn't seem to mind that I'm Canadian.

I am interested in possibly developing with the sdk in the near future (maybe just sound, or maybe also something with skeleton tracking if I can figure it out) but I have some real work and various summer activities coming up so nothing is going to happen fast in that department.  I certainly will do some more tests and a bit of reading and I'll report back any useful information I dig up.

Could you please post a link to the amazing E3 demo that you were talking about?

[jitterjames]

The sample skeleton tracking app that comes with the sdk works great and is quite fun and amazing.  Totally useless by itself of course, but good for a laugh, and encouraging to see how well it works.  The only other app that comes ready-to-run with the sdk called "sample shape game" crashes as soon as I try to launch it.  (Win7 64bit)

[Wanilton]

sample shape game - here work fine, win7 64 bits too....don´t crash for me...but it´s basic game...

[Auri]

I would not rush out to buy a Kinect at this point if I were you.  I just installed the sdk (which includes an audio driver to be able to use the kinect array microphone in any program, including VC) and in terms of just using the microphone without any special software running (I need to look into this) I am not very impressed.  I am sitting 5-6 feet away and it doesn't seem to understand a lot of simple commands.  As soon as music starts playing it understands nothing!   Maybe I am doing something wrong, but I was hoping to be impressed and so far it's a big let down.

I should say that I did create a new speech profile and I configured it to use the microphone as type "other" which is what we should use for a microphone array.  I did not do any training, but I generally can create a profile and go with other microphones because my accent is pretty standard for US english.  It doesn't seem to mind that I'm Canadian.

I am interested in possibly developing with the sdk in the near future (maybe just sound, or maybe also something with skeleton tracking if I can figure it out) but I have some real work and various summer activities coming up so nothing is going to happen fast in that department.  I certainly will do some more tests and a bit of reading and I'll report back any useful information I dig up.

Could you please post a link to the amazing E3 demo that you were talking about?

Hi james, Hi Wanilton ... (and hi kalle should you read this )

http://youtu.be/77c2S0V4__w This is part one of 5 I think of the E3 Microsoft press release (you can click/find to the next parts I guess) ... try and watch it as a hole as some parts are not so intresting but overall it's an awesome release.

About the SDK honestly I would be ONLY intrested to use kinect as a mic array.  Well ... array ... I was told there was only 1 microphone in the kinect, so you could not really talk about an array.  Perhaps thats the reason why it is so crap James ?

I really hope that kinect will be the solution many of us need to FULLY integrate programs such as Vox, XBMC, EG, etc ... into our dialy lives and routines ...  Fingers crossed.

[jitterjames]

sample shape game - here work fine, win7 64 bits too....don´t crash for me...but it´s basic game...

Wanilton, do you really find the Kinect microphone works "fine" with speech recognition?  I can't even get it to do the training because it has to think long and hard about every sentence.  This is usually a sign of a very bad microphone.  I had similar problems trying to do training with Bluetooth !  I recorded my voice and played it back and it was not too bad sounding, but kind of hollow with a mild tin can echo.  I need to try from different distances and listen with headphones to get a better idea...

@Auri, it does have a 4 microphone array.  This is supposed to ultimately work as a single microphone though.  It uses multiple microphones to do beam forming and noise suppression.  (It is supposed to guess where you are and listen only to sounds coming from that direction.

You can learn more here:
http://en.wikipedia.org/wiki/Microphone_array

I still think that if you are going to have a Kinect hooked up and burning fuel you might as well use skeleton tracking as well to improve the functionality, even if it is just to make motions that turn VC on and off.  I don't know if it is also possible for it to use the skeleton to improve beam forming.

...  But if I can't figure out how to get better performance from the microphone, my Kinect is going back in the box - until someone gives me a 360 

[Auri]

Thanks for the update either way James and Wanilton.

I just pray that you'll adjust some setting somewhere and that it will magicly become an awesome microphone for VC over night

/fingers crossed

Peace,
Auri

[jitterjames]

I had a chance to play with it the Kinect SDK a bit more today.  All hope is not lost!  However, it will certainly not be as simple as "throwing a switch".  It looks like it may be possible to adapt VC to make use of the SDK, gaining access to more specific Kinect audio properties and the acoustic model.  It might also be possible to define a "green zone" by saying "I'm over here" and then only accepting commands from that direction (give or take a few degrees).

Assuming I can figure this out it seems like we may get decent results.  The problem is that I'm not allowed to develop this stuff for a commercial product like VoxCommando.  Eventually Microsoft will release a commercial SDK, but I have no idea what kind of costs would be associated with that or how it would work.

If anyone knows of a way to get around this without breaking any laws I'd be interested to hear your thoughts.  In the mean time I'll experiment when I have time and try to educate myself.  That's the goal of this beta SDK release anyway.

I was hoping that we'd simply be able to use the Kinect device as a microphone without any special code or license, but that doesn't seem to work very well.  If you actually listen to the sound that is recorded it sounds pretty bad.  I'll probably upload some sound samples at some point.

[jitterjames]

Wanilton, Did you have to install visual studio express 2010 to be able to install the kinect sdk?  I am trying to install it on my laptop for testing with the minimum amount of extra software installed, but I get an error when I try to install the SDK.  Some bull about not having access rights to a dll.

[Wanilton]

J,

I Installed Visual Studio 2008 Express Edition, and SDK install fine for me.

Today I desinstall SDK and install drives from Kinemote, and work fine too...in this moment i will test all possibilities...the Kinect is new hardware to me..i need explore a little,
but is fun using kinemote, with XBMC, little stranger in a first moment, but work fine.

W

[jitterjames]

hmmm.  I have Visual Studio 2008 installed aready, but I can't install for some reason.

edit:  pretty much as soon as I start to install the kinect SDK beta I get the following error:

error writing to file:  INuiInstanceHelper.dll. Verify that you have access to that directory.

then whether I hit "retry" or "cancel" I get this error and then it quits

The installer has encountered an unexpected error installing this package.  This may indicate a problem with this package.  The error code is 2350

[Auri]

Hey James,

About the use of the SDK free beta towards commercial software.

What if ... you would code something with the use of the SDK and the lets say .. give it to wanilton who then ... lets say made a"free" plugin for the software he really loves ... lets say Vox Commando ?
Would that be OK for the microsoft licensing model/EULA that is on the SDK ?

Just thinking out loud here.

[jitterjames]

hmmm.  I have Visual Studio 2008 installed aready, but I can't install for some reason.

edit:  pretty much as soon as I start to install the kinect SDK beta I get the following error:

error writing to file:  INuiInstanceHelper.dll. Verify that you have access to that directory.

then whether I hit "retry" or "cancel" I get this error and then it quits

The installer has encountered an unexpected error installing this package.  This may indicate a problem with this package.  The error code is 2350

I found the solution.  I installed .net 4

http://www.microsoft.com/download/en/details.aspx?id=17851

then I was able to install the Kinect SDK with no problems.

[iswasdoes]

Very excited about Kinect developments - the major barrier for me (and I suspect many) for using Vox Commando is the lack of a microphone that can somehow filter out the ambient noise and stop VC from taking commands from it. Very hopeful that this 'green zone' idea could be the answer

VC functioning freely and naturlly, combined with integrated motion control of xbmc, and i'll be some kind of home media jedi.