Kinect

[jitterjames]

Hi,

Not sure if you can do anything with this.

http://groups.google.com/group/openni-dev/browse_thread/thread/e5f3e861c812b1a0

The read-me suggests that Audio streams are now available - what ever that means.
https://github.com/PrimeSense/Sensor/tree/unstable

Best wishes.

OOooff.  Stay away from those newer drivers.  I tried installing them.  Not only did I not get audio, I broke my connection to the kinect.  After uninstalling everything, rebooting, then reinstalling the original drivers that did work, I am still unable to connect.  At this point I am not sure what to do other than move the kinect to another computer in my office.  what a pita.
 

[jitterjames]

phew.  I got it working again.  In case it helps someone I will tell you how I fixed it.

As I said, a full uninstall of all components and reboot, followed by reinstalling as  I did originally did not work.

Then I tried using system restore to bring me back to the point where the Kinect was working.  This did not work either.

Then I did a system restored to a point at Jan. 1st, the day before I bought the kinect.  Then I reinstalled everything, and it works.

I guess I better not break it again, because in the future I won't want to restore back to Jan 1st!

[Foggy]

Whoops - Sorry about that.

I just looked at a couple of the links I previously sent you.

http://groups.google.com/group/openni-dev/browse_thread/thread/e5f3e861c812b1a0/c016a7c14ec8ec24?show_docid=c016a7c14ec8ec24

Perhaps in future it is worth leaving new drivers for a few days to see how they pan out for others.


Best wishes.

[jitterjames]

yes, probably a good idea.

So here is a little progress report:

First of all I should point out that I don't really understand how most of this stuff works.  For example, I have looked at the code for KinEmote and I have no clue how I would do something that used two hands, assuming that is even possible, but I would think it should be.

I am able to understand enough of the higher level code though, to be able to tap into all the events that KinEmote is generated, and I can read the position of the hand in the x/y grid (0,0) to (6,6).

So for example I can:
- assign an event to mid plane, hand at top left, and use it to turn VC on.
- assign an event to mid play, hand at bottom left, and use it to turn VC off.

this works.

or if I wanted I could simple use:
- hand moving into mid plane (i.e. forward) as VC.On
- hand moving into back plane (i.e. forward) as VC.Off

it should be no problem to send commands from VC to Kinemote as well, though I'm not sure what would make sense here.  I could do "connect" and "disconnect", though I'm not so sure how useful that would be.

It's a real brain teaser that's for sure.  I think if it's possible to do two-handed gestures that would be more useful.  As it is, any kind of movement can lead to trouble...

[Foggy]

Hi,

OpenNI have a new web site

http://www.openni.org/

The Videos in the Gallery, particularly The Warcraft one is rather cool.

Best wishes.

[Foggy]

Hi,

Well I'm glad you are getting some where.

The reason I thought Connect & Disconnect might be useful was.

Connect - Your tutorials show how you can launch a program - but with KinEmote you have to also click the Connect button before it becomes active, I was not sure how to do that.

Disconnect - Only because if you get up and walk away or accidentally do some thing with your hands - you may have unintended consequences.

Best wishes.

[jitterjames]

whoa.  your name is "Foggy" ?

all this time I thought it was "Froggy" !

I think I've been looking at these bright screens for too long.  I'm not sure which I like better...

It would be awesome if the Kinect could read sign language!  Just thinking about the heart shape on the open ni website...

If I have a good day tomorrow I might shoot some crappy video just to give you an idea of what is going on...

[Foggy]

Hi,

Not to worry 

The name comes from Carl Fogarty (My hero) - The great British - World Super Bike champion. I just wish I could ride a motor bike like him.

I liked the comments in the Warcraft video about getting kids fit - rather than having them sitting at a screen & controller all day.

With regard to the 2 hands - This may help you
http://groups.google.com/group/openni-dev/browse_thread/thread/7cb51f1fa3240111

Just turn down the volume if you watch the video 

Best wishes.

[jitterjames]

Hey, just a little update on my progress with Kinect and VoxCommando...

I've been talking with Johnny from KinEmote about a unified OSD that will display info from both KinEmote and VoxCommando

Here's an ugly demo of a "proof of concept".  Note that the osd does not need to be running on the same machine as either KinEmote or VoxCommando, but it does (currently) need to be running on windows.  The code for the OSD is really simple so people could probably come up with their own versions for a different look, or hopefully someone could adapt the code to linux etc.

http://dl.dropbox.com/u/514879/video/MVI_0523.mp4

[painy]

Cool developments! To bad i don't have the budget (as a student) available to buy a kinect.
My prediction is that we'll probably have to wait for similar camera's with a higher resolution before it'll become really practical, so it can detect hand and finger gestures (giving your tv the finger would shut it down or something :p). Or is the hardware capable of doing that already?

Nice to see vc is evolving! And what an awesome donation foggy

[Foggy]

Cool indeed.

It is not just the gestures that might be helpful. I am hoping that when Audio drivers become available we can use the built in microphones.

What I am hoping is that as the sound (XBMC) is going through the windows box any way and could possibly be captured by the kinect internally - it may be possible for the sound chip to effectively noise cancel. Thus a perfect microphone for VoxCommando.

Best wishes.

[jitterjames]

This sounds good in theory.  On my audio chipset I have something called "accoustic echo cancellation" that tries to do something similar, since it knows what is going out the line out and what is coming in the mic jack.  One of the problems is, that although the microphone may know what is being output by the sound card, it does not know a lot of things about what happens after it leaves the computer.  For example, what kind of output delay and volume will result after it passes through your amp and speakers, not to mention the acoustic characteristics of your room, and how sound bounces around it before coming back to the microphone.  I have actually seen vox pickup sounds from the computer even though the sound on the TV was muted, because it was subtracting the sound from nothing, and ending up with a negative something, which in the world of sound is not much different from the original something!

In any case, I have a strong feeling that even if they manage to get the microphone working, it will probably be with no support for this feature.  The only time this type of noise subtraction really works (that I have seen so far) is in the case of a headset with two mics, one pointed at your mouth and one pointed the other way.

On the other hand, the 4 microphone array could potentially be effective enough as it allows the microphone to focus in on a single source of sound.  Hopefully this will be you!  I am assuming that the width of the Kinect implies that it will be able to do this more effectively than the ps3eye does.  Also, for some reason the overall volume level on the ps3eye is very low, so hopefully Kinect will be better in this respect as well.

I personally feel that the only practical way to use the kinect gestures will be with a "wave to speak" concept, although possibly we could do things like say: "adjust volume" and then slide our hand up and down... maybe.  It's good excercise anyway, my arm was getting tired during testing...  

Edit:  I really shouldn't be so pessimistic.  Maybe it'll work exactly the way we hope.  You never know!

[iswasdoes]

Ive been playing with KinEmote and VC for a while now and I definitely think there are some cool ways they can work together.

From VC's perspective, it will be about switching the program that the kinect sends commands to, or the type of action that you want to do, as long as it makes sense to have a gesture.

For example, fast forward and rewind in a video could have an easy 'scroll' gesture. And the volume adjust up down gesture you suggest would also be good.

But for example, I have a program on my computer called qzoom that allows you to zoom in on whatever is on screen by pressing control and adjusting the scroll wheel (good for browsing the web on tv). It would be great if I could say 'Zoom' (or some other command if that conflicts) and VC could switch KinEmote to direct gestures to that program. Then a nice forward/backward gesture could be used to zoom in and out (or a pinch to zoom, if multitouch becomes available)

I think that VC is already customisable enough to do this (I think, though I have no idea how to do this yet), so it would be a matter of KinEmote being able to remember different commands for different programs and responding to VC's requests to switch while staying in the background.

Can VC send commands to different programs in the background? Eg, if I was playing a game and listening to music though iTunes, would commands meant for iTunes be picked up even though that was not the active window?

Edit - Ive started a thread on the KinEmote forum regarding the idea above if anyone is interested http://www.kinemote.net/viewtopic.php?f=2&t=18

[jitterjames]

Can VC send commands to different programs in the background? Eg, if I was playing a game and listening to music though iTunes, would commands meant for iTunes be picked up even though that was not the active window?

Edit - Ive started a thread on the KinEmote forum regarding the idea above if anyone is interested http://www.kinemote.net/viewtopic.php?f=2&t=18

It depends on the program.  Most commands like the ones sent to iTunes, MediaMonkey and XBMC are sent correctly regardless of what is in the foreground.

Some commands use keyboard emulation which is sent to whatever has focus.  Currently the scroll commands in xbmc are using keyboard emulation but they could be changed to use the http api or xbmc's event client/server.

It is possible to send commands directly to KinEmote through tcp or udp network commands, but KinEmote would need to know what to do with them...
It is also possible to send commands the other way, from KinEmote to VC, which is how I'm turning VC on and Off in the demo.

[jitterjames]

just a little Kinect news... or rumour really, that microsoft might support the use of Kinect on windows at some point.

http://news.cnet.com/8301-10805_3-20027872-75.html