Kinect Multi Skeleton Bridge
| Author: | hierro |
| Date: | 14 Dec, 2010 |
| Category: | tool |
| Credits: | hierro screamer nando |
Download
Description
Dismissed version, dll updated version here : /sites/default/files/uploads/OpenNIGenericNode.zip
What's included :
1)NITrackerBridge.exe 2)MultiSkeleton (Kinect) help.v4p
What you need :
- OpenNi framework
- PrimeSense sensor driver kinect mod
- NITE middleware 1.3
- vs2010 redist.
look www.openni.org for downloads and forums
Instructions :
- start NITrackerBridge.exe and MultiSkeleton (Kinect) help.v4p
- enjoy
Note: This is just a quick hack to play, PrimeSense and OpenNi will release more "tools" and documentation to do that easier; wrapping the program into a vvvv plugin will be helpful too(less latency and direct control within vvvv).
Comments
Comments are no longer accepted.Please create a new topic in the vvvv beta forum to discuss this contribution.
nutkicking awesomeness!!1 now some DHL dude finally needs to bring my kinect around..
excellent!!! :)
In your balls!!
O M G !
Great ! thx ;)
I ordered...
NiUserTracker.exe wont start and is asking for .NET framework 4.30319, you might want to ad that to the requirements list. thanks a lot for sharing ;)
WOW!!! Can't wait to get hands on my kinect and try this. Cooool!
@sapo, basically you should only need vs2010 redistributable to run that, got them installed ? Otherwise I'll try to add that, I'm not expert at all about visualstudio and c++.
@sapo: what os are you using? probably who use xp needs .net too (we developed in win7 x64, that should have already .net 4 installed)
yes i tested it on xp, don't remember if i installed vs2010 first but i think so,i got w7-64 too but i couldn't find drivers for 64bit, but prolly i was just confused ;)
thanks a lot for sharing that! for the moment couldn't get it to work (black screen, kinect looks to be off). will try deeper later :P
for installation i suggest to check the openni forum, all the info needed are there, here (should) work drivers http://www.bcaa.it/appoggio/Driver.rar
Trying this now.
Always get the message "Calibration failed for user N".
What I'm doing:
Any help, please?
Thanks so much.
super cool,just try it out,make sure to uninstall the clnui dirver b4 using the NiUserTracker. thank you so much!
WoW !! This is super cool ... Will try it in the weekend !
Great job guys.
Nevermind, I was being a fool. It's working and it's AWESOME! Great work!
Yay, works! Use this driver for the sensor https://github.com/avin2/SensorKinect
Oh and the calibration pose is described here: http://www.keyboardmods.com/2010/12/howto-kinect-openninite-skeleton.html
Good stuff!
Will try to get this somehow connected with the Skeleton nodes as soon as possible.
PrimeSense license key asked during NITE 1.3 installation? may i have one?
@aze http://openni.org/?q=node/2
it works, thanks!
there is so many things that can be done with this system that i can't focus on what to do.. one thing.. i'm triing to get the degree of rotation of the bones but i'm not having much luck, i have been triing with decomposer linked to a single bone (position pins) but it outputs the 3 values, xyz rotation, with stange scales and going negative/positive. So, anyone please got some clue on how to get the degree values(0-360) of those bones or the points making the bone? would be cool if the patch could provide this data already for evaluation of gestures and such.
someone tried this on windows xp64 ?
Yes, It works also on xp64... not realtime, because of the OSC bridge. The other NI compiled .exe work like a charm...
mmm, what am i doing wrong ? none of the openni samples work. i installed the amd64 driver which hierro posted or which one is right ? every sample states its not able to enumarate node...damn
ok works now....driver problem it was
skinned mesh animation using this, and Skeleton nodes: http://www.vimeo.com/18008386
@sapo: i calculated joint rotations by assuming some initial pose (in the video above the bind pose of the mesh), and then computing the angles between the joint positions of this initial pose, and the current one.
does anyone know, if there's some default pose for the OpenNI-Skeleton? Or even better, if there's a way to get joint rotations directly from OpenNI?
While in the UserTracker-Sourcecode there's a method SkeletonCapability.getJointPosition, i didn't find anything about rotations. Actually, I didn't find any documentation at all. Any hints?
hi sagishi, really appreciated your work :) Actually the opennI framework stores positions, rotations and confidence values for all those results. When we started sending osc, we just did for a quick playtime in vvvv, also cos it's not that convenient, anyway, actually is a nice crossSoftware solutions for prototyping.
Actually we are already sending rotations and ids over OSC, just we didnt have so much time to fix some buffer issues, to reduce latency, gonna post it soon.
and with a more cleaned helppatch ;)
great, hierro, that's good to hear.
I hope, OpenNI also makes some smart assumptions about the joints' roll-rotations. Otherwise one would have to deal with lots of joint limits and stuff, to preserve realistic poses.
well, what i can say is that we are having pain in the ass to use those rotations into unity, maybe for a lack of experience on matrix and quaternion, and also cos unity is an unclear platform, anyway openNi has a 3x3 matrix respect orientation, and we are assuming that rotation XYZ are on the first column of that matrix
rotationX=matrix<0> rotationY=matrix<3> rotationZ=matrix<6>
anyway is not working clean actually :)
those values seems to be inside {-1,1} interval
intresting stuff guys, nice to see it's not just me having happy times with rotations :) great work sagishi, could you post a simple exampleonhow to extract the anlges maybe only for an arm? else i'll wait the updates from hierro and screeaamer. now i stomped in another problem, if i store a #bundle blablabla in a iobox(string), i unlink it to keep the pose data within the patch and save it, the patch wont open anymore!!! From what i'v read on the forums i guess it's some problem about the strange characters saved in the patch xml. Yea, i could sample the data after the oscdecoders but it was more qick and convenient to do it at the source. can any of you reproduce this unwanted behiavour? should i post it in the forums as bug? cheerss :D
From what we noticed openNI doesn't consider rolling limits, cos it's the raw motion values extraction, by the way, that can be done later to respect every 3d model carachteristics, will release a new sample today, still testing some issues on unity...
a little big update. here it's a zip containing the new version (.exe and helppatch). Some of the new features are:
...is coming this night.. :)
added revison download http://vvvv.org/sites/all/modules/general/pubdlcnt/pubdlcnt.php?file=http://vvvv.org/sites/default/files/uploads/MultiSkeleton2%28Kinect%29.zip&nid=57127
by the way, this bridge was intended for protyping purposes, and is not a specific "best solution" for vvvv,needing are goin further by the way and better performance are always welcome, so waiting for a real programmer (mork calling vux...) to inject those stuff into a plugin for vvvv :)
have fun :P
and today openni released a lot of c# friendly stuff :)
oh nice! i'm on it! i see the rotations are all relative to the chest rotation, what a pain for the task i have in mind(make a robotic arm follow your with arduino and some servos).. but hey at least they are there ;) thanks! i'm noticing something strange btw, if you point an arm to the kinect and rotate the yaw of the camera the arm is vvvvery short, and same for the legs, looks like the depth is not correct or something. the previous version seem a little better but still is too short to be real when the camera is rotated. Great work anyway, the patch layout is also much better now :)
you are right sapo, this is a problem of the help patch. Really the positions comes relative to fraeme for x and y (0 to 640 - 0 to 480) and for the z comes in millimeters of distance between the sensor and the object tracked. But for now i'm doing 2 map (after the cons) that has source values from 0 to 10000 and the destination from -.35 to 1. But it's absolutely arbitrary, if you lower down the 10000 or increase the gap between min and max destination, you should obtain the wanted depth... tomorrow i'll try to find a realistic conversion approach :)
and for now take this file and overwrite the old, should be better ;)
http://www.bcaa.it/ni_skeleton.zip
got it thx! overwritten and now it looks good from any view :D
ok, i reuploaded the ni_skeleton.zip, seems more realistic, just overwrite
www.bcaa.it/ni_skeleton.zip
hi screamer, the ni_skeleton.zip seems to be corrupted
ops, i've uploaded directly the v4p instead of zip. now there's the zip, at the same url
soon a new version :D
here http://vvvv.org/sites/all/modules/general/pubdlcnt/pubdlcnt.php?file=http://vvvv.org/sites/default/files/uploads/vvvv_skeletons3.zip&nid=57127
nice update!
here the plugin, to test and integrate :D
http://vvvv.org/contribution/openni-multiskeleton-plugin
Hello vvvvers!
Congratulations on this forum and this multipurpose tool. I'm still a beginner with programming and vvvv but ever since I saw skyence on motionographer and then watched the video tutorials by west I was hooked. The possibilities of making things with 4v have me dreaming a lot. :) I've watched a lot of videos on the vvvv vimeo group also. Great and inspiring stuff there too. I've been trying to consume as much information as possible on the tecnical side of computer graphics and the programming behind it (which in my noob view vvvv does really awesome stuff there while also letting you integrate different things). I'm looking forward to overcoming this huge learning curve that I have to conquer.
I've read up on this kinect skeleton thread and have attempted to get the skeleton patch to work. I've followed the first instructions posted by hierro, screamer and nando.
I have:
I also downloaded the Driver from hierro's post on 15/12 via the http://bcaa.it/appoggio/Driver.rar link. I then went on to the SensorKinect github site with the detailed instructions that DieTapete posted on 15/12.
On the SensorKinect github site I followed the install notes. I got:
Had to get:
From the nsis-2.46-strlen_8192.zip I unpacked the Stubs folder and makensis.exe file into the NSIS directory and overwrote the files in there (as instructed in the Build Notes section).
I'm not sure what the build notes are (I'm new to programming jargon) but I went ahead and installed the other applications specified under the windows requirements section. Those being:
For the OpenNI v1.x requirement the SensorKinect github link takes us to the OpenNI homepage so I searched their site for "OpenNI v1" . The search result is confusing to me and I am unsure what to install. The most best result I think was:
I didn't install any of the optional requeriments.
THIS PART I DON'T UNDERSTAND: Building Sensor:
The visual studio solution is located in: Platform\Win32\Build\EngineWin32.sln.
Important: Please note that even though the directory is called Win32, you can also use it to compile it for 64-bit targets (AMD64/x64).
I tried to run the NiViewer sample to make sure depth & image streams are working and I got an error: "This application has requested the Runtime to terminate it in an unusual way. Contact support for more info." When I had version OpenNI-Bin-Win32-v1.0.0.25 installed this didn't happen.
I'm really lost. I think I installed things not in the proper order.
More info:
I wasn't asked for a PrimeSense license at anytime.
Before I installed OpenNI v1.0.0.23 when I ran NiViewer I would get the same error as u7angel: "One or more of the following nodes could not be enumerated. Press any key to continue."
When I tried running NiUserTracker.exe in "C:\Program Files (x86)\OpenNI\Samples\Bin\Release" folder (like theplastictoy's post says) I got an error: Could not load file or assembly 'OpenNI.net, Version=1.0.0.25, Culture=neutral, PublicToken=6b43d0c6cf74ee7f' or one of its dependencies. The system could not find the specified file. After installing OpenNI v1.0.023 I only get "This application has requested the Runtime to terminate it in an unusual way. Contact support for more info."
My C:\WINDOWS\Microsoft.NET\Framework directory has a v4.0.30319 folder. (I Googled "how to know what .net framework you have installed") (this refers to sapo's post on 14/12/2010)
"If the driver is installed correctly you should see "Kinect Camera" and "Kinect Motor" under the "PrimeSensor" category in the device manager." (from the tips section in the KinectSensor github site) This is I have and it seems OK.
I'm working on a Windows XP, Professional Version 2002 SP3 system on a Intel Core 2 Duo, T8100 2.10Ghz, 3GB laptop.
MultiSkeleton node shows up red in the vvvv patch.
I have a vcredist_x86 file in my kinect skeleton folder where I have all the installers for everything. It installs a Visual C++ 2010 x86 Redistributable Maintenance. I can't remember where and why I have this. I think is has to do with hierro's post on 15/12/2010 - 01:32.
Is the driver installer the dpinstx86 file?
My Kinect has a green blinking light.
What a mess! :) Any help?
Thanks so much.
Leo
OK now I understand what needed to be done in the "Platform\Win32\CreateRedist" part. A programmer friend helped. :) The only problem I had was that the Visual Studio version I had wasn't a correct one for this case. So I installed the VS2008 full.
Now the sample UserTracker is working, it recognizes a player, asks for the calibration pose, calibrates and creates a tracked skeleton.
Now the only problem holding me back is the NITrackerBridge. I'm not sure if I am running it from the correct directory. The error I get is "InitFromXml failed: File not found!"
I downloaded vvvv45_beta25.1 and tried out the kinect skeleton patches. I don't understand a lot that is going on in the patches but I still have red skeleton nodes.
The files: Sample-User.xml Sample-Sceme.xml Sample-Tracking.xml seem to be ok. They have the license and the 640x480 in them.
From the forum http://groups.google.com/group/openni-dev/browse_thread/thread/25eb6eeaff7d6b5e I read that: All samples are looking for the XML file in a specified relative path. Make sure you run the samples from their Bin dir.
How do I check what specified relative path xml the samples are looking for? What is the correct Bin dir for the samples?
Thanks!
Leo
This stuff is a bit tricky to install... pffft!
try use that latest plugin http://vvvv.org/contribution/openni-multiskeleton-plugin insead of this
hm, doesn't work for me... I'm guessing something with Visual Studio... running the "Redist.py" doesn't produce any exe's.
whatt? that version doesnt require any exe!
latest version @ orbbec-astra-openni