Kinect v2 HDFACE
| Author: | screamer |
| Date: | 12 Feb, 2015 |
| Category: | demo |
| Credits: | for work this patch needs dx11 a kinect2 packs installed, and to make it work, take the nuidatabase folder (that should be something like "C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409\bin\NuiDatabase" and you have to copy the entire folder to your kinect2 pack folder |
Download
64bit
demo for new HDFaceBuffer node
kinect v2 face tracking example
Description
Oh, and if you press spacebar you can save the actual face, and use it as a texture for the facemodel (so for example you can give your face to someone)
Comments
Comments are no longer accepted.Please create a new topic in the vvvv beta forum to discuss this contribution.
also, you will need (DX11.Texture2dArray) contribution: SetSlice (DX11.TextureArray) (dx11)
Hey, I guess HDface node is missing (inside Face patch). or might be my problem.
HDface is missing =(
HDface missing!!
unfortunately i cannot try it now, but it worked for me. Have you copied the NuiDatabase folder from the sdk path to the kinect 2 pack, as written in the credits of the contribution?
....for work this patch needs dx11 a kinect2 packs installed, and to make it work, take the nuidatabase folder (that should be something like "C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409\bin\NuiDatabase" and you have to copy the entire folder to your kinect2 pack folder
let me know, anyway i'm planning to try it out with the new version of vvvv and kinect2 pack as soon as i can
in my case i have a red split node?!
thx and greetz
christian
I understand, wich vvvv and kinect2 nodes are you using? I'm up to date :)
The split node should be in DX11.Texture2dArray contribution, is linked in the first comment.
hi esnho,
the split node seems to be a complete (split.v4p) patch what ist not in the link.
i am using a new kinect2 and last vvvv version.
greetz
christian
yep, sorry, my mistake. split is in the mremod2 contributuion (but not really needed for the face to work)
Thanks @screamer, and the HDFace node? Where we can find it? Is the HDFaceBuffer node? How it should be patched?
having the same problem as @esnho , did you get it to work?
i'm sorry, but something is changed from when i've done it, and now i'm in your same situation, it dosen't seems to work. if i'll find a solution i'll let you know
So, i've understood how the hdfacebuffer works. To work it needs that you connect the "user index" pin from skeleton node, to "Tracking id" pin on hdfacebuffer node. In that way, you can see the values of hdfacebuffer moving, so the head is readed. But here is where i'm stopped. Because don't know how to use the buffers sent out from hdfacebuffer, that are:
the values from here are all what we need to create the mesh of the head, but who know how? hope that vux, or someone else with this knowledge can help us understand ;)
Ooh nice vux might have released the multi version of hdface he made for us. In that case in your shader you have to read vertex position and uv's from those buffers via SV_VertexID coming from the geometry.
hey microdee, do you have an example? i'm really stoppe here, and my knowledge about shaders is not enough :)
Unfortunately i might only have an older version of that. I don't really have time to make an example though I'm flooded with work :(
added a basic helppatch for HDFaceBuffer. still the tracked face is somewhat off which seems to be related to some missing coordinatemapping. it seems though that BMinXYZ and BMaxXYZ can probably be used to achieve that. any clues on how to use those @microdee @vux?
wow, thank you joreg, that's a perfect starting point, thank you so much
don't know why it's shifted, but it happens only with rgb. Trying with ir or depth instead and it is perfectly aligned. A walkaround that works perfectly for me is to add a translate after the perspective node, and translate the camera view of x: 0.053 and y: 0.004 in that way the face is perfectly aligned with the rgb image. I've tried to understand why that numbers, and tried to compute all the numbers i can together to achieve that numbers, but no luck, anyway it works perfectly ;) thanks again
here is an explanation for the offset: https://social.msdn.microsoft.com/Forums/en-US/14b9bc56-e890-4652-ae65-6c5dea80a432/doubt-with-hd-face-basics?forum=kinectv2sdk
I'm trying to make a geomFX to output a dx11 gemometry from the buffer data from the HDFaceBuffer node, but it doesn't work - a shader will complain there is no POSIITON and TEXCOORD in the geometry, which doesn't make sense.
Any idea?
Patch here https://www.dropbox.com/s/s28arp8i88b9agv/HDFaceBuffer%20geomFX.zip?dl=1
sounds like something wrong with the streamout setup. There is already a gsfx for this in Instance Noodles if you want to check there.
https://vvvv.org/contribution/instance-noodles
well so there is. thank you indeed
one wee question - the PN tesselator doensn't seem to work on the resulting face mesh for some reason. Any idea why?
(It works fine on a sphere geometry)
edit - also the tesselator displace makes all the polys separate - common vertices don't stay together
Not sure. Sounds topology related
Same situation here...it doesn't seems work gsfx and the geometry from the buffer data from the HDFaceBuffer. Any tips for this?
I also tried to read the vertices of HDFaceBuffer, but the problem is that i can't even readback the buffer. I tried multiple settings with different ReadBacks. all I can get is 506 X-Coordinates. Not a single Y or Z! I don't know the right setup :(
https://www.dropbox.com/s/qbhg881ykjpj2qm/Kinect2_HDFace_readback.zip?dl=0
@Noir: I Fixed it: Try everyoneishappys InstanceNoodle Pack, there is a Kinect2Face Shader, with this you can use getVertexData. This works for me.
@avaos thanks man I did the same 'thing' a month ago ehhehe