Node set for skeletal tracking

.

I don’t have anymore votes but this is something I would like to see natively supported in Vuo, especially if it also supports the newer Kinect One (Kinect V2) hardware.

Before voting I would need to know if kinect v2 is supported.

@bLackburst, that’s a good question, and unfortunately we don’t know the answer yet. It depends on the library / 3rd-party code we would use to implement the nodes. Ideally we’d be able to find a good library that takes depth images as input, so with the Kinect v2 feature request you would automatically get Kinect v2 support in the skeletal tracking nodes as well. However, if the only available adequate libraries grab the Kinect depth images themselves instead of allowing them to be input by the user, it would depend whether those libraries support Kinect v2. We’re still researching libraries and are open to suggestions.

We recently surveyed a bunch of skeletal tracking libraries. Of the 27 we looked at, 3 are viable candidates to include in Vuo:

  • BodySkeletonTracker
    • head and arms only (no torso or legs)
    • depth camera required in practice, based on our brief testing
  • Skeltrack
    • head and arms only (no torso or legs)
    • depth camera required
  • PoseEstimation-CoreML
    • better tracking than the other two, based on our brief testing
    • uses regular camera, not depth camera
    • requires macOS 10.13+, not Windows/Linux-compatible

The other 24 libraries we considered wouldn’t currently work with Vuo for one reason or another: Windows only, proprietary or non-commercial license, incompatible programming language, etc.

So we will likely choose one of the above libraries, unless something better comes along by the time we implement this feature request.

All of the above libraries input an image rather than interfacing directly with a Kinect or other camera. PoseEstimation-CoreML uses a regular video camera. Skeltrack requires a depth camera. BodySkeletonTracker says that the depth camera is optional, but we weren’t able to get it to track without one.

Now that we have a better idea of the libraries we might use, I’ve bumped this feature request’s complexity rating down a notch, meaning we will be more likely to choose it compared to a more complex feature with a similar number of votes.

1 Like

I’d also like to see this happen!!!

I just picked up a Kinect v2 for cheap. If Vuo ever incorporates skeletal tracking down the road I hope it will still work with this sensor. Until then I will just play with the Depth data.