jstrecker's picture

Jaymie (@jstrecker)


  • Vuo Founder
  • Team Vuo
jstrecker's picture
Jaymie commented on Team Vuo's Feature Request, “Node set for skeletal tracking with Kinect

We recently surveyed a bunch of skeletal tracking libraries. Of the 27 we looked at, 3 are viable candidates to include in Vuo:

  • BodySkeletonTracker
    • head and arms only (no torso or legs)
    • depth camera required in practice, based on our brief testing
  • Skeltrack
    • head and arms only (no torso or legs)
    • depth camera required
  • PoseEstimation-CoreML
    • better tracking than the other two, based on our brief testing
    • uses regular camera, not depth camera
    • requires macOS 10.13+, not Windows/Linux-compatible

The other 24 libraries we considered wouldn't currently work with Vuo for one reason or another: Windows only, proprietary or non-commercial license, incompatible programming language, etc.

So we will likely choose one of the above libraries, unless something better comes along by the time we implement this feature request.

All of the above libraries input an image rather than interfacing directly with a Kinect or other camera. PoseEstimation-CoreML uses a regular video camera. Skeltrack requires a depth camera. BodySkeletonTracker says that the depth camera is optional, but we weren't able to get it to track without one.

Now that we have a better idea of the libraries we might use, I've bumped this feature request's complexity rating down a notch, meaning we will be more likely to choose it compared to a more complex feature with a similar number of votes.

jstrecker's picture

Chris, I discussed your suggestion with the team. While I totally understand not wanting to spend money on something that's going to be free soon, it would take us some time and possibly even a new release to unlock the trial version. We think it would be better to focus on getting the 2.0 beta release out instead.

jstrecker's picture

The problems that we've seen generally occur for certain combinations of GPU hardware and macOS version. That probably means they're bugs in Apple's driver code.

When two closely related GPUs use the same driver, such as the Intel Iris 640 and 650, it's likely that a problem seen for one GPU will also show up for the other.

However, sometimes more distantly related GPUs share the same driver, and not all manifest the same problems. For example, we think that the AMD Radeon Pro 555X uses a driver called AMDRadeonX4000GLDriver — which is also used by the AMD Radeon R9 M370X, the AMD FirePro D500, and the AMD Radeon 7970. The FirePro D500 and Radeon 7970 both have known issues. Yet my coworker is using a Radeon R9 M370X on macOS 10.14 and Vuo's graphics all work fine for him.

So that's why we always say, if you're thinking of buying some hardware, it's best if you can test Vuo on the system configuration before buying.

jstrecker's picture

Yeah, I'd recommend seeing what happens if you split it into two compositions. Also, is there some other Blackmagic capture app in which you could test receiving both streams simultaneously?

When the problem happens, do you see any messages from Vuo in Console?

jstrecker's picture

Joe, the problem with saving might be the Stop Composition. I thought it would be convenient but maybe it cuts off the composition before it's done saving. You could take that out and instead keep an eye on the timestamps. I just tested that with a longer recording (changed the 5s to 120s) and the framerate was steady. When you get a chance, maybe you could post a video that ended up with a variable framerate.

@eenixon, Vuo does timestamps, not timecode.