EDreams's picture

Hi all, first of all, thank you to the team and community for releasing such a great solution. Being new to Vuo, I've been spending time throughout the past month learning from examples and being impressed at the capabilities of Vuo. At this point, I'm still at an experiential and learning process, before delving into full compositions. That brings me to a roadblock .. as attached composition and described below;

  • Setup = Vuo running on Macbook Pro retina mid 2012,Mac Os Sierra, with kinect V1 (1414) +NiMate V2.12 OSC output

  • issue
    Taking @mic 's example & @Jaymie 's update, managed to get Vuo to receive the OSC messages from Nimate. However, the composition's frame rate/response would drop after 20 seconds or so. Suspect this might be in how I'd setup the nodes. Didn't have any response drop issue with Blender and Unity using Nimate's plugin.

  • intended results (definitely a long road, being new to Vuo and even QC) Create a skeletal frame/mask tracking + particles that interacts with a person as they walk/play using projectors, similar to this

  • https://www.youtube.com/watch?v=KLOB-T1mgdY

The sample above was done in processing, and is almost 3 years old. (There is a lot more examples and more recent as well, Google "Interactive wall/floor" =)_)

There is a lot of creative potential that can be achieved with VUO + Kinect + Leap Motion + Arduino, especially with brand activations projects.

Would really appreciate any help or experience that can be shared.

Thanks in advance.

Comments

With NI Mate's default

jstrecker's picture
Submitted by

With NI Mate's default settings, it sends about 1000 OSC messages per second. But on your computer it must be taking more than 1/1000 second to execute the nodes in the composition, so it falls further and further behind. (My coworker tested on a similar computer and got the same result.)

By default, NI Mate 2.x sends both Kinect skeletal data and Leap Motion hand data (even if a Leap Motion isn't plugged in). If you go to OpenNI > Skeleton Tracking and uncheck the "Hands" section, that should bring the rate down to about 700 OSC messages per second. That should improve your composition's performance, hopefully enough that it can keep up with the messages.

Another performance improvement is to avoid recreating the sphere every video frame, and just create it once on start (as in the attached composition).

To better address the underlying problem, I've created a bug report: Composition can't keep up with OSC messages sent from NI Mate.

A big thanks to Jaymie and

EDreams's picture
Submitted by

A big thanks to Jaymie and the team,

Tried the steps above and it still yielded the same results, guess would have to wait until the team further looks into it. In the meantime, would take a look and learn other compositions the community has to offer. Having OSC messages is integral to our R&D as we would like to explore the possibilities of having different I/O to add to the immersive experience in brand activation or show.

When i developed that

cremaschi's picture
Submitted by

When i developed that solution, i was using NiMate v1 ad had no issues. If it's still available, you could try with that; if not, you should find an intermediate solution that can limit the number of sent OSC. Quartz Composer output has a limit of 60Hz; you could try a QC bridge that accepts OSCs and output them, seeing if this works limiting them.

Found all the earlier

EDreams's picture
Submitted by

Found all the earlier versions on their forum NiMate V1.00++

NiMate V1.00++ needs license activation with the device serial number, so i would not have the chance to try it out until NiMate replied with a trial license (hopefully). Having a bridge through QC might help, but with my status as a novice in both QC and VUO, that might take some time =). However, on top of that using that solution would comes with a "con", where we are unable to pack the whole composition into a neat single app.

Currently i am experimenting with just using the IR/Depth camera to create a mask, but yet again, stuck in figuring out how to separate just the detected person out of the background. Tried an inversed color mask on a separate layer (based on examples), but that doesn't seems to work either. Attached composition at it's current state.

@EDreams, an update from Team

jstrecker's picture
Submitted by

@EDreams, an update from Team Vuo — We've been working on a way for Vuo to better interface with NI Mate 2.x, and have come up with a Filter Skeleton node that processes the OSC messages from NI Mate more efficiently. The node collects the OSC messages from NI Mate and holds onto them until it receives the final OSC message for the frame (/NI_mate_sync), then outputs the joint data for the entire frame at once. This greatly reduces the rate at which the rest of the composition has to process the skeleton data, and, in our testing, has enabled the composition to keep up with the data. The node will be included in the Vuo 1.2.4 release, which is coming within the next several days.

That is definitely great news

EDreams's picture
Submitted by

That is definitely great news. Would be looking forward to it. Guess the node would not only benefit nimate users, but generally applicable to OSC users as well.

I've further tried VUO with earlier versions of Nimate as @cresmaschi suggested, and realized that indeed vuo has less problem keeping up, though there is a noticeable lag of about 1 sec before it updates on the composition output. As the osc messages que up, the composition did not slow down, but the "live" output is rendered useless. Not appropriate if we were to use it in a live interaction session.