iaian7's picture

Vuo 2 is looking more and more promising, thanks for all the hard work!

As a test this afternoon I ported a screensaver I'd made in Quartz Composer earlier this year (reling very heavily on the GLSL patch at the time). Using the Make Image with ShaderToy node, I was able to port it to Vuo surprisingly easily. I lost all of my custom shader inputs, of course, which had to be hardcoded into the shader, and it couldn't load more than 4 images, but it did...work. I was able to make things a little more efficient while I was at it, which was nice. And on top of that, I was then able to (with the help of Chrome Console to load custom images) get it running on ShaderToy too!

However...full screen performance in Vuo, and performance of the packaged screensaver, is unusable. I'm using the same 4k textures across all platforms, and since the code really is the same in ShaderToy as it is in Vuo, I'm not sure what the problem could be.

  • The Quartz composition runs great in full screen (multiple monitors, 60fps confirmed)
  • Chrome runs great in full screen (~60fps)
  • Running it in a Vuo window appears to be fine (~60fps)
  • Running in full screen isn't great (~30fps with frequent frame drops and stuttering)
  • Exporting as a screen saver is worse (not sure, but <20fps with lots of frame drops and stuttering).

Is this a known issue? Has anyone else tested performance between Quartz Composer GLSL, Vuo, and ShaderToy, especially in fullscreen and/or as a screensaver?

Asking here before I file a bug report. :)

Moderator note: 

Adjusted formatting

Comments

One possible difference is

jstrecker's picture
Submitted by

One possible difference is that Vuo renders using Retina resolution when possible, whereas QC doesn't (it always renders low resolution). So the performance difference might be because the shader is trying to run at twice the resolution (4 times the number of pixels).

If you add a Divide node to divide the width and height each by 2 before rendering, is the performance more equal?

Excellent point! I hadn't

iaian7's picture
Submitted by

Excellent point! I hadn't thought of that, and thanks for your quick reply.

Resolution multiplied by 0.5:

  • Vuo Composition Loader in fullscreen now performs the same as Quartz Composer in fullscreen (Vuo periodically drops a few frames, but it's certainly improved, and maybe even better than a .QTZ running as a screensaver, which appears to drop in performance ever so slightly from the original Quartz Composer fullscreen window)
  • Vuo screensaver renders with improved performance over previous screensaver attempts, but only in a quarter of the screen with stretched pixels filling left/top/right (whoops...clearly not compatible)

Multiplied by 0.5, then scaled back up to the original screen size (Resize Image node):

  • Vuo screensaver renders full screen at somewhat improved performance, but still renders much slower than than the Vuo Composition Loader, and is still dropping a noticeable number of frames. Very jerky.

So it helps a little, but not enough to make it usable as a screensaver. In addition, this doesn't seem to explain (from my uneducated perspective) why in my initial testing the Vuo Composition Loader rendered more reliably when it was in a full screen window versus true full screen (I wouldn't have expected the pixel space of a dock and menubar to cause any measurable performance difference). And since Chrome appears to render the exact some code without issues in true full screen (not sure if it's retina!)...this is...curious.

Thanks! I'll give it a try.

iaian7's picture
Submitted by

Thanks! I'll give it a try. Up to this point I've been using "Fire on Display Refresh" as recommended in the "Make Image with Shadertoy" node documentation.

Test 1 - try "time" first

I figured I'd go ahead and test all the timing options, starting with the default "time" input from the Image Generator protocol. When playing in the Vuo Composition Loader in full screen mode, it performs about the same as fire on display refresh. Some problematic frame dropping in fullscreen, not as bad in a windowed view.

However, using the "time" input and exporting it as a screensaver boosts performance when compared to display refresh (which practically dies when used in a screensaver), up to the same levels as the Vuo Composition Loader in full screen mode (albeit still not reaching Quartz performance levels). It appears "Fire on Display Refresh" really isn't happy when embedded as a screensaver.

And that's all rendering at retina resolution (sorta; it's set to 1.5x since I'm on a Mac Pro with dual 24" 4k displays). Curiously, using "time" in conjunction with the half-resolution-then-scale-back-up technique described in a previous post actually kills screensaver performance again. Completely unusable frame rate, super jerky.

At which point I realised this was going to require documentation with a lot more detail...

Test 2 - test everything over again from scratch

Of note: performance in the Vuo Composition Loader was tested both in a window scaled to fill the entire screen (minus the dock/menubar), and in fullscreen mode, which uses the newer MacOS fullscreen setup (where takes over both monitors and leaves one of them blank, very frustrating). Performance is consistently better in windowed mode. However, Quartz Composer suffers no such performance hit: windowed mode and fullscreen mode are identical, however it's not the "new" fullscreen - it simply fills one monitor, and the second monitor is fully usable (oh how I long for the old days of MacOS supporting dual monitors nicely without using them as separate spaces, sigh).

  • Quartz: perfect performance in all situations
  • Display Refresh: not compatible with screensavers
  • Time: much better, not perfect
  • Periodically 0.0166... (~60fps): worse again
  • Periodically 0.0333... (~30fps): minimal improvement

...and also...

  • Half-res (tested individually with each of the above except for Quartz): seems to make no difference in screensaver mode

Also note: dividing the resolution by half doesn't actually match my monitor scaling factor, so it's both significantly lower performance and noticeably lower quality than Quartz, which renders at a somewhat higher resolution (my monitors are set to 1.5x scale, so quartz would be rendering at 66.6666% of full resolution instead of 50%).

Attached you'll find the spreadsheet of results. The percentages are simply my subjective evaluation of performance, with anything under 95% being (in my opinion) unreliable, and anything under 90% being unusable (noticeably jerky or low framerate). Unfortunately, limiting the frame rate by using a "Fire Periodically" node doesn't appear to improve performance at all, and in some cases, actually makes it worse. Is there another, more correct way to limit frame rate?

And just for reference, here's the node graph in Vuo:

  • Yellow = GLSL shader (almost exactly the same as what I'm using in Quartz, but with none of the exposed controls)
  • Green = Timing (this input is what I was testing with different firing options, not pictured here) (the add node combines the time input with a random number so each monitor displays a different pattern...I didn't have the Allow First Event hooked up to the Make Random Value node, so it wasn't working for my initial tests and this screenshot, whoops! Fixed now, working as expected)
  • Blue = Image loading (four different 4k textures, same ones I'm using in Quartz)
  • Orange = Resolution scaling (hooked up to a boolean so I can toggle it on and off for testing...based on the results above, it really needs to stay off for anything used as a screensaver!)

I think it's a matter of you

Scratchpole's picture
Submitted by

I think it's a matter of you finding the happy medium between resolution and fps, you could also try lowering the resolution of the input images maybe. I'm surprised to hear qtz is running it smoothly.

Fire periodically is what I had expected to work.

I have noticed a big drop in performance with multi-pass shaders...reducing to really low res, sometimes lower than 1280x720 and 25fps to make some function smoothly on my crappy mbp.

Not only is Quartz running

iaian7's picture
Submitted by

Not only is Quartz running this smoothly, so is Shadertoy via Google Chrome (100% performance in fullscreen). Using a .qtz as a screensaver, I have this same GLSL code running as a background element along with more complex animation layers in the office lobby, pulling settings and text content from a web server running on the same system. And that's all running on a stock 2011 Mac Mini; though there are a few frame drops periodically, it runs surprisingly well, and has for many months...after years of reliably running other versions of Quartz Composer screensavers.

Based on the poor performance I'm getting with Vuo on a newer Mac Pro with far higher end GPUs, I'm hoping this is a beta issue that will be resolved by the time its released. That's why I'm bringing it up now! There's no "happy medium" if Vuo is a trade down from a more reliable and performant (though deprecated) system like Quartz Composer.

My 2 cents on a non-technic

Bodysoulspirit's picture
Submitted by

My 2 cents on a non-technic level is :

I usually see some frames drops going fullscreen in Vuo too, compared to non-fullscreen even with a window the size of the screen.
Funnily enough usually checking the port events it does NOT register frame drops, but you can still see some chops.
But I did blame old 2011 hardware, or maybe the small amount of extra pixels rendered instead of the window bar.
There are also 2 fullscreen modes in Vuo if I'm right, cmd-f in the composition goes Vuo fullscreen, and the regular macOS fullscreen (green button).

Regarding timing, I guess the time published port instead of display refer is the way to go though for best results.
And instead of Fire Periodically use Allow Periodic Events.
Not that it will make a difference I guess, but it allows to be synced with the main time.

Couple of things comes to

MartinusMagneson's picture
Submitted by

Couple of things comes to mind. First it seems like you're always creating the full resolution image. In the case of the low-res selection, you only see the lower res one, but in the comp you also create the full-res one anyways. This is one of the major differences between QC and Vuo. In QC you wouldn't have created the one that wasn't selected, as it pulls the values to the output. In Vuo however, it pushes all calculations to the conclusion from the inputs enabling background calculations, but at the price of having to be a bit more careful about the values and blocking events.

I also wonder how you deal with the wrap-mode in QC? I haven't used this (vuo) node myself, but if I understand it correctly, it in this instance would check 4 4k frames/textures at every frame cycle. That is a pretty hefty operation - especially for older/mac hw. I would try removing those nodes to check performance.

Thanks for the notes and help

iaian7's picture
Submitted by

Thanks for the notes and help! I find myself continually forgetting that Vuo pushes updates instead of pulling them. Ok, a few things to cover now...

  • Glad I'm not the only one seeing performance drops when switching to fullscreen. I'd been using the native MacOS mode, totally missing that cmd-F uses the old style fullscreen (so much more useful!). Either way, I'm still seeing the same performance drop in both modes. I wouldn't blame it on old hardware if nearly-fullscreen windows are fine, right? There's something weird going on with how the full screen view is being rendered? I have a somewhat newer and much more powerful machine, and I'm seeing the same issues.

  • The Change Wrap Mode nodes shouldn't be modifying the image pixels, they should be modifying the metadata regarding how the images are rendered in a GLSL context, defining how the textures behave at UV borders (<0 and >1). Additionally, they're set up to process only once when the image is loaded (Allow Once node feeds into the image loading), so even if they were processing pixels, it'd only happen once. In Quartz Composer the wrap mode of any image is automatically set to repeat and I didn't need to customise it (I can't remember if that's an option in the QC mipmapping node? My memory is failing and I'm not sure you can change it to anything but repeat). Unity also defaults to repeat, as does Shadertoy. Vuo defaults to Clamp Edge, so the setting has to be customised to work as expected (the animation I'm working on right now animates the textures in an infinite UV scroll, thus the need for repeat).

  • I've implemented the Select Output node, even for the math operation just in case (seems silly, but hey, gotta try!). Checking the Show Events feature confirms that Resize Image is not being processed when LowRes is toggled off, and the resolutions being fed through the graph all check out as expected. Yay, fixed! But...

  • I've gone through performance testing after fixing my mistake on the resize image setup, and strangely...no change. I really thought that'd help. But I guess it makes some sense: I was having major performance issues before I created my testing setup, which is when I introduced the scaling selection error. I'm not seeing any improvements to the ratings I posted before. I'm still getting the same poor performance (low frame rate, dropped frames) when running in fullscreen or as a screensaver.

  • I've tested Allow Periodic Events versus Fire Periodically, and there doesn't seem to be any difference. They both result in significantly worse performance than simply using Time.

  • I ended up using the mouse position as a hacky way to get some sort of data input, which allows me to correctly adjust the UV map scale within the GLSL depending on the high/low resolution toggle (something that's specific to this animation). So that's why I have a +1/-1 Select Input node.

iaian7 — Thank you for

jstrecker's picture
Submitted by

iaian7 — Thank you for sharing your test results.

Another thing you could try is to go to System Preferences > Mission Control and toggle the "Displays have separate Spaces" setting, then log out and log back in, and see how the performance compares.

To get a more objective measure of performance, you could add a visualization of skipped frames as in the attached composition.

One thing we're thinking of changing based on your comments so far — As you observed, currently you need a Resize Image to scale the image back up to the screensaver's expected size. We could change the wrap mode on the output image from "clamp" to "stretch" so you wouldn't need the Resize Image (and extra processing that it incurs). Assuming that doesn't break anything else.

Jaymie — Thank you for

iaian7's picture
Submitted by

Jaymie — Thank you for checking out the tests, I appreciate the follow up.

On my main desktop I always run with "separate spaces" off, so I tested again with it on and saw no change in performance, except potentially fullscreen mode, which may have performed slightly better (still not great), and it used the "native" MacOS fullscreen method for both the expand button and command-f (makes sense, separate spaces means it doesn't have to use the old style fullscreen in order to keep the other screen usable). Screensaver performance with separate spaces turned on was same/worse.

I've also tested the two final Vuo screensavers (full-res, half-res) on two laptops (retina with Mojave, non-retina with High Sierra). Neither of them were able to render either of the screensavers at an acceptable frame rate. Meanwhile, that same ancient non-retina High Sierra MBP can run significantly more complex Quartz screensavers at a flawless 60fps...so in theory (big caveat, haha) it's not a computer performance issue, nor is it entirely a retina resolution issue...seeing as the non-retina laptop can run the same code (and more complex variations) in Quartz without a hiccup. Of note, I haven't tested Vuo extensively on the laptops outside of the screensaver test; I need to check the app preview performance too and see how that compares.

Today for yet another test I ported everything over to an ISF generator (yay! I get to use input variables again!), and got the same poor performance I had with the ShaderToy node. On the upside, outside of the easily-broken header information, ISF was a lot more pleasant to work with (I could use Sublime Text to edit the module and Vuo + Vuo Preview window would update every time I saved, nice!). Though unfortunately not helpful for actual production work (I can't store hundreds of modules across many different projects in a user library, especially when they have to be saved alongside the project for versioning and archival).

Thanks for the skipped frames test, that's great! That said, even testing it alone without any additional graphics, I can't get it to render smoothly as a screensaver. Completely erratic performance. If I get the time this week, I'll try adding a skipped frames test to the Quartz compositions, then render tests across as many computer setups as I can (but I wouldn't hold my breath...one of my teams is in the middle of a harrowing delivery week on a major mixed reality production!).