User interface (UI) node set — buttons, sliders, text/number boxes, menus

Hey guys,

As this feature request is gaining more popularity, I wanted to share some thoughts about this exciting group of feature requests.

The first thing to keep in mind is that people want customizable stuff I guess ! (most comments mention that). So I really think this whole UI thing has to be done the easy way and the most customizable way !

I tried to list all the different sub-requests mentioned here and split them in categories. Then I’ll try to share my thoughts about these and how I think things could be done.

So the difference things seen here are :

1 - Buttons :

   - Custom Styled buttons with :
       a - Borders
       b - Transparency
   - On/off States.
   - Pressed/Released/Unpressed States (with Image Skinning for each state).
   - Toggling.
   - Rollover.

2 - Other UI Stuff :

   - Checkbox/Toggle + Grid of Checkboxes (2D) Select Boxes.
   - Finder Windows.
   - Progress Indicators.
   - Sliders.
   - Text / Value Boxes.
   - Text Formatting Options.

   - DropDowns.
   - Momentary Buttons (Sticky VS momentary Buttons) & Group of Momentary Buttons.
   - Group Of radio buttons.

Ok, now let's go this list point by point down :

1 - Buttons & Custom Buttons :

I think the actual Node could become a « Make default Button » node for those who don't want to customize.

And that we could implement a new « Make Custom Button » node :

• It would have a « Layer » input (Or Image).

• It could have a similar Input for « Rollover and/or Pressed » states styles (however, modern Buttons are more about smooth animations).

• With « Combine Layers » you could customize the button’s style the style you want (Including Text, Borders, Shapes, Colors).

• The outputs could be a - Rollover. b - Pressed. c - Released.

• On/Off and Toggling could be easily done connecting these outputs to the actual « Switch » & « Toggle » Nodes I guess !?

Concerning Borders. If borders could be added to layers (or images), then all buttons could have customized borders (I am going to make a feature request for a generic Add borders to layer » node and a new discussion about the way all the « Get layer » stuff is retrieved in Vuo. (See mockup Below).

2 - Other UI Stuff

• Checkboxes & Select Boxes : You could easily turn a custom animated Button into a checkbox. Is it not the same ?

• Finder Windows : A total yes. A browsing Finder Window allowing us to pick up some images, folders, files …

• Progress Indicators : I think those can actually already be done. For example you can interpolate the width of a layer on whatever values !?

• Sliders : Yeah Sliders ! Great ! But again, we will want to customize them. Customizeabled buttons and bars for them. However, wit some tricks this can actually already be done. Check out my composition here, it has some custom sliders (see screenshot below) and here is the link Composition Link.

• Text and Value Boxes : Yes ! Definitely ! The big missing one so far to me ! But again, the need to be customizable ! For example a custom layer for the box + custom fonts for the text ! (see mockup below).

• Text Formatting Options : There is a specific Feature request for text wrapping : Wrap Text Request by Teo Dumbski. Could we use that ? Or should buttons have its own possibility here ?

• DropDowns : Mmm I can’t figure out yet how this could be done.

• Momentary Buttons (and group of momentary Buttons VS sticky Buttons) : I don’t understand this part. If this is about vanishing buttons after they have been triggered, I think that specific Request about Disabling Nodes could be useful ? Disabling Nodes.

[alexmitchellmus] Can we also have a GUI or UI class set for the API?

In the Vuo 1.1.1 source code, check out node/vuo.ui/vuo.ui.button.action.c — that demonstrates creating a layer group and adding rounded rectangle and text sublayers to it.

Going forward, I think we'll start building higher-level C APIs for that stuff, since a lot of code will be reused between, for example, the (existing) action button and the (forthcoming) toggle button.

I am starting to really like super simple UI's. Also from an implementation perspective it may be cheaper (load wise) to have a UI consisting of as many square edges as possible. Something like this looks lovely to me:

I understand that that is the LibCinder look, but something similar could be very classy for Vuo.

Link for ciUI: https://forum.libcinder.org/topic/ciui-a-user-interface-block-for-cinder

Great to see this has be chosen to be implemented !

Can we get some info on how the team plans to add and do this ? What sub-parts will be implemented and which not ?

Thanks ;)

Can we get some info on how the team plans to add and do this ? What sub-parts will be implemented and which not ?

Bodysoulspirit, we've made a plan and started implementing, but I don't want to get specific on promises in case we don't get to all of it in Vuo 1.3. We're taking this whole thread of comments into consideration.

For example, GUI elements lend themselves to fixed pixel dimensions even when the Viewer Window changes dimension. How does that whole can of worms get sliced and diced?

Ew :P

@Alastair, like the existing Make Button node, the UI nodes will output layers, and will scale with the window's width. Although we won't get into advanced UI layouts/sizing for Vuo 1.3, the existing Arrange Layers in Grid and Align Layer to Window could help.

Also what bearing does this have on the bigger discussion we had some time ago about Front Panel interfaces/VST type skinning for sub-compostions.

None. Feature request?

What about GUI elements on the graph canvas itself… some nodal environments allows for sliders and switches on the canvas.

Ditto.

i think everyone should familiarise themselves with Origami's features and ways of doing things as food for thought

How do you mean? My understanding is that Origami has a Button patch but none of the other UI widgets proposed on this feature request.

once the Vuo Editor has a free version and these nodes become free to use

Just to make sure this doesn't become a rumor — Team Vuo doesn't have plans at this time to make a free version. For info on how Vuo is funded, see https://vuo.org/story .

This is going to make interfacing with my VUO projects so much easier! Instead of relying on wireless Art-Net protocol and Luminair to make a less than ideal UI for my project I will be able to make a custom UI and use Duet Display to directly control it via a tethered iPad! I'm really looking forward to trying this out!

Keep doing what you're doing VUO team because it's awesome!

Jaymie great Jaymie !

Glad to hear a node to build a fully-custom theme from layers is on the roadmap !

I guess it is important that people can make it look like they need it.

Hope "border thickness" will be part of the appearance node you mention when you say "For each type of widget, there will be a node to specify font, colors for various states (normal, hovered, pressed), corner radius, etc., and a node to build a fully-custom theme from layers" as it seems that was mentioned several times by people in the comments above.

Any plans for a "Finder Window Explore" node to pickup stuff and retrieve their url's like with "Get Drag values" ? Or do you already think this should be a different feature request ?

Bodysoulspirit, we have plans for nodes that display a File Open dialog and a File Save dialog. They would have an event input port to show the dialog, and would output the selected file(s) when the dialog closed. Is that what you have in mind ?

Sounds great Jaymie ! Cool !

Yep ! Some Finder Explorer Popup similar to the one that comes when in the Vuo Editor you do "cmd-O" or File > Open Composition.

Glad to see Fonts make the cut, hopefully for all UI element types Fontface is user assignable.

✅ Custom font faces for text elements of all buttons

⚪️ Can embed fonts for use in Exported Apps and Plugins

I'm pretty sure I found a way to bundle some assets in Quartz Builder but I forget if fonts was one of those things, sure they will bundle but will they be loaded by the host OS?

Some kind of shell script that can run an AppleScript at startup would be quite nice. I'm sure Team Vuo have much higher priorities but it would be nice, and I could envisage some low-cost sellable screensavers and apps coming out of that.

Chris,

You can use the Save Data and Fetch Data nodes, optionally in conjunction with the Tree nodes, to retain the state of the UI nodes.

Is there any timeline for when we might see a text input UI node? Quickly installed 2.0.1 ... nope. Urgently need this one. :-)

For my current use case I have realtime graphics being created by Vuo on a secondary screen that is projected, this is what the audience sees. I wanted the operator (teacher) to be able to feed text questions into the realtime graphics.

Paul, William,

Perhaps you could use Display Console Window as a workaround for now. As the node documentation says, "Typed Line — When a linebreak is typed in the window, this port fires an event with the line just completed." So you can enter a line of text in the Console Window and output it.

I agree with bLackburst. Console window is not a viable workaround from real-time operation of anything.

Years ago I spend countless hours making a multi-bank switcher in QC for QC that used spooky patches along with some pretty simple JS to sample data at any point of the composition and record that value (bool, int, real, string, array or complex object) and then save that to a patch like an old analogue synthesiser (think Roland synths like the Juno 60). Then at any time you could restore any of these patches and those data values, including objects would replace the value on the noodle coming out of it. If you changed the value coming into the patchbank node then it would change the value again just like moving a slider on an old analogue synth would edit the current patch. The JS code for the controller app was hundreds of lines long and became too much of a headache for me when i realised I needed to refactor it using an improved coding style to get the bugs out (I was teaching myself JS along the way!)

Anyhow, I put in a feature request to put the framework for that concept into Vuo. Basically the ability to sniff state and write state. Either in dedicated Vuo nodes or as a new API interface for all new Vuo nodes.

But another work-around is another thing I requested. A websocket node. Websockets allow two way binding between web pages and other programming tools. there was a 3rd part websocket tool i used with QC that allowed for reasonably sophisticated UIs to be built competitively effortlessly with HTML & CSS and a tiny bit of JS just to implement the websocket binding for each button, slider, enumerated list etc.

EDIT> FR links added

I'll say it more simply, bLackburst!

Team Vuo clearly has a capacity issue in delivering FRs and updates for what many consider to be core-need Vuo features. Therefore we need to make it as easy as possible for Team Vuo in terms of development hours and integration complexity with the existing Vuo codebase and code roadmap. Not to mention bug fixing down the track.

UI tools are inherently behind the times because expectations are always moving on to the next big thing. In my experience using QC, it takes a long time to smooth all the rough edges and corners making generalised tools that work for any window size and play nice with resizable windows etc etc.

In my view deploying the websocket API offers the most bang for buck for @teamVuo's effort to provide a workable UI solution. All they need to do is build a bridge using the tried and true websocket framework. All the UI tools are already there in internet-land. Anybody who can teach themselves VUO can learn to make a few buttons, radio buttons, sliders and drop down lists and arrange them on an HTML/CSS page with other text and images. And i can guarantee you whatever UI tools Team Vuo can release in the next year, some people will find them lacking compared to what they're used to in other apps.

One down side is that it will be outside of a compiled Vuo app that we might want to distribute to other people, but a browser based UI will still be able to talk to the Vuo app via websocket and we might even be able bundle the HTML/CSS/JS files and to script the webpage to launch?

If Team Vuo think that they can do a set of custom UI features with less time invested in dev and polishing than it takes to make a websocket API implementation inside Vuo then go for it but I'm, extremely doubtful. Remember that a Vuo UI feature set will require all the logic processing nodes to do the view/controller/data back-end of the screen elements also. To me, websocket is the best option for step one down this path.

Also i found this thing Mozilla developed called HumbleNet: A cross-platform networking library that works in the browser. It consists of a C wrapper around WebSockets and WebRTC that abstracts away cross-browser differences, facilitating the creation of multi-user networking functionality for games and other apps. It might be a shortcut for Team Vuo to get websockets and the WebRTC low-latency protocol happening ASAP, although I'm not sure how much it allows for websocket exclusive use, seems ot be used just for the authorisation phase of the connection.

Magneson! This is awesome! Thanks for providing not only a working example of text input, but also a tutorial on how you built it.

Until there's a textfield node built in to Vuo, people can reuse your composition in their own compositions. At the end of your tutorial, I like how you explain how to turn it into a subcomposition for easier reuse.

bLackburst, please keep in mind the community agreement, specifically about generalized negative statement. Ideas about alternative ways to accomplish a task in Vuo are always welcome on this forum. Criticisms are also welcome, but they need to focus on specific points that could be improved.

To all who've asked about a textfield node — We're still planning to add one. We actually started work on a textfield node while developing the Vuo 2.0 release. I would not have guessed this, but it is surprisingly, fiendishly difficult to make a textfield with customizable appearance that behaves like a proper macOS textfield. After a lot of hours spent, we decided that it wasn't worth holding up the release for this, and we would come back to it as soon as we could. Right now our focus is on getting Vuo ready for the macOS 11 release and Apple's ARM processors. Once those are under control, we plan to pick up where we left off with the textfield node (which, based on the comments on this issue, is resoundingly the most wanted of the remaining potential UI nodes).

Thanks Martinus for the compositions and the reversed queue node !

I got a bit lost between the different subcombs and how to use them, but thanks to you, I was able to tweak your original composition to match my needs, use some buttons as some text fields (clicking a text field resets the field however) (see joined composition below).

One thing I wonder though reading Jaymie's comment about the work needed for an UI text input, is I wonder if under the hood some changes could not be made to Vuo to make these kind of things easier to implement.
Is it because of the push only system it's so hard to code ?
I'm no code expert at all, but UI text fiels are basically everywhere, internet, apps etc, do under the hood they all have major code ?
I was thinking about this also because you can't create lists of buttons since they need the window port, and this isn't supported within process list loops.
I wonder if it will be possible with the "Allow single ports to accept lists" feature request to create a list of buttons on the fly, with each of them be clickable and editable.

bLackburst and Chris when a websocket plugin was made available for QC i had an low-latency interface with a self hosted (same Mac) GUI that was used to drive giant video screens in front of audiences of 100,000+ people. I built the GUI i wanted with drop down lists etc and push button preview and live states in a few hours. If you want to be dismissive of such solutions, bLackburst then i think that's more on you than anything else. I'm only suggesting that this is a low-effort implementation for Team Vuo, given that they seem to be running into significant hurdles on the UI element nodes.

Chris I can understand why you want stock VUO tools to do it. It's a shame it's so hard to make them using Vuo itself (I haven't ever tried but I'd have thought someone would have by now b/c they are so essential for production tools if you aren't using VDMX or similar). Many people made such things for QC over the journey. I made an advanced GUI inside QC using JS and it had 30 or so buttons and several modes of operation, flashing LED animations when moving between modes and two digital readouts for numbers (it wasn't any improvement on latency than Websocket though).

There's a VUO node for loading an HTML page that could be embedded in your app. VUO may even get an HTML server node one day since Kineme built one for QC, in which case it could be all bundled inside the app.

One thing I wonder though reading Jaymie's comment about the work needed for an UI text input, is I wonder if under the hood some changes could not be made to Vuo to make these kind of things easier to implement.
Is it because of the push only system it's so hard to code ?
I'm no code expert at all, but UI text fiels are basically everywhere, internet, apps etc, do under the hood they all have major code ?

If you're building, say, an iPhone app or a webpage, it's easy to pop in a textfield because the app or webpage is built on top of a platform that provides the widget. For iPhone, there's a UITextField class within the UIKit framework that you just pop into your code. For webpages, the HTML standard defines an <input type="text"> element that all browsers implement.

For Vuo, we have to build the underlying platform that provides a textfield widget that's cross-platform-compatible and has a customizable appearance. Luckily, we don't have to start from scratch. We're building on top of the open-source stb_textfield. Even so, there's a ton of tweaking needed for the appearance and behavior. Textfields have a lot of little behaviors that most people don't really think about but would miss if they weren't there, or if they behaved differently from typical macOS textfields (cursor, highlight, copy-paste, etc.).

Our next step on the textfield node will be to reassess where we left off and come up with a minimum set of changes, with the goal of not letting the perfect get in the way of the good. Then we can release a textfield node that is hopefully decent, and over time tweak the behaviors that the community considers most important based on bug reports.

In Vuo 2.2.0, we added Make Text Field, Make Number Field, Make Text Field Theme (Rounded), Display "Open" Window, and Display "Save" Window.

Known limitations of Make Text Field and Make Number Field:

• right-to-left languages (such as Arabic) aren't currently supported
• 4-byte glyphs (e.g., 💯) aren't currently supported; they show up as a question mark in a box
• keyboard shortcut to show the character picker (control-command-space) isn't currently supported
• System Preferences > Keyboard > Dictation isn't currently supported
• System Preferences > Accessibility > VoiceOver isn't currently supported
• tabbing from one field to the next isn't currently supported
• context menu isn't currently supported (though the shortcut keys for common operations (cut/copy/paste/undo/redo) work)
• cursor doesn't flash
• spell/grammar-checking isn't currently supported
• phrase-substitutions, smart quotes, and auto-capitalization (System Preferences > Keyboard > Text) aren't currently supported
• should scroll when the text width exceeds the widget width, rather than expanding the widget
• double-click-drag should select by word, but currently it selects by character

We're going to call this feature request "released". Lesson learned: it's easier to track feature requests that are smaller and more focused :) If there's another UI widget you'd like to see, or limitation of a UI node that you'd like us to prioritize, please create a separate feature request.

Jaymie, now that I realise macOS apps exported from Vuo don't natively support published inputs the way image filters and image generators do because the host app deals with all of that, I'm wondering if Qt (community licensing permitting) might be the way to implement more GUI features with a consistent underlying code base that makes expansion of features more easy in the future?

Does Qt to some extent get all the little tiny things sorted without heaps of custom coding?

Does Qt to some extent get all the little tiny things sorted without heaps of custom coding?

We've looked into that, but unfortunately Qt doesn't solve the problem for several reasons:

• Qt doesn't support rendering widgets within Vuo's Layer system (3D graphics rendering, currently implemented with OpenGL, soon to be replaced with Metal).
• Qt doesn't support the amount of customization of widget appearance that community members indicated they wanted.
• Including the Qt framework would significantly increase the size of the Vuo framework, affecting the size of exported apps and plugins as well as other applications that integrate Vuo (VDMX and CoGe).
• Several years ago, when we tried to use Qt for composition windows, we ran into some pretty hairy macOS-specific issues. We ended up switching to the macOS Cocoa frameworks.

So that's why we don't use Qt within compositions, though we still use it in the Vuo editor.

Yes, please create feature requests or bug reports for anything further with the UI nodes.

.

Chris, Bodysoulspirit, and other community members, we'd like to have individual feature requests for specific UI elements so the community can help us prioritize them. Bodysoulspirit's list might be a good start:

⚪️ Grid (2D) of Checkboxes
⚪️ Radio buttons
⚪️ Group of radio buttons
⚪️ Momentary buttons (sticky vs momentary)
⚪️ Group of momentary buttons (menu)
⚪️ Select box (same as Finder window?)

We've also seen discussion about knobs/rotary sliders. I left out Dropdown menus from the list, since PBourke has created a separate Popup Menu UI node feature request.