Lambdas/closures — ability to pass nodes into other nodes

[root (Team Vuo)]

Be able to have a node (or subcomposition) execute another node (or subcomposition). Some examples where this would be useful…

Provide higher-level nodes that encapsulate bigger tasks, so that you can save time by using these nodes instead of having to build up the same composition structures over and over. For example, a “Slideshow” node could input a list of images and a list of nodes that transition between images.

Another higher-level node, suggested by @ajm, is a “Filter” node that could input a list of data and a node that outputs a boolean (yes/no) to decide if each data item should be passed through to the output list or filtered out.

Select between different nodes to execute, such as different image filters. You can do this currently with the Select nodes, but it can be clumsy to build a composition that does it optimally (only executing the currently selected node).

“For Each” node (or “Map” node, mentioned by @ajm) — Do something with each item in a list. A different way (perhaps more convenient in some cases) to accomplish the same thing as Iteration: Turn most nodes into iterators by allowing single-value ports to accept lists.

Implementation:

• Add a special output port to some/all nodes — “function port”.
• Enable a node to have an input port that can be connected with a cable to another node’s function port.
• Enable a node or subcomposition to execute a node passed in through a function port.

To be determined:

• Will all nodes have function ports, or only certain ones (such as stateless nodes)?
• Many design and UI issues.

[useful_design (Alastair)]

This would be next level Vuo. The killer feature, but I think Macros (in QC terminology) or child level sub-comps that can be editetd within parent level files must come as a first step.

[ajm]

This sounds cool but I’m having trouble imagining how it would work.

Can all nodes be passed as values (I’m picturing an extra port on the bottom), or only subcompositions?

Passing functions makes me think of a couple things:

clojures / capturing binding / scope / partial application

It seems sort of central to Vuo that nodes don’t have any scope/bidings that they aren’t passed explicitly, so it’s difficult to imagine what a closure would capture.

We can already do partial application explicitly by, for example:

• Instantiating a Count Within Range
• Set several (but not all) of the input ports
• Publishing a couple input ports and the output port
• Saving that single node as a submodule to the node library

classical higher order functions like map/filter/reduce

iteration seems pretty similar to map, in practice—apply a function to a collection of values and return a new collection of values.

Process List and map are isomorphic: they do the same thing. It gives us a model for how to implement other higher order functions.

I tried out implementing filter, both with and without function passing.

Assume the orange nodes are published ports. The top clump is the submodule definition, and the bottom clump uses the defined submodule in green.

Here it is as a regular submodule:

!

And here it is using function passing. Assume that Apply accepts a function in its event port, and applies it to the value coming in its value port:

!

I think I like the first one more, it feels simpler and more explicit. A more complex example may shed more light.

Process List is essentially a higher order function already. We get to explicitly pass both the input and output ports, which I think is even more powerful than just passing a function because we can pass a more complex structure of nodes without composing it into a subcomposition, if we want to.

Since Vuo doesn’t really have a notion of scope, I can’t think of anything I could do with function passing that I couldn’t do as easily with the existing subcomposition system. This is probably a lack of imagination on my part, but I think improving submodule support, (including local submodules) would achieve the same benefits as function passing, but with more generic and broadly applicable benefits.  

[jstrecker (Jaymie)]

@ajm, I just filled in the description on this feature request. Does that answer all your questions?

Filtering a list actually can’t be accomplished with Process List, or not easily. The reason your mockup with the Select Input inside of a Process List loop wouldn’t work is that the events fired from Process Item need to match the events received by Processed Item one-to-one, or else Processed List won’t fire.  

[useful_design (Alastair)]

Passing functions in theory allows for much more reusable code (does in other languages like Haskell). Possibly less coding errors, due to both code reuse and easier architecture for breaking problems down into small steps which can be tested for reliability and then reused.

Lambdas also at least in theory offer potential for better performance (due to optimisations in the compiler) with lazy evaluation when a chain of functions is applied to dataset, a bit like CoreImage architecture that optimises the calculation of pixels to just those pixel that map to the final region of interest in final image. Maybe @jstrecker can confirm or dispel that idea.  

[MartinusMagneson (Magneson)]

What I find intriguing about this is the implied parallell processing of the items. Am I right in this assumption? It would be huge in terms of the recent trends in processors, shifting to a large amount of cores/threads.

Could perhaps a multiplexing solution also fit into this some way? If you create a struct of input types from generic ports, could the function itself be a port type (wildly speculating)?

[useful_design (Alastair)]

Wouldn’t like to say about the parallelism, I’m still learning lambdas in Haskell and at very early stages of a general comprehension of what goes on behind the curtain.

One of the big wins is the lazy evaluation (which QC has) so you can have a function that is the Fibonacci sequence {0, 1, 1, 2, 3, 5,…} (an infinite list which QC doesn’t have ;-) ) and pass that to a second function that takes every second element in a list (potentially an infinite list if input is infinite list) and pass that to a third function that takes the 1000th and 1001th elements and adds them. It will not do two iterations over infinite lists like an imperative language would be inclined to do (and crash on the first maybe) to add the 1000th and 1001th elements of the every second element of the Fib series, it will only recursively calculate enough times to find the 1000th and 1001th elements of the every second element list (and it will only do one recursive dive not two).

That’s a trivial example but in geometry, say, there can be big wins with that kind of efficiency as a fundamental feature of the language/compilation. As against that modern CPUs are designed for imperative languages with multi-level caching and so on, so it’s all moot. To me the biggest gain potentially in compositional expressiveness that reads well when you return to the code. That and the ability to break problems down into smaller functional steps that each can be tested.

There’s all kinds of complex stuff on how paralysation works in chains of functions and how referential transparency is maintained, not confident to say much more though, the disadvantages often outway the gains in terms of sheer computation efficiency :-)

The other question (multiplexing and port as a type rather than ports be assigned a strict type) I don’t understand.

I could make reference to how Haskell defines datatypes and uses type signatures as part of all function definitions, but I"m not sure that’s particularly helpful in this context.  

[useful_design (Alastair)]

Thinking some more about pure functional programming and Vuo, @MartinusMagneson, one thing revealed to me in a course I did on Haskell is that many functions we write are similar in functional terms to each other, and we could be using more generalised functions over and over, in combination with other generalised functions as function chains and avoid reinventing the wheel all the time with special case iterators. Iterators in Vuo are dangerous in the sense I have to watch myself that I have walls so wrong events don’t feed into the loop itself, completely wrecking the loop execution. There’s lots of traps for beginners I made over and over, and will make again when I get back into Vuo around iterators.

For example the Make Dictionary node in Vuo takes a list of Keys and a list of Values and folds them into a particular kind of data type output we refer to as a dictionary. But you could generalise this function as a fold function, in this case which inputs two lists and outputs tuples. But Fold node might also input a drawer of lists and output any number of arguments per element in the output list.

Fold could be written to not just take in a String type Key list, but any type of data including other functions, say a 3D Object list that gets folded with individualised 3D Object Shader list (if that even makes sense but you get the picture I think).

That’s kind of where functional programming and lambdas in Vuo connect to me, the passing of functions to other functions, to optimise execution only at the optimal time in the composition execution. It will become very compositionally expressive once I become literate with a vocabulary of powerful but generalised functional nodes, and less necessary to remember dozens of very specific use case instances of these nodes for each and every application.

Dealing with node extensions for new datatypes, the Fold node could have a port that accepts a drawer of functions and each of these functions defines Fold for a specific group of data inputs, one for lists of Stings, one for numbers, one for functions, and if the one you want, say 3D Objects doesn’t exist, you can make a sub composition and add that to it’s input of functions to expand the node. The Fold node itself would need to be written such that it accepts a specific function list and has a way of deciding which function to apply depending on the two input data lists. That could happen on the canvas for each instance of Fold it was needed for, or in some meta-canvas function definition place that redefines every instance of the node in that Vuo composition. Just like if you went to the API and edited the C code for Fold. Make sense? Maybe more effort than gain, I’m not sure, but it’s more of a side issue of node expansion than using lambdas in Vuo per se which still could be pretty powerful I suspect, especially to handle lists and lists of non-homogeneous lists of data and functions.

This could be a way to get the FR for “Iteration: Turn most nodes into iterators by allowing single-value ports to accept lists” done without requiring Team Vuo to rewrite all the nodes. We could have a primitive set of functional nodes which accept Vuo sub compositions with lambda output as functions as allowable inputs and lambda outputs. But I’m not sure how hard this will be to implement under the hood… no idea actually!  

[MartinusMagneson (Magneson)]

I’m thinking way simpler in terms of parallelisation! As the “Build/Process List” are highly serial in the way they work, I’m assuming that being able to apply different image filters for instance would mean splitting each transition into its own thread and then have a timer/sync function in the collecting/output node to sync up the processes. I think there is some parallell action going on in Vuo already, but this could mean having greater control over it - which again could mean a possible greater efficiency (or a lot worse as well I guess…).

A struct in C is a collection of items, or a collection of a collection of items. A real number in Vuo for instance is a struct called VuoReal containing a float. Similarly a 3d point, VuoPoint3d, is also a struct - but one containing three floats with the identifiers x, y, z. The way Vuo works internally when it comes to generic types, is that when you write a node you specify the kinds of inputs a node can receive through a generic port by a placeholder. For instance, if you want to specify a variable to be a real number, the code would be

VuoReal number;

. If you want it to act on different types (2/3/4D) in addition to this, instead of creating 4 different nodes, you just specify what a generic type can be used as in the metadata of the node. Then the code would be

VuoGenericType1 number;

. At runtime, as far as I understand, the code then swaps VuoGenericType into whatever type you have attached to the composition input port.

If you look through the code examples and tutorials at api.vuo.org, You’ll for instance see that there are functions to add reals/2/3/4d points. This is so that you can use the same code with the generic placeholder. Instead of using

VuoReal sum = number1 + number2;

, you can use

VuoGenericType1 sum = VuoGenericType1_add(number1, number2);

and have the same node calculate a lot of different structs.

The question then is, can you have functions as structs? Then you could type

VuoGenericType1 whatJustHappened = VuoGenericFunction1(VuoGenericType2, VuoGenericType3);

On a different tangent, the Vuo API is really easy to work with. It looks like it works a lot like how you describe Haskell, although I guess it’s more of an abstraction in terms of what and how it achieves it. Personally, I think it is one of Vuo’s strongest features, and I hope more people get interested in that aspect as well, as it cuts out a lot of limitations you get with other software, and also makes it achievable to use once you have a decent result.

I think it could be a good idea in theory though. As an example, having an “Apply to 3d Object” node with an Object input port, and a generic input port could simplify a lot of tasks. Then you could add transforms, shaders, blending modes etc. through a single node - but I don’t know how difficult or work intensive it would get for team Vuo to change something that seems to be very fundamental to how it works in the first place.

[useful_design (Alastair)]

Thanks for explaining, Martius. Yes I remember when the first prototypes of Vuo came out each different number types had to have it’s own version of any given node process the data, then came the generic types to be a placeholder for nodes combining. I’ve never got into the Vuo API though, even though that was my big interest in Vuo, the idea of being able to make our own compiled nodes to manipulate data or whatever, because while JS in QC was a bit buggy/crash prone and frustratingly slow at some things, learning Obj C to make patches was a big learning curve.

I still think there could be some really nice compositional advantages in having a basic set of nodes that work on lists in interesting ways and being about to send these functions as lambdas to each other could solve /enhance a lot of the feature requests for nodes that can take lists, lists of lists, and a bunch of the structure manipulation FR around dictionaries and so on.

Can C structs take functions references (lambdas) as values? I guess if not directly, Vuo could code VuoGenericType2 that accepts a function reference from other special nodes with λ output ports jut using a special string or unique to Vuo lambda type reference.

Map is a the generalised function (commonly used JS function and in many other languages because it’s such a ubiquitous pattern) that covers the Apply to 3D Object node case. Filter and Sort are others that take specific function inputs to help define their functionality. A set of nodes like this would be quite powerful in Vuo I think and cover a wide range of Yrs for specific nodes, even for people who can code their own nodes.

Please take a look at my FR for Documentation of data types in Vuo and how all work together, common patterns/chains of these datatypes to produce visuals. Be great if you could help put together the buildings blocks, I’m happy to do the graphic design on the diagrams and so on when I understand the relationships. Perhaps we could also look at how general purpose functional nodes like Map, Filter, Sort, Zip, Unzip, Fold, etc etc could be applied to each of these types and how much use that might be.  

[adeveloper]

‘Provide higher-level nodes that encapsulate bigger tasks’

Hierarchical composition is a must for ANY creation tool or language (visual or textual). This fundamental requirement is blatantly missing in Vuo. Compositions with more than 20 or so nodes quickly become unwieldy. At the moment Vuo, to its detriment, manages to hide its actual object-orientated basis.

The Lambda suggestions are useful too but may come afterwards if too complex to implement at the same time. Perhaps they should be a separate request?

[useful_design (Alastair)]

Without getting into a bike shed type argument about it, having strict datatypes as Vuo does makes it more functional than object oriented I think. I’m not sure if it has lazy evaluational the way QC does for example, another characteristic of a functional language. One of the problems though is the types aren’t expansive enough as they stand (no lists of lists for e.g.) and users can only extend types by writing C code.

Any functional language thrives or dies on compositional expressiveness and the ability to create higher level functions from lower level functions. To me sub compositions are not really doing that yet in Vuo, it’s too hard to manufacture higher level functions from lower level ones. With QC type Macros as an intermediary step for development I feel like it will engender more of that kind of coding. I agree having too many nodes on the canvas is a deterrent to major work in Vuo.

I’m not smart enough or skilled enough to know if Lambda’s in Vuo could be implemented successfully or not. I’m just teaching myself Haskell after a 3 day training course for professional coders run by Lab61 in Australia — I was quickly out of my depth but am totally hooked on the concept of pure functional programming as the coding method of the future.