Lists within lists

Yes, lists of lists!

Out of curiosity, @smokris or @jstrecker, will the (eventual) implementation let us toss around lists via cables like any other type of data, or will there be some important differences?

I’d love to see the ability to treat a list like any other piece of data, doing away with the distinction between, for example, the Share Value and Share List nodes.

I’m wondering if you can’t sort of do this already… maybe I’m wrong but if a list is a bunch of elements you could assume structure to the list were no actually is evident in the list.

say the list had 120 elements. you you define each element in the top level list as having 120 elements. that would make each element have 10 of it’s own elements. (Stay with me here…)

The first, say, three items of each top level element could be (a, b ,c) followed by elements 4-10 null if not required (d, e, f, g, h, i, j). Now (a,b,c) could describe the indexes for a 3 dimensional Array, while (d, e, f, g, h, i ,j) could be the data elements.

I think the only change that would need to be made to Vuo and list handling to do this would be to allow different datatypes in lists. I don’t know if that is allowed, not allowed or not disallowed but also not possible with existing Vuo nodes. I imagine one code write a node to put various datatypes into the elements if a new class of datatype was created to handle it. Almost all current nodes would not be able to receive such a deep list — only a couple would get modified to accept such complex data.

We would need a new Make Deep Listnode to accept multiple single element lists of various things incoming and building the master list from them and metadata about the nature of the construction from other value ports. It’s how I imagined I would create my ‘triangle structures’ referred to on other Q&As i posted with 3x 3D points for each vertexes along with a colour value for each vertex. Then write a node to fiddle with some of the positioning using transforms etc either before initial master list is made or by modifying it (so either before Build a List or using Process a List and clumping 6 or more elements as blocks of data when I process it, each block describing a vertex and its colour) Then parse in a final node that converts the data to scene objects for rendering.

Lists already have a datatype other than ‘list’ associated with them obviously since we can have a list of integers as well as a list of 4D points and the Editor knows the difference. This in a sense is List metadata. Maybe we just need a bit more List metadata to describe more complex set ups.

Team Vuo could create two new datatypes, one of deep list or matrix (but it’s a multi-typed matrix not the typical kind where all elements are the same type) and another datatype say multi-list for the deep list elements of, say, 10 items where the first 3 or 4 are positive integers that represent the array indexes (a, b, c) allowing say a 3 deep lists of lists of lists, and the following items in this object could also be predetermined so Vuo always expects the same basic datatype at each element, so d is real; e is text; f is 2D point; g is 3D point (which colours can be converted into); h is 4D Point (which CMYK colours can be in); i is image; j is layer elements… and so on. Or maybe just allow them to be anything and the user must set up the code to process them carefully.

Normal list handling nodes wouldn’t know how to handle these Deep Lists but we could write our own nodes to peak and poke at them and perhaps Vuo could provide basic Make a Deep List and Get Elements from Deep List and Choose element from Deep List and Sort Elements by Index Orders from Deep List or Extract Data from Deep List nodes both of which would output a conventional list from a subset of the full data contained in the element. Plus a Change an Element in Deep List using Indexesfor poking at a bit of data in a single Multi-list element in the Deep List.

@Pianomatic you mentioned that @jstrecker demoed a way of constructing a list using the Share List node somewhere else (forget where). Wondering if either of you have a link handy, if not no probs I expect I’ll figure out how at some point.

I’d love to see the ability to treat a list like any other piece of data, doing away with the distinction between, for example, the Share Value and Share List nodes.

Philip, yes, that’s exactly what we had in mind :)

I’m wondering if you can’t sort of do this already… maybe I’m wrong but if a list is a bunch of elements you could assume structure to the list were no actually is evident in the list.

Alastair, yes, one could build subcompositions or C nodes that input or output lists that have a certain structure, implementing a complex data type such as list-of-list on top of the simpler existing list data type. This could be a workaround until true lists-of-lists are implemented.

Ideally it would be better to be able to treat lists, lists-of-lists, etc. like any other data type, as Philip said.

Hi have no credit anymore to vote on this… but +1 by me! Lists of lists, or in other words, lost treated as every other kind of data type, is a must-have feature for me.

This is now my second composition where I got stuck because of missing this.
Casted 130 votes. Hope it helps!
Certainly implies a lot of work, but would be a game changer for Vuo.  

2 Likes

Oh wow. Can’t believe I suggested this in 2015.

Thanks for the votes sinemod. Hopefully this gets looked at soon!

1 Like

Let’s bump this up! 10 new votes!

1 Like

Woohoo! List-in-list to the moon! ;-)

Here are my 10 votes! :-)
Anyway, this feature is already scheduled for 2.5.0 release.

1 Like

Oh gosh. Didn’t know it was already scheduled! Save your votes then. :grimacing:

Feature status is not updated yet ;-)

1 Like

Yep, we (Vuo team) are still strongly thinking of adding lists of lists in Vuo 2.5.0, more so now with the recent votes. The plan for next releases is probable but not set in stone. We’ll be marking feature requests as “chosen” when we can definitely commit to them being in the next release.

2 Likes

do i vote or hold on to my chips @jstrecker ?!

yes, also running into this, when parsing xml… voted

related: I’d like to sort lists on (attributes of) nested elements…
We can then run ‘layered dynamic content’ into a composition when importing (or refreshing) xml from a database (with users interacting in real time via the web to drive content to our vuo app… exciting ;-)

I have a question for @teamvuo/@jstrecker, lists in Vuo currently have to be homogenous (AFAIK)? So this will apply to lists of lists too right, all elements in any list of lists will be of the same (predefined elsewhere) datatype?

Most XML files have heterogeneous datatypes in them (often serialised as strings), so will need to be parsed outside of any list of lists to get say, real numbers from the data right?  

Sorry @sinemod I simplified the question, so your response doesn’t make sense now.

What I was talking about in original version of the question is that in many programming languages there are ways to make heterogeneous definitions of types, data-types or classes.

In Haskell for example it’s trivial to create a new type which may be a combination of more elementary types, say a pair comprising of a string and an integer number value

e.g. ("pineapple", 5)

nothing special is required to make a list of instances of that type

e.g. [ ("pineapple", 5), ("apples", 1), ("oranges", 0), …]

the only way to create a new datatype in Vuo is to do so by making a node or nodes in C (I assume). Would Lists of Lists benefit from some nodes to create heterogeneous datatypes that can be used in lists?  

Another question for @teamvuo / @jstrecker, will Lists of List automatically allow Lists of Lists of Lists, simply because it’s a recursive approach to lists, or would that require extra coding? I note that recursion is included in @alexmitchellmus’s FR, so assuming it’s included unless we hear otherwise.  

@useful_design,

Lists in Vuo currently have to be homogenous (AFAIK)? So this will apply to lists of lists too right, all elements in any list of lists will be of the same (predefined elsewhere) datatype?

Correct.

Most XML files have heterogeneous datatypes in them (often serialised as strings), so will need to be parsed outside of any list of lists to get say, real numbers from the data right?

Vuo’s existing Tree data type and related nodes handle that.

Will Lists of List automatically allow Lists of Lists of Lists?

That’s the plan.  

1 Like

wishing for this today…

not sure how the tool tips /floating data previews are going to cope with lists of lists without a code write to give lists of lists more screen space.

lists of heterogeneous data for generating 3D objects like point clouds, triangle families etc is a pretty approach in QC. eg 3D points along with their RGBA colour per node, real value for size etc etc.

Be good to start thinking how this can eventually be accommodated in lists and lists or lists, rather than mapping it to a JSON tree. I guess that is going to make for problems with general nodes being able to accept lists on value input ports, but maybe Vuo editor just blocks complex type lists (for e.g. of type [(3D point, RGBAcolor, size value as a Real)] ) and only allows lists of the relevant data type(s) for any given input port. An advanced editor might let the user insert a conversion node the same way a 3D point gets converted to a 1D point where appropriate ATM.