A node for loading structured data from file, URL, etc.
Imports: XML, JSON, CSV, TXT, etc. for parsing
Filters: ability to filter structure data streams
Multi dimensional: ability to have nested data (not just 1-dimensional lists)
Could go as far as ability to connect to DB’s but that’s probably pushing the complexity of this feature.
I see now that the FAQ and Roadmap pages lists “File I/O” as a possible 0.5.10 feature and the “Dictionary types” for 0.5.9, which hopefully cover most of the ideas of this feature request!
Nice.
The DB connection reminds me of my only foray into Kineme/QC development, which I never finished. :(
@joeladria: You’re right. We’re already planning to implement much of this during the next few releases. Let’s reexamine your structured data needs after Vuo has dictionary support.
Could you submit a separate feature request for database connectivity? Which databases would you like to connect to (ODBC, SQLite, …)? If you could also provide descriptions of a few scenarios in which you’d use it, that’d be great.
Hey Joeladria,
We looked into what it would take to pull this off — it’s no small task! We’re interested in making this happen but won’t be able to pursue it until well after 1.0. We’re going to table this feature request for now and look at it again after 1.0. Thanks!
Now that we’ve released Vuo 1.0, we’ve opened this feature request for voting.
I think you don’t need all kinds of formats(xml, cvs, json) supported. a simple txt file i/o would be fine. Existing nodes such as split text could be used extract the data to a list.
This is an absolute must in my view, things like storing and loading preferences, logging events and so on.
v.
As discussed with @jstrecker here on this Websocket feature request : https://community.vuo.org/t/-/4616
I was wondering if this would include HTML structure parsing for websites ? Something like XML parser from the Kineme Network Tools ?
Or should I make a seperate feature request for online website structures ?
And as @volkerku said, before some specific nodes for every kind of different structures, wouldn’t it be great be able to open local or online XML, Texts, DB’s HTML and JSON things and retrieve some rough unsorted (or savage parsed) I/O text file where we could extract some data using text sorting descriptors or structure get items ? A faster easier node to wait for some more powerfull one’s.
@Khenkel released a text file open node as far as I remember, but it will only open real text files, and not turn all files into rough I/O text.
So this would be a big mix of the Kineme File Tools and the Network Tools
@bodysoulspirit: Yes, this feature request includes the ability to retrieve data via HTTP, and the ability to parse XML.
1 - What is understood with “Add filters to Parse” ?
Will it allow us to retrieve some specific data from those structures ? O can I encourage people who voted on this to also vote for Request by @Smokris here : More List Nodes
2 - @Smokris : If this will load HTTP too, could you please update the title for this feature request too ? Might be useful for people to vote for this !
@bodysoulspirit, I updated the title to mention HTTP(S). For structured data loading to be useful, yes, we’d need to implement some additional list nodes. When we’re plan the release that includes structured data loading, depending on the voting results, we’d either implement that entire other feature request, or just the most important parts needed for working with structured data.
This is one of the last major features I’m looking for to complete the transition from QC to Vuo for our major annual project that I direct and produce. Really hoping this is implemented by the fall so I can finally finish the transition and drop QC for next year. Here’s hoping!!!
This feature would greatly extend Vuo’s usefulness, and genuinely add value. I would be using it tonight, instead of hacking together a Serial bridge in Python. :)
(Don’t get me wrong - not hating on Serial - Serial support super helpful and very glad to have it.)
Hey, thanks again for all the amazing features you’ve been adding to VUO over the past months! I can see you’ve all been very hard at work adding some amazing features to the Nov 2015 release! I figure “Loading Structured Data” hasn’t even been slated yet but I’d like to say that I am still VERY interested in using XML importing and parsing for a large project, then I can completely replace QC. It would be amazing if it can be put on the list for the next update cycle! :)
+1 Chris!
I’m so needing file input that I am considering adding data to a custom node! ;-)
Most of the libraries are already used in Vuo correct? I think I saw LibXML2 show up somewhere?
We weren’t able to get this entire feature request into the upcoming Vuo 1.2 release, but we were able to implement one key piece: reading text from a file. With the existing text nodes, you’ll be able to do some rudimentary parsing (split by line, split on commas).
I don’t want to put my foot in my mouth by promising anything prematurely, but I can say that since this feature request is one of the top-voted ones (#7 overall, #4 among features up to a few weeks of work) it’s high on our list of candidates for Vuo 1.3.
alexmitchellmus, keen observation. Yes, the Fetch RSS Items node does some XML parsing. And we already use the json-c library in many places.
Awesome Jaymie!
Chicken or egg kind of stuff- but list in list is a very necessary data type for this feature as well. No point loading in 1D values… Always more fun with friends. Keep up the great work.
Is the load text file in alpha 4?
list in list is a very necessary data type for this feature as well.
Yep, adding structured data support beyond 1D lists is one of the challenges of this feature request.
Is the load text file in alpha 4?
Yes indeed.