How does one receive the current time, width and height values into a composition when one has selected "new shader", then "image generator"?

When I write:

uniform float width, height; void main() { vec2 uv=gl_FragCoord.xy/vec2(width,height); gl_FragColor = vec4(uv.x,uv.y,.5,1); }

...then run the composition, it appears that the shader does not receive the current window width and height values from the VUO editor app.

Comments

What do you mean by “running

George_Toledo's picture
Submitted by

What do you do to run the code?

It is just a fragment shader, there is no vertex shader, so I am very confused by that error.

It seems like you must have done something to link it to a vertex shader to get that warning. That said, I think gl_FragCoord doesn’t have to be defined within the vertex shader because it is part of the GLSL language.

I really appreciate you looking at this, I am still unsure why it doesn’t work.

Why is all of this ISF stuff part of the shader? Is it not possible to use regular GLSL version code?

Also, I do not understand how to receive width, height, and time values, while writing the shader in VUO. Does the VUO Editor not send these values when writing a protocol, the way (sorry) QC did sent audio data for music visualizers, x/y for image filters, and other things like that? I see these ports, but the shader does not seem to receive the values from the composition runner.

I think Jaymie have to answer

MartinusMagneson's picture
Submitted by

I think Jaymie have to answer the more technical questions as to why this is so, but I'm just running the shader either as a node or in the shader editor. Note that this is shown in the MacOS console not the Vuo console (that I forgot about). You can just filter the MacOS console in the top right corner with "Vuo" to get rid of most of the irrelevant OS stuff and get any code-related warnings there.

There is also a quick GLSL/ISF reference in the view menu where the input uniforms are listed. Might be worth a read-through if you haven't looked at it yet :)

Thanks Martinus!

George_Toledo's picture
Submitted by

Thanks Martinus!

What do I do to run it as a node? Guessing the file has to go somewhere?

I did notice the ISF methods listed in the panel to the right of the editor, but it hadn’t occurred to me that the shader might not be able to use typical GLSL functions such as gl_FragCoord…but it may be that the problem there is just that width and height isn’t being received by the shader when running the shader editor.

Just put it in the user

MartinusMagneson's picture
Submitted by

Just put it in the user modules folder (tools -> open user modules folder, or something like that), and then it should be available as a node in the library.

I struggled a bit with getting inputs to work, but I was in a bit of a rush. Might be able to look more into it tomorrow. The cool thing about it being ISF compatible is that it should be relatively easy to port or tweak MadMapper materials over to Vuo, or vice versa.

some libraries/porting info (for MadMapper but might be relevant?): https://editor.isf.video/shaders?q=&category=&sort=Date+Created+%E2%86%9... https://projectileobjects.com/2020/10/22/isf-shaders-and-madmapper-mater...

Ok, so it is pretty easy, a

MartinusMagneson's picture
Submitted by

Ok, so it is pretty easy, a published port is automagically declared as a uniform when you add it. The resolution is RENDERSIZE.xy, and the time input is TIME (both capitalized).

This will make a color based on time and render size along with a published Real port named blue:

void main()
{
    float red = mod(TIME, 1.0);
    float green =  RENDERSIZE.y / RENDERSIZE.x;
    gl_FragColor = vec4(red,green,blue,1);
}

Interesting. So those are

George_Toledo's picture
Submitted by

Interesting.

So this is after loading into the modules folder that you can see something working?

When you open a new shader composition from the editor, and there are the input ports there already in the protocol comp, are you seeing the shader able to get receive the time value, or the other values?

Oh, I see it working by

MartinusMagneson's picture
Submitted by

Oh, I see it working by running it in the shader editor (like a composition). Time and resolution are supported without declaring any uniforms. If you use the Image filter protocol/editor there is also provided an image of a bird when you run it standalone.

The code above is the only thing you need to input for a simple shader utilizing time, resolution and inputs. It will modulate the red channel by time, the green channel by the rendered window size, and the blue channel by a published port/slider.

I have to admit, I am

George_Toledo's picture
Submitted by

I have to admit, I am completely confused at this point.

Do the capitalized TIME and RENDERSIZE you refer to in your shader code directly correspond to the pre-existing time, width, and height ports that are at the right hand side of a shader protocol? And the data is coming through the ports? Or are these uniform values somehow supplied by an unseen vertex shader that exists somewhere and is linked to the fragment?

Very cool.

jersmi's picture
Submitted by

Very cool.

Super basic -- next I tried changing red to:

float red = (sin(hz * TIME * 2PI) + 1.0) / 2.0;

But using #define 2PI 6.2831853076; at the top generates an error.

Yes - to both the questions I

MartinusMagneson's picture
Submitted by

Yes - to both the questions I think actually. If I have understood it correctly, a fragment shader always needs a vertex shader to display anything (at least in Vuo). My guess is that the vertex shader is just a very basic quad and some default uniforms that sets up a framework for less boilerplate coding.

At the risk of confusing you further, if you look at the code for a custom GLSL image filter node in the Vuo source (that is the C source, has nothing to do with the shader editor), you can find the following line setting up the shader:

VuoShader_addSource(instance->shader, VuoMesh_IndividualTriangles, NULL, NULL, fragmentShaderSource);

Looking at VuoShader_addSource in the API, it lists the inputs to the function as follows:

void VuoShader_addSource (  VuoShader   shader,
    const VuoMesh_ElementAssemblyMethod    inputPrimitiveMode,
    const char *     vertexShaderSource,
    const char *     geometryShaderSource,
    const char *     fragmentShaderSource 
    )

With this in mind, it can seem like there are no vertex shader for the fragment shader that does the filters, but in the notes of the function we can also read:

GLSL vertex shader source code. If NULL, a default vertex shader is used (it passes texture coordinates through, and projects the vertex).

Vuo Shader example: https://github.com/vuo/vuo/blob/main/example/node/imageFilterGLSL/exampl...

VuoShader_addSource in the API: https://api.vuo.org/2.3.2/group___vuo_shader.html#gac8d913c0f6523783ba6b...

Edit: this works (with ports

jersmi's picture
Submitted by

This works (with ports added for blue and hz):

const float TWOPI = 6.2831853076;

void main()
{
    float red = (sin(TIME * TWOPI * hz)) + 1.0 / 2.0;
    float green =  RENDERSIZE.y / RENDERSIZE.x;
    gl_FragColor = vec4(red,green,blue,1);
}

Vuo is a little slow to update glsl error reporting at the bottom of the shader comp, need to close the comp, maybe save/close/reopen, etc.

Oh, alright. Yeah, I saw your

jersmi's picture
Submitted by

Oh, alright. Yeah, I saw your bug post. Thank you! Above code worked for me with hz (after declaring in code, I guess). I'm a GLSL newb....

Edit: #define TWOPI 6.2831853076; generates "syntax error: ';' (Fragment line 5)" (<-- line with red, of course)

const float TWOPI = 6.2831853076; works fine.

And regarding error reporting, updates when comp runs, no problem.

(note: Just found that jpeg

George_Toledo's picture
Submitted by

(note: Just found that jpeg has to be turned to jpg to attach.) Just to make sure I am being clear...

I have started by selecting to make a new shader with an image generator protocol.

I then see what is below.

My next step is to just attempt to use time as a variable in the code, assuming it is implicitly a uniform given that I see it on the green "Image Generator" interface to the right. This creates an error that time is an undeclared identifier.

Next I declare it as a uniform. No more warning, but there seems to be no data from the time value being supplied to the image generator protocol.

All that happens when the composition is run is that the viewer is a static blue output, which would not happen if the time value on the right side of the composition was being obtained by the shader.

...so, I wish to understand

George_Toledo's picture
Submitted by

...so, I wish to understand what to do to obtain these values in the shader when attempting to write a shader.

I understand that it would be odd to create arbitrary uniforms/input values and receive them in the shader in this kind of editing situation, because there is no way to create nodes to supply values.

But am I correct in thinking that all of the protocol ports are supposed to get some data supplied from the VUO editor, that one can "do stuff with" when programming, to see how things work? Is there any way to get live time, width and height data input port values I see at the right hand side of the editor, supplied into the shader, while working on this specific kind of protocol file?

Attempting to be clear that I

George_Toledo's picture
Submitted by

Attempting to be clear that I am talking about the time value on the protocol input port to the right of the screen.

If “TIME” is capitalized and it is working, it is not the same “time” variable that corresponds with the protocol input port for time, it appears to be a value that the shader can receive via standard uniforms, without receiving any data via ports.

What I am trying to understand is, when a shader is created this way, there are these time, width and height ports existing as part of the protocol. How do I get these exact port values into the shader to work with?

The protocol composition appears to be receiving some sort of values, because it does init on viewer run. So some sort of fire event has to hit it, I think? Why does it appear these protocol ports receive no data? Is it expected that no values flow in this scenario; they aren’t supplied by the editor? If so, what is the purpose of the ports existing by default?

Do you have to work on a shader a bit, then save and load it in a modules folder, and then hook it up to a new test composition to send the time, width, and height port values into the node? Is that the idea of the workflow?

I am talking about the time

jersmi's picture
Submitted by

I am talking about the time value on the protocol input port to the right of the screen.

I think you mean left of the screen in the "Image Generator" green box, along with width and height....

Do you have to work on a shader a bit, then save and load it in a modules folder, and then hook it up to a new test composition to send the time, width, and height port values into the node? Is that the idea of the workflow?

I bet you are right about this -- that is, ports become active when the shader is loaded as a module into another (parent) comp. (Then ideally editing the shader + save would update in the parent comp -- not sure about this, I'll test).

At any rate, I appreciate the time you (and Magneson ) take to spell it out here. These are valuable bits for those of us with much to learn.

Usually…and of course I may

George_Toledo's picture
Submitted by

Usually…and of course I may be wrong…the point of creating any kind of protocol of this type is that these graphs will be utilized by an application which runs the protocol.

So, when protocol input ports are a default part of a given protocol, it seems expected for there to be values supplied to them by the host app, which in this case is the VUO editor app.

In QC, when one selects a given protocol, there is a variety of data that the composition can receive from the input ports, which are supplied to the composition from the QC editor app. For example, x/y mouse events are forwarded from the viewer window to the composition. Image data is supplied to compositions. Audio struct and float data are supplied to compositions.

This way, when someone is working on a protocol they can see if the code they are writing works.

I don’t mention this to suggest anything has to necessarily happen exactly the QC way of things, only to attempt to figure out how to achieve a similarly functional workflow. It seems like driving blind if one cannot obtain the protocol values while creating a protocol composition, but again, I feel that maybe there is a way to do this I am overlooking.

If “TIME” is capitalized and

MartinusMagneson's picture
Submitted by

If “TIME” is capitalized and it is working, it is not the same “time” variable that corresponds with the protocol input port for time, it appears to be a value that the shader can receive via standard uniforms, without receiving any data via ports.

What I am trying to understand is, when a shader is created this way, there are these time, width and height ports existing as part of the protocol. How do I get these exact port values into the shader to work with?

The time port, the TIME variable along with width and height are in a way just arbitrary input ports. If you save your shader to the library, they will work as regular input ports for their type, and they won't act on anything unless they get some sort of data/event. However, when you edit/run the shader in the shader editor, these values will be provided to you through those ports. If you run the code I first posted, and change the width/height of the rendering window the green amount will be decided by that relationship. In short, I think they are pre-defined uniforms that are reserved for the editor so that it can take over those for easier programming :)

For most published ports, the

jstrecker's picture
Submitted by

For most published ports, the variable name is the same as the port name. The exceptions are:

  • width port -> RENDERSIZE.x
  • height port -> RENDERSIZE.y
  • time port -> TIME
  • image port -> inputImage

I've made a note that we should improve the documentation, and possibly provide the more expected variable names as aliases. The reason for the odd variable names is that they're from the ISF specification.

gl_FragCoord works in Vuo shaders. I guess it just looked like it failed because of width and height.

In case you haven't seen it, the manual section on Turning graphics shaders into nodes might provide more context about running shaders and knowing the available variables.

Why is all of this ISF stuff part of the shader? Is it not possible to use regular GLSL version code?

In QC, there's the GLSL patch. You can have different instances of the GLSL patch containing different shaders.

In Vuo, there's not one universal GLSL node class as a container. Instead, each shader is its own node class. This means the shader can have whatever properties a regular node class has — keywords and a description that appear when you search the node library, port details like the default value and the min/max for the slider in the input editor.

In order to support those properties, we needed some sort of metadata format for shaders. Since VIDVOX had already specified such a format with ISF, we decided to use that rather than creating yet another format.

keywords and a description

jersmi's picture
Submitted by

keywords and a description that appear when you search the node library

Jaymie, can I add this stuff to a .fs file in the modules library?

Edit: nevermind, got it -- Edit > Composition Information

Jaymie,

George_Toledo's picture
Submitted by

Jaymie,

To make sure I follow, I think this is the correct question to ask...

If I use TIME in the protocol shader composition, I am now getting an animated output when I run the composition:

void main() { gl_FragColor = vec4(.3+sin(TIME),.4,.5,1); }

If I understand correctly, when I take this and load it in some modules folder to then pull it onto the editor from the library and use in a new composition, I will see the node with time, width and height ports.

When I feed a value to the "time" port it will actually correspond directly with the "TIME" value though they are written in different case? And if I use "RENDERSIZE.x", that the width port will then inform the "RENDERSIZE.x" value when used in code, though they are different variable names...they map to each other 100%? It is pulling the values directly from these ports, not from the editor runtime/some unseen vertex shader, etc?

I wouldn't necessarily characterize the ISF names as odd, it is that the names on the Image Generator input ports that come up by default when creating the protocol comp aren't the same names which (for me) makes the relationship unclear. I wouldn't have thought that ISF uniforms would have any connection to the key names on the input ports, unless maybe they were the same names. I was thinking they were independent of one another until this moment. It's understandable that since the Image Generator protocol already existed, you don't want to use these ISF variable names...unless shader subtype this was a totally different protocol type than the existing Image Generator, maybe.

Thanks for all of the feedback on this Jaymie (and everyone else!!!), hopefully it is discussion that people will find generally useful when using this feature.

If I understand correctly,

jstrecker's picture
Submitted by

If I understand correctly, when I take this and load it in some modules folder to then pull it onto the editor from the library and use in a new composition, I will see the node with time, width and height ports.

When I feed a value to the "time" port it will actually correspond directly with the "TIME" value though they are written in different case? And if I use "RENDERSIZE.x", that the width port will then inform the "RENDERSIZE.x" value when used in code, though they are different variable names...they map to each other 100%? It is pulling the values directly from these ports, not from the editor runtime/some unseen vertex shader, etc?

Yep! When the node executes, it takes the values in the time, width, and height input ports and passes them to the fragment shader (as TIME and RENDERSIZE) through glUniform calls.

Can/should "fs" files generated by VUO be shared in the composition gallery?

Sure! We just modified the upload form fields of the composition gallery and node gallery to accept .fs files. Generally the node gallery would be good for presenting a shader as a component that people can take as-is and reuse in their own compositions, while the composition gallery is more suited to sharing artistic work, like if you're using a shader within the context of a composition.

Ahh so it works through those

George_Toledo's picture
Submitted by

Ahh so it works through those being differently named uniforms than the input port names, with the data from said ports getting passed to the shader via the uniform path. I think I got it. Thanks for the explanation.

I have been finding this shader editor mode to work very well, regardless of the confusion I have had about this particular issue.

The question about posting an

George_Toledo's picture
Submitted by

The question about posting an fs to the gallery was related to the fact that I was unsure if the fs generated would open up as a VUO composition when I posted that question. But I now see that when I drop it into VUO, it does open in the composition editing mode that I see when creating it.

Pages