shaders

Shaders
understanding and writing shaders for complex animations with Processing and GLGraphics

Last week-end I took part in a masterclass workshop organized by Processing Paris about shaders. The workshop was brilliantly led by v3ga (v3ga.net) with about 20 participants locked in a large room full of computers for 3 days.

what is a shader

A shader is a program that runs directly on the graphical card (GPU) of a computer. Using information sent by the processor (CPU), shaders are responsible for rasterizing and displaying any graphics elements displayed on a screen. The term shader comes from their first use on older computers, where they were mainly responsible for calculating levels of darkness of surfaces to create the illusion of depth. However, on recent operating systems, they are in charge of all the rendering stages, from simple geometry to camera transformations and the final pixel-color calculations.

why shaders

Since they run on the GPU, shaders are very, very fast. However, being on the GPU also means they are not easy to work with. The syntax we used to write shaders in OpenGL (the default rendering engine in processing 2.0+) is GLSL, a far cry from processing in terms of accessibility. This langage is used both for vertex processing (geometry) and fragment processing (pixels). For this workshop, we used processing 1.5.1 with the library GLGraphics. On one side we wrote our processing sketches and at the same time in another programming environment we programmed the shaders that turned these sketches into pixels.

Screenshot of a text editor (Sublime Text) showing a glsl vertex and fragment shader. frag_vert

sending data from processing

In some cases, it is better to do as much as possible with shaders. For example, instead of displaying an image in processing and then manipulate it in a shader (or worse, in processing), it is much more efficient to send that image as a texture to the GPU and do all the work with shaders. In the example above, the fragment shader (simple_frag.glsl) is receiving three values from processing : mouse, resolution and time. Sending this kind of data from processing to GLSL is simple:


// sending mouse position
theShader.setVecUniform("mouse", float(mouseX), height-float(mouseY));
// sending screen resolution
theShader.setVecUniform("resolution", float(width), float(height));
// sending a texture
theShader.setTexUniform("texture", tex);

compromises

But it is not always a good idea to do everything in shaders. First of all, the syntax is not intuitive at all compared to processing. Also, there is no debugging tools when writing for OpenGL: when a semicolon is missing or a value is out of range, the shader just does not execute and doesn’t tell you where the error might be either (which is a huge pain to deal with). Also, since shaders are running directly on the hardware, it is not possible to run the app with a web browser with processing.js.

Having said that, shaders are really powerful and enable graphically complex programs to run smoothly on most hardware, a must when creating interactive systems. Unloading most of the work from the CPU helps reduce latency between a user action and the system’s feedbacks, and makes the whole experience much more pleasant.
For example, the topmost image is a screenshot of an app where all the pixel work is done in the shaders. It runs at 200+ frames per seconds on 4 year old hardware, and is very reactive to mouse movements.

Also, since shaders are loaded dynamically at each frame, they can be modified or switched with one another while the program is running. This makes it possible to code live and completely change what is rendered with no perceptible delay. I will explore these possibilities in a future article.

a little exploration

The following images are screenshots from Processing/GLSL apps I wrote with v3ga’s workshop examples as references. Touch/hover on the images to see the animation and wait for a few seconds, these gif are about 500kb each. Since they are animated gifs and not programs, they run much more slowly than they do on my machine. When I get the time I will rewrite them in WebGL (probably with three.js by the way).

using a simple textures glslshader_texture displaying an image through a mask glslshader_texture projecting light on a complex shape glslshader_texture light and dithering on a torus knot glslshader_texture projecting dark light on a white shape glslshader_texture dithering the previous shape in real-time glslshader_texture dynamic downsampling and tinting glslshader_texture text deformation with the mouse glslshader_texture text deformation + live input from keyboard glslshader_texture text blur (blur algorithm by grgrdvrt) glslshader_texture

These last experimentations with typography were made with an eye-tracker in mind. Shaders are particularly adapted for eye-tracking because they execute so fast that they can keep-up with a user’s glance at a screen. This setup will be installed soon, and will be the topic of a future article here. In the meantime feel free to comment at the bottom of this post.

references

— The Processing library used in these examples glgraphics.sourceforge.net/

— An introduction to shaders in GLSL gamedev.net/…glsl_an_introduction/

— Projection matrix and camera references songho.ca/…/projectionmatrix.html

— Fantastic shaders running in WebGL with source code shadertoy.com/

— Understanding the implementation of shaders in Processing 2.0 trunk.processing.org/tutorials/pshader/

4 Responses to “shaders”

  1. andres

    Hello, nice write up about shaders in Processing. Just wanted to point out that the posts in my blog about shaders in Processing 2.0 are bit outdated as I haven’t had the time to work on it. However, there is this up-to-date tutorial on the upcoming website for Processing 2.0:

    http://trunk.processing.org/learning/pshader/

    which should reflect the API and naming conventions for the final release of 2.0

  2. louis

    Hello Andrés, I just updated the link, thanks for the feedback!

    And thanks for the outstanding work on GLGraphics and Processing 2.0, I have been working with almost all the alphas and betas of Processing 2.0 and I really like where this is going.

  3. louis

    Hi Till
    I updated the links, thanks for the heads up!

Comments are closed.