3D with Dither

One of the most wonderful features of Dither is that you can write shaders (mini-programs for GPU's) and regular code in the same Dither language, in the same file. Your CPU and GPU code can share functions and constant variables — the compiler automatically figures out which pieces are needed for what. You can even debug shaders by running them on the CPU, where you have the power of print!

Let's make some fun 3D stuff so you can see for yourself!

Overview

Dither comes with a standard library std/g3d for making 3D graphics. g3d's current backends include OpenGL and WebGL, with more on the roadmap. It is supplemented by std/frag, a standard library for working with fragment shaders. frag can be used to shade g3d scenes, as well as on its own to create shadertoy-style renderings, without the need to (manually) set up a 3D environment.

Getting Started

Let's start with the simplest 3D program: an empty window filled with a single color:

First, we include the win library (for window management) and the g3d library. Next we create a 3D rendering context of 200x200 pixels. We initialize g3d by passing the context to it.

In the infinite while loop, we fill the screen with a background color (each RGB component ranges from 0.0 to 1.0). Finally, don't forget to win.poll() to listen for events and refresh the screen (otherwise we'll get stuck in a tight loop!)

Now, let's draw the “hello world” of 3D graphics, a triangle:

We generated a mesh by creating a g3d.Mesh object and filling its vertex list. We set up a camera cam but did not configure it — so everything will be drawn with default coordinate system (X and Y axes from -1 to 1).

In the main loop, we call the draw method of the mesh, surrounded by cam.begin() and cam.end(). The draw method takes a single argument, which is a transformation matrix (called a model matrix) specifying exactly where and at what angle and scale to draw the mesh. Here we pass in a constant identity matrix to mean no transformation need to be applied. (More on transformation matrices later.)

By filling the color list of the mesh, we can draw a colorful triangle:

We only specify the color at each vertex, and the in-between colors are interpolated. For more complicated shading, we can make use of the frag library. But before we start mixing the two libraries, let's try out frag by itself.

Fragment Shader

We introduce frag (and shader-writing in Dither) by rendering a gradient:

We define a gradient function that takes a single argument uv — but notice the @varying in front of it — it is an annotation telling the shader compiler that uv is a special argument passed down from a previous point in the pipeline — when the code is run normally on CPU, such annotations are safely ignored. Anyways, uv refers to UV-coordinates, also known as texcoords, which specifies the mapping between each vertex and a 2D location on a texture.

Here we simply copy the X and Y components (each from 0.0 to 1.0) of the UV coordinate to the red and green components of the output color. The shader is executed per pixel in the GPU, and that's why we see a gradient when all the output pixels are put together.

Next comes the key step: the embed statement takes a function as entry point and passes it to a plugin — in this case one named "fragment", the fragment shader translator-compiler. The plugin generates a string as output (e.g. GLSL code, depending on selected 3D backend), which gets passed into frag.program. frag.program compiles the shader program for later use.

In the main loop, we enable the shader with frag.begin, and call frag.render, which is a shorthand that simply draws a fullscreen rectangle with the shader.

In addition to @varying, we have @uniform, for passing custom variables to the shader.

In this example, we import std/time library to access the number of milliseconds passed since the start of the program, and pass it into a @uniform argument with the frag.uniform method.

We'll learn more about the @varyings, @uniforms (and later @builtins) that can be used in fragment shader later.

A Cube

Now let's draw something that actually looks 3D! A cube.

We all know that a cube has 8 vertices. However here we specify 4x6=24 of them. This is because we don't want to share vertices between faces. In 3D graphics, normals directions are specified per-vertex. Therefore, if faces share vertices, they'll look smooth and blended when shaded, which is great if we have a sphere or something actually smooth, but for cubes, this is probably not what we want.

In addition to vertices, we also specify an indices list. This is telling the graphics backend how we want to connect these vertices. Each of the index refers to a vertex by, well, its index, and by default, we'll be drawing triangle by triangle. So the indices go like: first vertex of first triangle, second vertex of first triangle, third vertex of first triangle, first vertex of second triangle, and so on. For a cube, each face consists of two triangles, so we have 12 triangles resulting in 36 indices.

You might be wondering why there isn't a built-in primitive in g3d called cube (and sphere and cone and so on). This is because we figured there're so many ways you might want a primitive, and that g3d should focus on what's definite and what's essential. That said, the official Dither examples include a file prim.dh, an implementation for all the common primitives, that you can reuse.

We configured the camera by giving it a perspective projection (closer things look bigger), with field of view of 45 degrees, and aspect ratio of W/H. The closest thing and furthest thing that can be rendered is 0.1 and 100 respectively. We placed the camera at {3,3,3}, pointing toward {0,0,0}. The third argument of cam.look_at is the “up vector”, which we set to the positive Y direction.

In g3d, positive Y points upward, positive X points toward your right, and positive Z points out of your screen.

The above cube requires some imagination to look 3D. This is because we haven't shaded it yet, which we'll do next.

Notice how we included the frag library, and had it share the same win context with g3d. In addition to vertices and indices, we have a normals list, which specifies the direction each vertex is pointing toward. For each of the faces, the 4 vertices points in the same direction.

We also created a shader with entry point function normal_mat. This shader simply normalizes the normals passed down, and copies the components to the RGB components of the output color.

In the main loop, we surround cam.begin/cam.end with frag.begin and frag.end. The order is important. Also notice how we replaced frag.render found in earlier shader examples with camera use and mesh drawing. In fact, frag.render is internally implemented as drawing a very simple mesh (fullscreen rectangle).

Let's rotate the cube around to make it look more exciting, as well as to make sure we generated all the faces correctly.

We rotate the cube by updating the transformation matrix used to draw the mesh (model matrix) each frame. Here we introduce the @* operator — a special operator denoting matrix multiplication in Dither. @*= is the shorthand for matrix multiply then assign — the same as * and *=. The g3d.mat.rotate_deg function is a convenience function that generates a rotation matrix that rotates anything multiplied to it around a certain axis by certain degrees.

Notice how the color of each face changes as the cube is rotating — this is because normals are in reference to the world.

In Dither, matrices are row-major and left-multiplied (order matters!). Let's illustrate that.

Here, one cube is translated first then rotated, so it exhibits this orbital motion, while the other copy is rotated then translated, so it just spins by itself somewhere.

We can also add texture to the cube.

First, we need to supply UV-coordinates for each vertex of the cube. We update our shader to sample the texture at given UV-coordinates. We employ std/img library to load pixels and upload it to a frag.Texture, which then gets passed in as a uniform.

Recall that the default mode is to render triangle by triangle, but we do have other options. For example, we can render lines by overwriting the mode field of g3d.Mesh, like the below wireframe example:

Notice that the indices list is also updated to specify pairs of line segment endpoints. Available options for mode include MODE_TRIG_LIST (default), MODE_TRIG_STRIP (continuous strip of triangles sharing adjacent edges), MODE_LINE_LIST, MODE_LINE_STRIP (polyline), and MODE_POINT_LIST.

Materials

So far we've been writing (somewhat) trivial shaders, now let's look at what we can do if we want to render something (somewhat) realistic.

But first, let's “upgrade” our mesh from the cube to something more interesting, so we have a better view of the effects. How about an icosahedron?

This time, we use a function to generate the mesh — the icosahedron has 20 faces so it becomes a bit tedious to duplicate all the vertices for each face. Therefore, we loop over each of the faces and copy the vertices programmatically. Since the vertices are now in order, and each face is just 1 triangle, we no longer need the indices array.

We computed the face normals by doing a cross product on two edges of each face. Finally we applied a normal material like before, to ensure we got everything wired correctly.

Now, let's do the simplest form of shading, the Lambertian, also known as n dot l.

As the name suggests, we take the dot product of n(normal) and l(light direction, or more precisely, negative light direction) and use the result to determine the brightness at each given spot.

Intuitively, this makes sense, because dot product is related to the cosine of the angle, so larger the dot product, smaller the angle, the more aligned the light direction is with the normal, and the more “head on” the light hits the surface, hence brighter the surface.

Notice how we clipped n•l to 0-1 then remapped to 0.2-1. The 0.2 can be regarded as ambient color, so even the darkest spot on the mesh is not pitch black.

Try playing with the "lightdir" uniform, or even animate it!

The Lambertian shading is simple, and great for matte-looking stuff. Now let's look at a slightly more sophisticated model, called Blinn-Phong, which can give us shiny surfaces.

This shading consists of two components: one is a diffuse shading using the same n•l formula, and on top of it, we add a specular shading for the highlights. In theory, the intensity of the specular is related to how aligned are the direction of the reflected light and the direction of viewing (Phong model). However, we take an approximation and calculate the “half vector” (view direction + light direction, normalized), and use the dot product of half vector and normal to determine intensity of specular (Blinn's modification). There's also the shininess constant, which makes the highlight more focused or spread out by raising the specular to a power.

Finally let's look at a simple yet useful shading — depth.

Here we pass the camera's view matrix back into the fragment shader, and use that to calculate each point's position in camera space. then we take that Z and normalize it to given range. An alternative method is also possible, which utilizes @builtin frag_coord's z to calculate Z before projection.

Bump Maps

To have a more detailed-looking mesh, we can, of course, just add more faces to the mesh and makes it actually more detailed. But that comes at the expense of, well, being more expensive. One way to cheat is to use a bump map, which allows us to fake small details on surfaces by perturbing the normals and causing shading to behave as if the details are there.

A bump map can be stored as an image. Each pixel denotes how much to bulge or dent at each given location (kinda like a height map). We can also go an extra step, to generate what's called a normal map, which saves us a step in the shader. But since we're doing everything procedurally anyways, we'll just go with a bump map.

Here we'll be lazy and just generate a Perlin noise for our bump map. This will give our mesh a natural bumpy look. We could also generate our map on the CPU and use write_pixels method of frag.Texture to upload it (if our bump map algorithm contains complicated stuff not easily expressed in parallel manner). But here, since GPU implementation of Perlin noise is built into Dither, we trivially write:

Now, let's put that bump map into use. Before we slap on additional shading, let's just normalize the resultant normals first.

There're quite a few new ideas introduced here.

First, we need to compute local normal edits by sampling neighboring pixels on the bump map and calculating the slope, i.e. how steep or flat it is at that point. This is done in the gradient function.

Next, we need to figure out how to transform those local normal edits to the space of the surface of the cube. To do so we need to compute the tangent — bitangent — normal matrix (or TBN for short). You might think you can nail it just by knowing the normal, but turns out we want things to be aligned with the UV too, hence the math is a bit more involved. We make use of d_position_dx/y and d_uv_dx/y, which gives us the delta change in position and UV for neighboring pixels. All this calculation is done in compute_tbn.

Finally we actually perturb the normals, which is relatively straightforward in comparison. This is done in perturb.

Great, now let's try out the Blinn-Phong shading on our bump-mapped cube. You can see that it looks kinda realistic!

Tweak the strength of the bump map by changing the scale argument passed into perturb. Play with the shininess and spec uniforms too to see what different materials you can simulate!

Shadows

Without shadows things can look as if they're floating and just a bit uncanny in general. However shadows are tricky to render too — it's not like you can just tick a checkbox called “shadows” and now you get shadows for everything magically.

Before we introduce “real” shadows, let's look at some sneaky tricks that are good enough for some situations — maybe those include yours!

First, you can bake the shadows into a texture. In another 3D software (such as Blender), you can use whatever expensive raytracing stuff to render the shadow very nicely, and generate a texture and matching UV coordinates. There're many tutorials for that. Then in g3d, you'd just render it like the textured cube example we showed earlier. We won't be going into further details.

Another method is to do planar shadows. If you just have some objects and they all cast shadows onto a flat floor (and not onto each other), then this method will easily give cleaner-looking shadows.

The idea is pretty simple. We create a special camera, whose projection matrix simply projects the whole scene onto the y=0 plane given a light direction. We draw the scene with that camera and the simplest shader possible: all objects are completely black and the background is white. Then we take that B&W rendering of the scene and use it as the texture for the floor.

The shadow looks quite crisp (“hard”), which might be what you want, but sometimes we'd like softer shadows. This can be achieved by doing a blur when sampling the shadow. Here I also reduced the contrast by changing the fore/background color in the shadow texture.

Now let's look at how to make “real” shadows, that allows anything to cast shadows on anything.

It follows a similar general idea: we first render some sort of texture from the light's view, then use that to somehow inform the drawing of the main scene.

From the light's view (this time encoded as an ortho projection matrix), we render the depth of the scene. This tells us for each point visible to the light, how far it is from the light. This texture is called the shadow map. Then, in the main shader, we transform each pixel's corresponding 3D coordinate into the space of the light, and figures out if the distance to the light is greater than that stored in our shadow map. If it is greater, then it means there's certain point that is closer to the light, hence blocking the light, which means the current point is in the shadows.

We add a small bias to reduce a type of artifact called shadow acne caused by precision limitations. The bias is adjusted according to the sharpness of the angle between the surface and the light at each given point. Try tweaking the amount of bias (or getting rid of it completely) to have fun with artifacts.

Again, we can add some blur to make the shadow look softer, just like with the planar shadows, please try it out yourself!

Text

Let's take a small break from the math-heavy stuff and look at text drawing in g3d.

Let's start with drawing some 2D text on the screen.

Let's have some text of various position, scale and even rotation:

Notice how you can give a full transformation matrix for posing the text — you can actually place it at different depths, and if the depth buffer has not been cleared (with background), then it can be occluded by 3D geometry! This is very handy for drawing info that follows moving objects in the scene. See the following example:

Here we computed the projection of every vertex we wanted to label by reusing the camera's matrices. This ensures the depth calculation is consistent and occlusion are rendered correctly.

The text doesn't have to face the screen — we can also flap it around like any mesh:

Conclusion

We've looked at quite a few things with Dither and 3D. While this is not a comprehensive guide, we hope it provides a good foundation for you to figure the rest of the things (such as different type of light sources and multiple copies of them) out.

Let's put everything we've learned together and make this neat little demo: