schmelczer.dev

A Fleeting Garden on Your GPU

A WebGPU slime mould simulation. Control millions of agents with your paintbrush and watch them bring your drawing to life.

4 min read
Magenta, violet, and yellow agent trails looping into a mirrored, kaleidoscopic mesh on a near-black canvas.

Watching millions of independent agents converge into art is fascinating for a while, but really not that long. My favourite art pieces make the viewer part of the work, so this project had to be interactive to live up to my expectations. That was the hard part. The rest was a slime mould simulation inspired by Sebastian Lague’s video1, of the kind Sage Jenson has taken furthest2. I built it while the WebGPU spec3 was still a moving target, because I’d felt WebGL’s limits while ray-tracing 2D scenes4 and wanted compute shaders as a first-class concept.

Architecture

The agents’ lifecycle is managed by a compute shader updating an agent array in place. Each individual follows a simple decision-making loop: on every iteration it takes three samples, one ahead of its position and one to each side, then chooses between going straight, turning left, or turning right. Each theme of the app carries a matrix specifying how the agent types relate to one another, either attracting or avoiding each other, or staying indifferent:

3 by 3 matrix showing the interactions between 3 trail types

The samples come from a map that keeps the history of the agents’ paths and slowly diffuses over time. It’s a texture where the R, G, and B channels each correspond to one agent type’s deposits. I considered using the fourth channel too, but three colours already end up busy enough, and a fourth would’ve made colour theory a fair bit harder for my colour-blind brain.

Make it good

My first experiment let the user draw over the diffusion texture, swaying the mould into changing direction. It turned out to be one layer too detached: it barely changed the behaviour unless the weight was so high that the strokes looked thick. In the end, drawing only spawns new agents on the pen’s path. When there are too many agents on the canvas, old ones get removed as new ones arrive.

Drawing was really fun, but I can’t draw well, and there isn’t enough line smoothing in the world to change that (though adding some did improve my work a bit). So I also added mirroring, a fairly popular tool in drawing programs for turning scribbles into artsy kaleidoscopic symmetries.

Then, to deepen the immersion and give drawing more weight, I added procedurally generated piano sounds that reward more drawing with more complex note progressions.

A lot of minor touches give the app its polish. Theme changes go through the history stack, so the back button works as expected. The toolbar reads the pixels behind it, a 13×7 grid of them, and adjusts its own opacity so it stays legible against any background. And of course, the first thing a visitor sees is 180,000 agents drawing the word Fleeting out of their own trails, then wandering off as the simulation takes over and the letters come apart.

Then make it fast

The simulation is split across six compute-and-render stages, spread over ten WGSL files: agent step, diffusion and decay, brush, eraser (one variant clears the trail, the other kills agents), agent generation with resize and compaction, and the final palette render. A calm frame only touches three of them (agent step, diffusion, and render), and the whole frame goes into a single command buffer with one submit.

The code is kept simple and mostly branch-free; three larger decisions do the rest. Agents write their deposits without blending, because concurrent compute writes to a storage texture don’t blend anyway: when two agents land on the same pixel, one write simply wins. The next diffusion pass smears the survivor across its neighbours, so the lost one is impossible to spot. Keeping the agents contiguous took more thought: each workgroup prefix-sums its own survivors, so compaction costs one atomic per workgroup rather than one per agent. And the agent count follows the frame rate. It starts at a million, sheds 200,000 a second whenever the smoothed rate drops below 90% of 60 FPS, and never falls below 50,000, so a weaker device gets a thinner garden instead of a slideshow.

Give it a try

There’s only so much a blog post can do to describe an immersive art experience. Ultimately, the best way to learn more about it is to give it a try, recommended with the sound on: fleeting.schmelczer.dev5.

  1. Sebastian Lague’s video: https://www.youtube.com/watch?v=X-iSQQgOd1A↩
  2. Sage Jenson has taken furthest: https://cargocollective.com/sagejenson/physarum↩
  3. WebGPU spec: https://www.w3.org/TR/webgpu/↩
  4. ray-tracing 2D scenes: https://schmelczer.dev/articles/sdf-2d-ray-tracing/↩
  5. https://fleeting.schmelczer.dev/↩