A Fleeting Garden on Your GPU

Watching art emerge from millions of independent agents is fascinating for a while, but not for all that long. My favourite art makes the viewer part of the work, so this project had to be interactive to live up to my expectations. That was the hard part. The rest was a slime mould simulation inspired by Sebastian Lague’s video¹ and Sage Jenson’s work². I built it while the WebGPU spec³ was still a moving target because I’d reached WebGL’s limits when ray-tracing 2D scenes and wanted first-class compute shaders.

Architecture

A compute shader manages the agents’ lifecycle by updating an array in place. Each agent follows a simple loop: every iteration, it takes three samples, one ahead and one on either side, then goes straight, turns left, or turns right based on which sensor reports the strongest trail scent. Each theme has a matrix that defines how every agent type responds to every other type’s trail: attraction, repulsion, or indifference.

The samples come from a map that records the agents’ paths and slowly diffuses them over time. Its R, G, and B texture channels each hold deposits from one agent type. I considered using the fourth channel too, but three colours already make the result busy enough, and a fourth would’ve made colour theory a fair bit harder for my colour-blind brain.

Make it good

My first experiment let the user draw over the diffusion texture, nudging the mould to change direction. That proved one layer too indirect: it barely changed the agents’ behaviour unless the effect was strong enough to make the strokes look overly thick. In the final version, drawing spawns new agents along the pen’s path. When there are too many agents on the canvas, old ones get removed as new ones arrive.

Drawing is great fun, but I can’t draw well. No amount of line smoothing will change that, though it helped a little. So I added mirroring, a common drawing-tool feature that turns scribbles into artsy kaleidoscopic patterns.

To deepen the immersion and give each stroke more weight, I added procedurally generated piano sounds that reward drawing with increasingly complex note progressions.

I’m a firm believer that small touches give the app its polish. Theme changes go through the history stack, so the back button works as expected. The toolbar samples a 13×7 grid of pixels behind it and adjusts its opacity to stay legible against any background. The first thing a visitor sees is 180,000 agents drawing the word Fleeting with their trails, then wandering off as the simulation takes over and the letters fall apart. Together, these details make the whole experience feel guided and well thought out.

Then make it fast

The simulation is split into six compute-and-render stages across ten WGSL files: agent step, diffusion and decay, brush, eraser (one variant clears trails, the other kills agents), agent generation with resizing and compaction, and the final palette render. An idle frame touches only three stages: agent step, diffusion, and render.

The code is simple and mostly branch-free. Three other decisions help keep the simulation performant. Agents write deposits without blending because concurrent compute writes to a storage texture don’t blend anyway. When two agents land on the same pixel, their writes race, so one deposit may be lost. The next diffusion pass smears the surviving deposit across neighbouring pixels, so the loss is impossible to spot. Keeping the agents contiguous took more thought: each workgroup prefix-sums its survivors, so compaction needs one atomic operation per workgroup rather than one per agent. Finally, the agent count adapts to the frame rate. It starts at a million, sheds 200,000 per second whenever the smoothed rate drops below 90% of 60 FPS, and never falls below 50,000. A weaker device gets a thinner garden instead of a slideshow. My earlier 2D ray tracer⁴ used a similar quality scaling function to maintain a steady framerate.

Give it a try

A blog post can only do so much to describe an immersive artwork. The best way to understand it is to try it, preferably with the sound on: fleeting.schmelczer.dev⁵.

1. Sebastian Lague’s video: https://www.youtube.com/watch?v=X-iSQQgOd1A↩
2. Sage Jenson’s work: https://cargocollective.com/sagejenson/physarum↩
3. WebGPU spec: https://www.w3.org/TR/webgpu/↩
4. 2D ray tracer: https://schmelczer.dev/articles/sdf-2d-ray-tracing/↩
5. https://fleeting.schmelczer.dev/↩