All articles

This was my first real project and the one that paved the way for the ATtiny85 handheld¹ four years later.

When I bought my first Raspberry Pi, a Raspberry Pi 1 Model A², I had no idea what I was doing. Trying to drive a 12 V RGB LED strip burnt me both figuratively and literally, but it taught me how to wire MOSFETs and control them with PWM³. The obvious next step was to make the strip do something. I’d seen LEDs react to music and then stumbled across an article on the FFT⁴. The article was thoroughly arcane to me, but it gave me the last building block I needed.

Without quite meaning to, I’d built a full-stack application. I learnt about Ajax⁵ and XMLHttpRequest⁶, became fascinated by CSS, and discovered that programming can feel like real-life magic. I wish more kids had that first experience of programming: one that makes you curious to build things and doesn’t punish you too harshly for making mistakes along the way.

1. ATtiny85 handheld: https://schmelczer.dev/articles/ad-astra-attiny85-game-engine/↩
2. Raspberry Pi 1 Model A: https://raspi.tv/2013/raspberry-pi-model-a-first-impressions-and-photos↩
3. PWM: https://raspi.tv/2013/rpi-gpio-0-5-2a-now-has-software-pwm-how-to-use-it↩
4. an article on the FFT: https://betterexplained.com/articles/an-interactive-guide-to-the-fourier-transform/↩
5. Ajax: https://developer.mozilla.org/en-US/docs/Glossary/AJAX↩
6. XMLHttpRequest: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest↩

Producing value is the ultimate test of software. Users don’t care about the cool tech or clever architecture; often, they don’t even care about half the features. They have a problem they want your software to solve.

The problem

My classmates and I were preparing for the advanced physics component of the Hungarian érettségi, the secondary-school leaving exam whose results also count towards university admission. We were advised to start two years in advance by solving questions from past exams¹. But the archive mixes questions from across the curriculum. That makes it tedious to practise a specific topic and hard to tell whether you’ve found a gap in your understanding or a question simply covers material you haven’t learnt yet.

The solution

I manually extracted the questions and diagrams from years of exam PDFs, a perfect way to procrastinate on studying. Then my physics teacher, the project’s other stakeholder, and I categorised every question by topic. Finally, I built a small web interface over the database. The site still runs at fizika.schmelczer.dev², making the questions searchable and generating mock exams using only topics a student should already have mastered.

I built it early on, with only a rudimentary understanding of full-stack development. Yet it was my most popular app for years, measured by both active-user count and time spent per user. It has stood the test of time too. Students at my old school, PTE Babits Mihály Gyakorló Gimnázium³, still use it to prepare for the same exam.

The evolution of the solution

The app has gone through a few iterations. It started as the most spaghetti-like PHP app imaginable, backed by an entirely unnecessary MySQL database. But it worked. I got a good score on the exam and left the site running for future students.

Eventually, hosting a full-blown database became too expensive to justify, so I migrated everything to a static site, hosted first on Firebase, then on GitHub Pages, and now on my own server. Year after year, students emailed me asking how to add the latest questions, and guiding each of them through the GitHub workflow became tiresome. So at the beginning of 2026, I vibecoded a small CRUD interface that edits a JSON file on disk. The file is under version control, so a bad edit can always be undone. I’m back to running a backend server, but the infrastructure was already in place, so the marginal cost was negligible. Happy users, happy maintainer.

Conclusion

Two takeaways from this have shaped my values. First, data can be far more important than the app around it: Fizika’s wrapper has been replaced twice without changing its core value. Second, building software for the thrill of engineering is fun. But building software to create value at minimal cost is how you create actual value.

1. past exams: https://www.oktatas.hu/kozneveles/erettsegi/feladatsorok↩
2. https://fizika.schmelczer.dev/↩
3. PTE Babits Mihály Gyakorló Gimnázium: https://babits.pte.hu/↩

When I was fifteen, I spent the salary from my first summer job on a Nikon D3200¹, my first proper camera. I’ve been photographing whatever captures my imagination ever since. I wanted a personal way to share the results without using social media, which was the perfect excuse to build a portfolio site.

It started as a PHP site running on a Raspberry Pi in my room. To simplify maintenance, I replaced it with a small Vite script that takes a folder of photos and generates static HTML and several sizes of each image.

My only rule for good technology is that it must offer more value than the effort required to use and maintain it. The part of photography I enjoy is taking photos. The website itself is just a nice-to-have, so publishing must take almost no time. Committing a JPEG to a Git repository and pushing it is about the right amount of effort. It’s still one more step than the PHP version required, but I have taste in technology too, not just in photography.

1. Nikon D3200: https://www.cameralabs.com/nikon_d3200/↩

You’re a dot; other dots come in from the edges and chase you. Your only defence is to outmanoeuvre them until they crash into each other. That’s the entire game, and it’s genuinely fun for the first few minutes: a decent return on so little code.

It was also my first time using the <canvas> element. I didn’t know it then, but that one API started an obsession with graphics programming that hasn’t let go of me since.

Later, at university, the game got a second life as the basis for our design society’s web workshop. We rebuilt it step by step with people who’d never drawn a pixel using code. It was exactly the right size: small enough to finish in a few hours, but rewarding enough to feel worth finishing. The handout offered one piece of process advice:

Get it working first, make it pretty afterwards.

That’s still roughly the only process I follow.

2026 update: I haven’t used Windows for a long time, but I’d been wanting to play this game again, and the original binary was compiled only for Windows. I’d never got around to migrating it to SDL2, but an AI agent finally let me just one-shot the migration. So the game I built nine years ago is playable again at platform-game.schmelczer.dev¹.

I titled my first proper university project Platform Game. I’ve clearly come a long way since then, yet one thing hasn’t changed: I’m still bad at naming things.

Back in 2017, I created an actually fun 3D game in pure C with the help of SDL 1.2². I’d wanted to build it ever since I first downloaded GameMaker³ and realised that I could make a video game. I’d already tried with Avoid⁴, and Platform Game is essentially an upgraded 3D version of it.

The maps are randomly generated, but the platforms are laid out so that every jump remains possible with the player’s gravity and jump-strength settings. The world is also fully destructible, voxel by voxel. You can build structures to hide from flying enemies, although once they merge and grow large enough, they can destroy the terrain too. After collecting enough power-ups, you can shoot back or sacrifice points to slow down time.

One feature I distinctly recall is the save file format. Initially, the game dumped the entire procedurally generated map into a file, so one saved map could reach hundreds of megabytes. I realised that each map had long runs of similar voxels, mainly air at the top and ground at the bottom. Encoding those runs as repetitions massively reduced the file size. I ended up discovering run-length encoding⁵, although I later learnt I wasn’t the first person to do so.

One thing I didn’t discover during this project was how to do computer graphics properly: I settled on a software renderer capable only of drawing cubes after deriving a cube’s 3D-to-2D projection using trigonometry and a lot of Post-it notes. Doing it properly had to wait a few more years for a 2D ray tracer⁶.

I’ve made the source code public, more as a reminder to myself than as something others should learn from. The project is neither elegant nor efficient, but it produced a genuinely fun game and it made me fully realise my passion for creating virtual worlds.

1. https://platform-game.schmelczer.dev/↩
2. SDL 1.2: https://www.libsdl.org/↩
3. GameMaker: https://gamemaker.io/↩
4. Avoid: https://schmelczer.dev/articles/avoid-early-web-game/↩
5. run-length encoding: https://hydrolix.io/blog/run-length-encoding↩
6. a 2D ray tracer: https://schmelczer.dev/articles/sdf-2d-ray-tracing/↩

While working at CrySyS¹, the cybersecurity research group at my first university, I got the fun job of simulating traffic in a small city. The lab was preparing a hacking challenge targeting PLCs that controlled traffic lights, so a successful attack would cause mayhem on the roads.

I built everything in Unity². C# scripts controlled the drivers, with driving and self-driving models designed to look realistic, and a REST API exposed the individual traffic lights for remote control. The drivers had to obey the lights unconditionally, tunnel vision and all. That was the point: faulty control logic had to lead to wacky disasters on screen.

The art was on me too, so I ended up watching a bunch of Blender Guru³ tutorials and modelling everything in a low-poly style. Far too much of that time went into trying to get the swing animation right; it wasn’t great, but it looked passable from a distance.

The CTF’s large venue, with screens dotted everywhere, added one final challenge: netcode. The whole simulation ran on a server, with each screen acting as a thin client that extrapolated from the latest state it had received.

1. CrySyS: https://www.crysys.hu/↩
2. Unity: https://unity.com/↩
3. Blender Guru: https://www.youtube.com/@blenderguru↩

I love keeping todos. I also like simplicity and have never subscribed to the notion of over-optimising todo processes, either in my personal or professional life. So I always just aim to find the most convenient and minimal way of keeping todos, which usually ends up as a list of items I delete once I complete them.

In the late summer of 2019, it occurred to me: have you even done a task if you delete its todo? I wanted to keep track of historical todos in a stylised way so that they wouldn’t clutter up my list. That’s why I built the Life Towers goal tracker. It’s more of a UX experiment than a production-ready app, but it conveys my idea for todo management.

You can fill it with a bunch of tasks, and it only shows you the ones you actually want to see at a given moment. The tasks are organised under pages, with each page allowing you to add multiple sections. In my case, each section corresponds to a different aspect of life. The app also keeps track of all historical tasks but turns them into colourful squares that form a tower within each section to show how far you’ve come in different aspects of your life/project/hobbies. There’s a timeline at the bottom of each page allowing you to filter out old tasks or to time-travel and see the evolution of your towers through satisfying animations.

I still believe it’s an interesting idea, and I did use it for a few weeks, but its workflow is just a tad too prescriptive, requires too many clicks, and doesn’t allow for quick reorganising or note-taking, which I actually do quite often. In the end, it has proven to me that nothing beats a simple Markdown file.

I grew up playing with little gadgets: fake Tamagotchis, Game Boy imitations with 100+ built-in games, and the like. Naturally, I wanted to build my own.

The game

You keep a broken spaceship alive. Asteroids drift past; you mine them and spend the points on upgrades. Each upgrade unlocks either a station with a new capability or a crewmate to automate a station for you.

The hardware

I settled on the ATtiny85V¹, driving a common 128×64 1-bit OLED². To keep the footprint small, I used a TSOP4838 IR receiver and a compact decoder for the NEC protocol instead of built-in buttons or a joystick. Mine was paired with a generic hi-fi remote I bought on AliExpress for pennies. Connecting the components was quite simple, even if my breadboard suggested otherwise:

Routing the PCB from this design was relatively straightforward. The real challenge was the firmware.

Reality check

I had planned an open world, “advanced” AI, and more, but I quickly got a reality check. Even after bit-packing, the framebuffer alone would’ve needed twice the microcontroller’s 512 bytes of RAM. The 8 KB of program memory left little room for grand ambitions either. Still, I made it work.

Rendering engine

To achieve 50 FPS and fit into RAM, the engine renders at a quarter of the display’s resolution (64 by 32 pixels). The OLED’s zoom setting doubles the image vertically, while sending each byte twice over the wire doubles it horizontally. This gives nearest-neighbour upscaling without storing any extra pixels.

The screen’s pixels are either on or off, so 1 bit per pixel should be enough. But a rendering engine also needs compositing, which requires a second bit per texel to represent transparency. That’s what lets the black pixels inside the spaceship’s sprite hide the stars behind them, while the black pixels outside it stay transparent and let the stars show through.

A shortcoming of the sprite format is that it uses 2 bits per texel for only three distinct results: black, white, and transparent. The two possible encodings of white produce the same output regardless of what’s behind them.

A complete 64 × 32 one-bit framebuffer would still consume 256 of the microcontroller’s 512 bytes, leaving too little room for the game state. The pragmatic choice would’ve been a bigger microcontroller, but hobby projects don’t always have to be pragmatic. Instead, the renderer keeps a 64-byte buffer for one 8-pixel-high stripe. It renders the frame in four passes, with each byte operation compositing eight vertical pixels in a SIMD-like fashion.

Sprites are stored as 16-bit columns: the high byte is a mask, and the low byte contains the fill bits. Compositing eight vertical pixels of a sprite onto the stripe buffer is a single expression:

newColumn = (oldColumn & transparencyMask) | fill;

The sprites are C arrays injected into the source code by a Python script, so the somewhat strange layout doesn’t leak into the sprite creation workflow.

Game logic

For the game engine’s scripting model, I borrowed ideas from JavaScript prototypes³ and Unity objects⁴. Of course, the size constraints allowed only a lightweight version. All game objects live in one array. Each object holds a union of possible object states and a pointer to its actual prototype: essentially a virtual function table⁵ containing the object’s tick (to update the object and interact with others on every game tick) and draw functions. Although simulation and rendering share the same 50 Hz tick, they need separate functions because draw renders one 8-pixel stripe at a time and may run several times per object per frame. This quasi-vtable doesn’t become a bottleneck either. The beauty of the ATtiny is that it has no complex cache hierarchy which would make jumping around in memory more expensive. Everything is equally slow.

In the end, the game does have “AI” too, in the form of a finite state machine. Your astronaut crewmates have a prioritised list of activities to attempt based on the player’s current activity.

Saves

The game’s binary already fills the 8 KB of program memory, so the sprites have to live in EEPROM. There is still enough room alongside them for a save file: a few dozen bytes containing the serialised game-object list. To prevent an unexpected shutdown from corrupting the save, the EEPROM holds two save buffers and a flag identifying the last completed one. The flag flips only after the older buffer has been fully overwritten, making saves atomic. It all happens in the background through interrupts to avoid jitter in the rendering loop.

Was it worth it?

Absolutely. Don’t get me wrong, the gameplay gets old after 15 minutes. You spend the first five completing the game, then watch your crewmates run the spaceship as a soothing loop emerges from their state machines.

Reading the datasheets cover to cover, understanding the limits, and learning how to push them was incredibly fun. This is about as full-stack as a project gets, and it made me think about performance on a completely different scale from what I was used to.

By the end, though, the game and my software-UART logging code no longer fit in program memory at the same time, so I had to disable parts of the game whenever I needed logs. If I built it again, I’d start with a simulator. Skipping the flashing step would speed up iteration and make “there’s no output, what now?” failures much easier to debug. It would also enable end-to-end tests in CI. An 8 KB program can only hold so much complexity, but developing anything larger without better testing would’ve been prohibitively slow.

1. ATtiny85V: https://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-2586-AVR-8-bit-Microcontroller-ATtiny25-ATtiny45-ATtiny85_Datasheet.pdf↩
2. 128×64 1-bit OLED: https://www.olimex.com/Products/Modules/LCD/MOD-OLED-128x64/resources/SSD1306.pdf↩
3. JavaScript prototypes: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Inheritance_and_the_prototype_chain↩
4. Unity objects: https://docs.unity3d.com/Manual/Components.html↩
5. virtual function table: https://eli.thegreenplace.net/2013/12/05/the-cost-of-dynamic-virtual-calls-vs-static-crtp-dispatch-in-c↩

2026 update: I recently revived the game: made the characters’ eyes less creepy and improved the gameplay. I also stopped paying for the original decla.red domain, so I rebranded the game as doppler. It’s playable at doppler.schmelczer.dev¹.

My BSc thesis was on a novel ray-tracing renderer, SDF-2D², and this project was about eating the pudding. The concept is simple: the scene is set in space. Two teams have to conquer tiny planets and can also shoot at the other team. Points are given based on the number of planets controlled, and the first team to reach a predefined score wins.

Design patterns

Given I already had a bespoke renderer, I figured: might as well write the game engine too. How hard could it be? Fairly hard, turns out. However, there were a few design patterns which made it tractable.

Both the game client and server are written in TypeScript (TS), which made it easy to define a third TS package: a library which sits between them and contains the shared logic. So, for example, the wire schema and even the movement logic are defined in one place. This eliminates a class of bugs caused by discrepancies between, for instance, the server-side movement physics and the client-side prediction of this movement.

When it comes to the game objects, I decided to borrow inspiration from Smalltalk’s message passing. Objects send events to other objects: these can be broadcasts, messages limited to an area, or perhaps messages between two objects that physically overlap. The objects can then declare which messages they’re interested in receiving. This includes the concept of “doesNotUnderstand”³, which allows messages to be passed on to nested objects. This all results in a very decoupled system where everything can interact with anything it wishes to through standard messages without having to implement #objects × #interaction methods.

Conclusion

Ultimately, the game proved that the renderer works in practice. It was a great deal of fun drawing every object with signed distance fields, and I was delighted that the rendering just worked by the end. Unfortunately, I didn’t continue working on the game as the gameplay didn’t feel interesting to me even after I’d tried tuning it and adding/removing features. So I had to realise that I’m just not fond of 2D games after all, and I lost the motivation to improve it further. The technical challenges were fun, though.

1. https://doppler.schmelczer.dev/↩
2. SDF-2D: https://schmelczer.dev/articles/sdf-2d-ray-tracing/↩
3. “doesNotUnderstand”: https://wiki.c2.com/?DoesNotUnderstand↩

My BSc thesis¹ needed two things: a renderer and something to render. The former became SDF-2D, a browser-based 2D ray tracer built on signed distance fields. The latter became doppler², a multiplayer space shooter built on it. This article is about the renderer.

2D games mostly composite flat, unlit sprites while running on ever-faster GPUs. Ray tracing could give them shadows and area lights, but of course, a naive implementation would be unusably slow on most devices people play 2D games on, especially phones. SDF-2D shows how to make this practical and provides a simple API for rendering complex scenes defined by signed distance fields on most devices.

Circle tracing

It’s pretty convenient if we can describe a scene using a signed distance field (SDF). We can skip the triangles and use sphere tracing³ to render the scene at arbitrarily high resolutions with complex lighting effects. Sphere tracing renders surfaces defined by an SDF by marching along each ray in steps equal to the field’s value at the current point. A negative field value means the point is inside an object, and values near zero can be used for antialiasing.

Circle tracing is the 2D version. The only downside is the massive amount of compute required to run it. Rays are only needed for lighting, so performance depends on how often the field is evaluated, which in turn depends on the number of lights. A naive renderer could therefore evaluate the field pixels × ray_marching_steps × lights times per frame.

Optimisations

If we ignore reflections, deferred shading (memoising the full screen’s distance field) replaces repeated procedural distance-field evaluations with cached texture reads. The geometry pass evaluates the field once per distance-texture pixel; the lighting pass still performs roughly shading pixels × ray_marching_steps × lights texture samples, but those samples are much cheaper than evaluating the full scene SDF. One pass renders the distance field and base colour into a texture, and the lighting pass reads it back. This also lets us change the geometry’s resolution independently of the shading resolution. An upscaled gradient is easy to spot because of the artefacts, but a circle rendered at half resolution and then shaded at full resolution looks convincingly full resolution.

The second optimisation borrows from tiled renderers. The screen is split into an 8 × 8 grid, and each tile receives only the objects that overlap it or are reasonably close (within a distance of d_min). The CPU assigns objects to tiles, which takes a negligible amount of time for a few dozen objects. In my 200-object test scene, the average tile needed only 23 objects: about 8.7× fewer object evaluations.

The tiles’ distance fields can be rendered in parallel. The only caveat is that, when combining them, we have to cap the maximum distance at d_min. The stored value is therefore a conservative lower bound on the true distance rather than an exact distance field.

Here are the results for that scene (200 objects, two lights, 16 shadow steps, 2560 × 1080) on a desktop RX 590, with GPU time measured using EXT_disjoint_timer_query:

That’s a 70-fold speed-up, giving a mid-range phone enough headroom.

Efficient shadow approximation

I could’ve used a textbook approach to soft SDF shadows, such as Quilez’s ray-marched penumbra technique⁴. In my baseline implementation, getting acceptable results took 64 to 128 steps per light, and the artificially capped lower-bound field produced artefacts at sharp edges. But I found that, in 2D, simpler approximations work just as well. So I used the following heuristic: march towards the light for 16 steps, divide the distance travelled by the distance to the light, and raise the ratio to the power of an INTENSITY constant (0.3 works well).

float shadowTransparency(float lightDistance, vec2 lightDirection) {
  float rayLength = 0.0;

  for (int j = 0; j < 16; j++) {
    rayLength += max(0.0, getDistance(uvCoordinates + lightDirection * rayLength));
  }

  return min(1.0, pow(rayLength / lightDistance, INTENSITY));
}

The closer the ray gets to the light, the less shadow its starting point receives. The heuristic has no physical basis, but it runs several times faster and accepts lower-bound fields. Anything from 8 to 32 steps looks acceptable. There’s also the obviously very intentional artistic choice: objects already inside another’s shadow darken that shadow again.

And the API

I believe theory has value only when put into practice. Therefore, the optimisations are implemented in SDF-2D⁵. Here’s a minimal scene in TypeScript, with a circle orbiting a light:

import { CircleFactory, CircleLight, hsl, runAnimation } from 'sdf-2d';

const canvas = document.querySelector('canvas');
const Circle = CircleFactory(hsl(180, 100, 40));

const draw = (renderer, time) => {
  renderer.addDrawable(
    new Circle([150 + 50 * Math.cos(time / 1000), 75 + 50 * Math.sin(time / 1000)], 25)
  );
  renderer.addDrawable(new CircleLight([150, 75], hsl(270, 100, 40), 0.1));

  return true; // flag that more frames are coming
};

runAnimation(canvas, [Circle.descriptor, CircleLight.descriptor], draw);

You can find more details about the API in the docs⁶. One interesting bit is the need to specify the objects and their types separately (the descriptor list on the last line). This is because SDF-2D supports both WebGL1 and WebGL2, as iOS didn’t support WebGL2 back in 2020. WebGL1 requires loop bounds to be compile-time constants. The library can’t know how many circles you’ll draw, so it generates its shaders at runtime.

Each drawable’s descriptor declares its GLSL code, the uniform arrays carrying its instances, and the object counts to compile for, such as [0, 1, 2, 4, 8, 16]. The generator substitutes these into GLSL templates and compiles a program for every combination across drawable types. The number of programs grows exponentially, but keeping it under a few hundred is doable. Each tile is drawn with the smallest program covering its object counts.

Field testing in an electronics store

The demo page⁷ anonymously logged each scene’s frame rate. Organic visitors and pestered friends provided some data; I gathered the rest by running the demo on every device on display in an electronics store, since device farms charged more than a student could afford. Of 37 unique devices, 33 held 30 FPS or better, and over three quarters stayed at the display’s cap of 60 FPS in every scene. The bars above 60 represent a 120 Hz Samsung tablet and a 90 Hz Pixel 5. Some of the slowest results came from the SwiftShader software renderer and a 4K TV running a four-year-old version of Chrome.

Not bad.

1. BSc thesis: https://schmelczer.dev/media/downloads/sdf2d-andras-schmelczer.pdf↩
2. doppler: https://schmelczer.dev/articles/doppler-shared-simulation-code/↩
3. sphere tracing: https://doi.org/10.1007/s003710050084↩
4. Quilez’s ray-marched penumbra technique: https://iquilezles.org/articles/rmshadows/↩
5. SDF-2D: https://www.npmjs.com/package/sdf-2d↩
6. docs: https://sdf-2d-docs.schmelczer.dev/↩
7. demo page: https://sdf2d.schmelczer.dev/↩

The more critical a system is, the simpler it should be. In practice, that means building on battle-tested foundations and adding only the thin layer of logic that’s strictly necessary.

My home server’s backup is a good example. I aim to use SQLite wherever I can, but my self-hosted stack has grown to include several Postgres instances, Redis, message queues, and more. This couldn’t stop me from wanting Time Machine-style backups with granular file history, which means backing up often: hourly, in my case. Shutting down the stack every hour would’ve greatly inconvenienced its active user base of two, my partner and me, so I chose to back up everything live.

I could have gone down the rabbit hole of coordinating dedicated backup tooling: a bespoke method for every service, each with its own failure modes. That would have been anything but simple. Instead, I rely on the databases’ battle-tested crash-recovery paths.

With that assumption, the backup container needs just a simple shell script. All container volumes live in the same Btrfs¹ subvolume, so the script can take an atomic snapshot of all state. It then feeds the snapshots into Borg², which creates complete logical archives while deduplicating their chunks, so only new or changed chunks need storing and transferring to local or SSH backup targets. A database restored from one of these backups is in a state indistinguishable from that left by a power cut. The backup script supplies only the plumbing: configuring Btrfs and Borg, handling failures gracefully, and exposing a health-check status when human intervention is needed.

I’ve been using this since 2023. I’ve restored various files and folders and performed two full (albeit planned) disaster recoveries. Nothing builds confidence like a restore that works: Borg, Btrfs, and this little script have worked well, and I don’t expect to migrate away any time soon.

1. Btrfs: https://btrfs.readthedocs.io/en/latest/↩
2. Borg: https://www.borgbackup.org/↩

Watching art emerge from millions of independent agents is fascinating for a while, but not for all that long. My favourite art makes the viewer part of the work, so this project had to be interactive to live up to my expectations. I was inspired by Sebastian Lague’s slime mould video¹ and Sage Jenson’s work², but I wanted more control over a running simulation rather than just passively observing it.

Architecture

A few compute shaders manage the agents’ lifecycle by updating an array in place. Each agent follows a simple loop. On every iteration, it takes three samples: one ahead and one on either side. It then goes straight, turns left, or turns right based on which sensor reports the strongest trail scent left behind by other agents. The samples come from a trail map that records the agents’ paths and slowly diffuses them over time. Its R, G, and B texture channels each hold deposits from one agent type. I considered using the fourth channel too, but three colours already make the result busy enough, and a fourth would’ve made colour theory a fair bit harder for my colour-blind eyes.

The website has multiple scenes, each with its own simulation parameters. Each scene also has a matrix that defines how every agent type responds to every other type’s trail: with attraction, repulsion, or indifference.

Make it good

My first experiment let the user draw over the trail texture, nudging the mould to change direction. That proved one layer too indirect: it barely changed the agents’ behaviour unless the effect was strong enough to make the strokes look overly thick. In the final version, drawing spawns new agents along the pen’s path. When there are too many agents on the canvas, old ones get removed as new ones arrive.

Drawing is great fun, but not everyone can draw well. No amount of line smoothing will change that, though it helped me a little. That’s why I added mirroring, a common drawing-tool feature that turns scribbles into artsy kaleidoscopic patterns.

To deepen the immersion and give each stroke more weight, I added procedurally generated piano sounds that reward drawing with increasingly complex note progressions.

I’m a firm believer that small touches give experiences polish. For instance, the toolbar samples a 13×7 grid of pixels behind it and adjusts its opacity to stay legible against any background. And the first thing a visitor sees is 180,000 agents drawing the word Fleeting with their trails, then wandering off as the simulation takes over and the letters fall apart.

Then make it fast

The simulation is split into six conditional compute-and-render stages across ten WGSL files: agent step, diffusion and decay, brush, eraser (one variant clears trails, the other removes agents), agent generation with resizing and compaction, and the final palette render.

The first step in making shaders fast is to keep the logic simple and mostly branch-free. A few other ideas helped too:

• Agents write deposits directly to a storage texture without blending, which concurrent compute writes don’t support anyway. This is deliberately racy: if two agents target the same texel, WGSL does not guarantee which value survives, or even that the result behaves like a simple last-writer-wins update. In practice, the next diffusion pass spreads the surviving deposit across neighbouring pixels, so we don’t need to sacrifice parallelism.
• Keeping the agents packed together in memory took more care. During compaction, each workgroup uses a prefix sum to find positions for its surviving agents, so compaction needs just one atomic operation per workgroup instead of one per agent.
• The agent count also adjusts to the frame rate. It starts at a million and drops by 200,000 per second when the smoothed rate falls below 54 FPS (90% of the 60 FPS target), down to a minimum of 50,000. A weaker device gets a thinner garden instead of a slideshow.

Give it a try

A blog post can only do so much to describe an immersive artwork. The best way to understand it is to try it, preferably with the sound on: fleeting.schmelczer.dev³.

1. Sebastian Lague’s slime mould video: https://www.youtube.com/watch?v=X-iSQQgOd1A↩
2. Sage Jenson’s work: https://cargocollective.com/sagejenson/physarum↩
3. https://fleeting.schmelczer.dev/↩

I built a small e-ink photo frame for our home. It pulls photos from a self-hosted Immich¹ library, which has already run face detection and geocoding on them. It checks a self-hosted Home Assistant² instance to see whether anyone is home, then displays a photo on the PhotoPainter³ for everyone to enjoy.

Why

Most digital frames offer a choice between two bad options: preprocess the photos and copy them to an SD card yourself, or hand your library to a cloud service like Google Photos. Realistically, the SD card gets updated once a year, and I’d rather not give a cloud provider access to my most cherished memories.

Yet I’d imagined that coming home to a photo from earlier that day, or one from five years ago, would feel different if it were in a real picture frame instead of buried on my phone. So I decided to spend a weekend building a personal version with Claude Code. I experimented with dithering and post-processing, fine-tuned the photo-picking algorithm, and ended up with this project.

It only took a weekend because so much came for free. Immich had already detected faces in every photo and reverse-geocoded the place names shown on the overlay, and it knew which photos we’d marked as favourites. Home Assistant already knew who everyone was and who was home. That’s the compounding return of self-hosting: every service exposes an easy-to-integrate API, and each new project gets to stand on all of them.

How it works

The frame is driven by a Raspberry Pi Zero 2 W, which is way overpowered, but at least it’s fast enough to let me write plain Python and mostly ignore performance. A cron job runs a script every 15 minutes. Each run:

1. Quits if it’s between midnight and 7 am. The e-ink display holds its image without power, so the last photo of the evening simply stays up overnight.
2. Asks Home Assistant whether anyone is home. If not, it quits; there’s no point wasting power and straining the display.
3. Picks a random photo from Immich, drawn from four overlapping pools: “on this day” memories, favourites, the last 30 days, and the whole library. A rolling 7-day history log prevents repeats. Photos matching the frame’s orientation get 4× the weight, while any candidate whose crop would clip a head is rejected (more on that below).
4. Crops around any detected faces, boosts contrast and saturation (e-ink is short on both), dithers the image down to the six-colour palette, and pushes the result to the panel. The photo’s age and EXIF location are painted into the bottom corners as white text with a black stroke, so the dither can’t smear the edges.

Image pipeline

The final look depends most on two choices: how the image is cropped around faces and how dithering is applied.

Cropping

The photo frame stands in one orientation, but I didn’t want to limit it to photos that happen to match. A face-aware algorithm therefore resizes and crops each image to fill the panel, biasing the crop window towards faces Immich has already detected. A landscape shot with some room around the subject will often crop cleanly to portrait this way. If the crop would cut into a head, the photo is rejected and the picker moves to the next candidate.

The examples below show how the head bounding boxes steer the final crop and which candidates get rejected.

Dithering

The panel shows exactly six colours: black, white, red, yellow, blue, and green. Unlike an LCD, it has no intensity control, so every pixel must use one of those six colours. Anything photo-like has to be dithered, and the available algorithms differ wildly in output quality and running time. Here’s how a few of them compare:

I settled on Atkinson dithering⁴, which preserved the most contrast without too many visible artefacts. Unlike the alternatives, it diffuses only 6/8 of each pixel’s quantisation error, so highlights stay bright instead of being dragged grey by their neighbours. Dithering was also the first part of the pipeline where pure Python became unusably slow on the Pi. The inner loop⁵, including perceptually weighted nearest-colour matching, is therefore JIT-compiled with Numba⁶. Once the cache is warm, it runs roughly two orders of magnitude faster.

Learnings

The Pi Zero 2 W is overkill. It drains a battery far too quickly: it wastes power sitting idle for most of its life, waiting for the next cron tick. An external RTC-controlled power circuit could have fixed that, but I couldn’t be bothered to hack one in. For a battery-powered build, I’d use an ESP32 with deep sleep instead. With 15 minutes between refreshes, even a slow chip has plenty of time for image processing and dithering.

A few reliability quirks are worth mentioning. In my setup, the Zero 2 W’s Wi-Fi connection drops when power saving kicks in, so a separate cron job runs wifi-check.sh every 5 minutes to reconnect. I also masked swap and set journald to volatile because SD-card wear is the only thing likely to slowly kill the frame.

For the next frame, I’d like to try an Inky Impression⁷ in a custom enclosure, with a larger display and perhaps integrated lighting. E-ink looks a little muddled in evening light, so a dedicated light source would be the biggest improvement by far.

All in all, the frame already does its job. It stands in the living room as a pleasant speck of curiosity, subtle enough that most visitors never notice it until I inevitably point it out. Once you know it’s there, catching the panel mid-refresh brings a small rush of excitement.

1. Immich: https://immich.app/↩
2. Home Assistant: https://www.home-assistant.io/↩
3. PhotoPainter: https://www.waveshare.com/wiki/PhotoPainter↩
4. Atkinson dithering: https://beyondloom.com/blog/dither.html↩
5. inner loop: https://git.schmelczer.dev/andras/frame/src/commit/c0c2b779d7b43793b594b9c52b694133859edfd8/src/lib/waveshare_epd/epd7in3e.py#L59-L109↩
6. Numba: https://numba.pydata.org/↩
7. Inky Impression: https://shop.pimoroni.com/products/inky-impression?variant=55186435244411↩

Why

Merging concurrent edits is a solved problem when you control the whole editing stack¹: conflict-free replicated data types² (CRDTs) and operational transformation³ (OT) capture changes as they happen, which works well when you own the editor, transport layer, and storage format. However, there are workflows where that’s not the case.

For instance, text files on disk can be edited in Obsidian⁴, Vim, or VS Code and updated through Syncthing or even OneDrive. A sync engine therefore sees only the resulting file snapshots, often after a delay, rather than the individual edits that produced them. This is the same constraint Git works under: Git-tracked files may be edited by many programs, all outside Git’s control. Git rightly highlights ambiguous write-write conflicts with <<<<<<< markers: a bad source-code merge may introduce a bug, so a human should verify the result.

Git’s union merge driver is the closest to a built-in solution. The union driver works line by line and resolves a conflict by retaining both sides’ line sequences. As a result, independent word-level edits can duplicate whole lines, while overlapping deletions can restore or reorder text that one side deleted.

But I wanted fully automated merges for my text files with intuitive merge results. That’s why I created reconcile-text, which is built on the assumption that human languages are more forgiving than code: a slightly imperfect sentence is usually better than conflict markers interrupting a document. So reconcile-text essentially implements git merge (technically, it’s git merge-file⁵) without conflict markers. And it does this without client-side tombstones or an operation history. Let’s look at an example:

import { reconcile } from 'reconcile-text';

// the version both sides start from
const parent = 'Hello world';

// one user added "beautiful"
const left = 'Hello beautiful world';

// the other changed the greeting
const right = 'Hi world';

// "Hi beautiful world"
reconcile(parent, left, right).text;

You can try it out yourself in the interactive playground at reconcile.schmelczer.dev⁶.

The API is essentially just this one function, plus a few helpers to make transport easier. It takes a parent and two diverging versions, derives and applies both sets of changes, and repositions cursors and selections along the way. Cursor support makes it possible to use reconcile-text in a text editor too.

How it works

The first step is splitting the inputs into tokens, which is why the demo website offers a choice of tokeniser. Even though human text is forgiving of merge artefacts, I still wouldn’t want characters within words interleaved into nonsense. That’s why treating words, or even sentences, as atoms can be useful. The built-in Markdown tokeniser extends word tokenisation to account for Markdown syntax. The Rust API also lets you specify custom tokenisation rules. We can’t guarantee that the merged Markdown, or other marked-up text, will be syntactically correct⁷, but we can guarantee that the selected tokeniser’s tokens won’t be split up.

After tokenising, the algorithm diffs each edited version against the parent (diff(parent, left), diff(parent, right)), as diff3⁸ does. It uses Myers’ diff algorithm⁹ to compute those diffs, then cleans them up to make insertions and deletions contiguous, keeping as many of each side’s changes together as possible.

Finally, the two sides are zipped together using OT principles: each side’s edits are transformed against the other side’s. Cursor offsets are updated along the way, making each cursor “stick” to the nearest token and travel with it during the merge.

I didn’t set out to implement OT for its own sake. Transforming batched diffs simply turned out to be an elegant way to merge the outputs of two Myers diffs. A CRDT could achieve the same result. But when all you can observe is the final state, merge quality is limited by the underlying two-way diffs, regardless of the machinery used to combine them.

How to implement a syncer

The library’s merging primitive can support a wide range of text-syncing apps, including ones inspired by differential synchronisation¹⁰. A simple starting point is Git’s fetch-merge-push loop, with reconcile-text handling conflicts automatically. An implementation could work as follows:

The server keeps immutable versions of each document, each with a unique ID, and tracks which is the latest. Each client has an editable file and remembers the ID and content hash of the last server version it incorporated. Let’s call that version the base. The client gets it from an initial fetch; after that, it only needs to fetch the base’s text when a merge is necessary.

For each document, the client runs one sync loop at a time:

1. Check for local edits. Take a snapshot of the file and compare its hash with the base’s hash. If they differ, push the snapshot along with the base’s ID to the server.

2. Accept if the server hasn’t moved on. The server saves the snapshot only if the base is still its latest version. Checking the ID, saving the new version, and making it the latest happen atomically. Otherwise, it rejects the push and returns its latest version’s ID and text.

3. On success, remember what was sent. Record the returned ID and the pushed snapshot’s hash as the new base. Keep the live file as it is: the user may have edited it while the request was in flight.

4. On rejection, merge locally. Fetch the old base’s text and merge it with the current file and the server’s latest text. Record that latest server version’s ID and hash as the new base. The local merge¹¹ is just the same three-way call as before:

   file = reconcile(base, file, latest.text).text;

After either outcome, check for local edits again. A merge may need pushing, and so may edits made during a successful push. The loop pauses once the file’s hash matches the base’s hash, then wakes on local edits, a timer, or a server notification. The timer or notification triggers a fetch: if the server’s version ID has changed, merge and update the base as above, even if the local file hasn’t changed.

Extensions

A real sync app also needs to handle lost responses and application crashes. Giving each push a unique idempotency key lets the client safely retry it: if the server already accepted it, the server can return the original acknowledgement.

To save bandwidth after a merge, the client can use to_diff¹² to send only the changes relative to the fetched server version. The server reconstructs the snapshot with from_diff¹³ and its stored copy of that version.

If fairness becomes an issue, we can let a high-latency client lease a server-side lock long enough to get its edits through.

Where reconcile-text sits among alternatives

diff3 and git merge-file do the hard work of merging but explicitly don’t resolve most write-write conflicts. Neil Fraser’s diff-match-patch is the closest tool in spirit, but it applies two-way patches heuristically rather than reasoning over a common ancestor and both edited versions. So failed patches can lose edits. CRDTs preserve concurrent edits, but they require CRDT state and metadata tied to the document’s editing history rather than three plain strings. That’s the niche reconcile-text fills: syncing systems that need automatic merges but only have snapshots to work with.

One implementation, many ecosystems

The Rust library compiles to WebAssembly through wasm-bindgen¹⁴ for the npm package and provides native Python bindings through PyO3¹⁵. This makes reconcile-text available from Rust, Python, and JavaScript or TypeScript across Node.js, the web, and React Native. Since React Native’s Hermes engine doesn’t support WebAssembly, that entry point ships a pure-JavaScript build produced by Binaryen’s wasm2js.

1. a solved problem when you control the whole editing stack: https://marijnhaverbeke.nl/blog/collaborative-editing-cm.html↩
2. conflict-free replicated data types: https://crdt.tech/↩
3. operational transformation: https://sigmodrecord.org/1989/06/15/concurrency-control-in-groupware-systems/↩
4. Obsidian: https://obsidian.md/↩
5. git merge-file: https://git-scm.com/docs/git-merge-file↩
6. https://reconcile.schmelczer.dev/↩
7. Ink & Switch’s Peritext¹⁶ is a good write-up on this problem for rich text.↩
8. diff3: https://blog.jcoglan.com/2017/05/08/merging-with-diff3/↩
9. Myers’ diff algorithm: https://blog.jcoglan.com/2017/02/12/the-myers-diff-algorithm-part-1/↩
10. differential synchronisation: https://neil.fraser.name/writing/sync/↩
11. local merge: https://git.schmelczer.dev/andras/reconcile/src/commit/08a656c6ed32d7c307b36bc6dec0134c40ac0a43/reconcile-js/src/core.ts↩
12. to_diff: https://git.schmelczer.dev/andras/reconcile/src/commit/08a656c6ed32d7c307b36bc6dec0134c40ac0a43/src/operation_transformation/edited_text.rs#L406↩
13. from_diff: https://git.schmelczer.dev/andras/reconcile/src/commit/08a656c6ed32d7c307b36bc6dec0134c40ac0a43/src/operation_transformation/edited_text.rs#L475↩
14. wasm-bindgen: https://docs.rs/wasm-bindgen↩
15. PyO3: https://pyo3.rs/↩
16. Peritext: https://www.inkandswitch.com/peritext↩

The pitch

There’s an immense amount of public, highly granular data about the UK, especially England and Wales: the prices of millions of recorded property sales¹; floor areas and construction years², all with full addresses; street-level crime³; noise levels at 10 m by 10 m granularity⁴; public transport timetables⁵; mapped woodlands⁶; conservation areas, council houses, listed buildings⁷, and schools⁸; plus detailed demographics⁹ on education, renting, and a lot more. When I was looking to buy a property, these datasets helped me narrow down where to move and what to expect.

Sites such as HouseMetric¹⁰ and CrystalRoof¹¹ already provide some of this data for a given postcode, but I needed the search the other way around. I had been living in London for only four years, so I didn’t know its many neighbourhoods well enough. That’s why I wanted to flip the lookup: instead of going from a postcode to its attributes, I wanted to go from the description of my ideal area to matching postcodes. This is how perfect-postcode.co.uk¹² began.

Perfect Postcode introduced me to several hidden gems I’d never heard of but fell in love with after visiting. It pushed me beyond familiar areas and gave me a broader picture of London. It also showed me what my expectations would cost and how the equation changed with a compromise or two.

In short, users can set numeric ranges or choose allowed categories for property and area attributes. For example, a user can ask to see only areas within a 10-minute walk of a station, with noise below 56 dB, with a 2-bed costing less than £600k, plus any of the 83 other criteria. The result is a heatmap showing which areas contain the highest number of matching past sales, and therefore where the criteria are most likely to be met. From there, users can narrow their search, contact estate agents, get access to off-market properties, and keep an eye on Rightmove and Zoopla.

Rough architecture

Filtering happens per property: all the public data is joined onto roughly 25 million property records, which are then aggregated at various granularities. Matches appear as H3¹³ hexagons at lower zoom levels and postcode boundaries at the highest zoom.

The basic architecture is straightforward: download the data, normalise and join it, add a filtering UI, then write a backend that returns matching areas. The difficult part is the main non-functional requirement: filtering must feel instant so users can see how each change affects the results to allow developing an intuition.

The full dataset contains 25 million rows. That’s really not that much data for today’s servers; it’s not too large to keep in memory, and it also won’t grow exponentially. That’s why I decided to see how far a brute-force linear scan could take me with it. Making brute force fast requires some preparation. So most attributes are quantised to 16 bits because values such as crime counts and room counts need no greater range or precision. There is only one index: a spatial grid (0.01 deg) that prunes addresses outside the user’s viewport. The data uses a row-major layout because queries often filter on dozens of attributes at once. Rows are also sorted by spatial position, so each scan touches contiguous chunks. This keeps scans cache-friendly and easy to parallelise.

After the intentional data layout, the brute-force search logic is as simple as for each row:

let base = row * NUM_FEATURES;
let should_include = filters.iter().all(|f| {
    let raw = feature_data[base + f.feat_idx];
    raw != NAN_U16 && raw >= f.min_u16 && raw <= f.max_u16
})

In practice, this results in sub-100 ms p99 query latency for our userbase which easily justifies the server’s roughly 12 GB memory footprint.

Still, 100 ms plus network latency is too slow for instantaneous feedback in the UI. Fortunately, filters are always combined with AND, and a numeric slider changes only one attribute’s bounds at a time. When a user starts dragging one, the frontend requests that attribute’s minimum, average, and maximum for every visible hexagon or postcode (with the active filter excluded). It uses those aggregates to recolour the map immediately and to dim areas whose value range cannot overlap the selected range. Releasing the slider sends the filter to the backend and replaces the preview with exact counts. You can see both stages in action here:

Of course, there’s a bit more to it, especially at the coarser H3 levels, where expensive queries benefit most from caching. Other data stays outside the hot path, including individual property histories and travel times. But the short version is that brute force can be the best solution when applied in a smart way.

Derived data

Besides the open datasets, the pipelines powering the app create interesting derived values, for example:

• Tree canopy density percentile: based on mapped tree-canopy and woodland coverage
• Price growth percentile: showing where prices are rising or falling fastest
• Public transport travel time: calculated between stations and neighbourhoods using R5¹⁴ which is precomputed for every postcode and destination combination under several constraints

What’s next?

Six months of viewings and scouting areas in-person gave the perfect feedback loop. Whenever the numbers and my impression of an area differed, it usually pointed to a missing feature. For example, one Saturday we visited an area of Loughton that had looked perfect based on the numbers. However, we counted more than a dozen England flags hanging from properties, which prompted me to add a Reform UK voter share filter. Now that the product works well for us, it’s time to iterate on user feedback.

Originally, the app’s target audience was my fiancée and I. But as we added features, it became clear that other house hunters could benefit from it too. That meant improving the user experience with saved and shareable filters, clearer explanations, and a genuinely usable mobile site, then adding a payment flow and doing some marketing. Now, we’re looking at the usage metrics to see where to take the project from here.

1. prices of millions of recorded property sales: https://www.gov.uk/guidance/about-the-price-paid-data↩
2. floor areas and construction years: https://epc.opendatacommunities.org/↩
3. street-level crime: https://data.police.uk/↩
4. noise levels at 10 m by 10 m granularity: https://environment.data.gov.uk/dataset/562c9d56-7c2d-4d42-83bb-578d6e97a517↩
5. public transport timetables: https://www.bus-data.dft.gov.uk/↩
6. woodlands: https://www.forestresearch.gov.uk/tools-and-resources/national-forest-inventory/↩
7. listed buildings: https://opendata-historicengland.hub.arcgis.com/↩
8. schools: https://get-information-schools.service.gov.uk/↩
9. detailed demographics: https://www.ons.gov.uk/census↩
10. HouseMetric: https://housemetric.co.uk/↩
11. CrystalRoof: https://crystalroof.co.uk/↩
12. https://perfect-postcode.co.uk/↩
13. H3: https://h3geo.org/↩
14. R5: https://conveyal.com/↩