The page you came from is one hand-written HTML file. No framework, no build
step, no analytics, no tracker — view-source shows you everything, and the full
commit history is public.
The waterfall behind my name is not a video or a texture: it's a spectrogram
computed live — carriers, a log-scale dB floor, receiver noise that's correlated
across bins the way real noise is, because I spent my degree around SDRs and the
fake kind bothered me. You can write on it (it rasters your word like a
Hellschreiber), drag its carriers, and hold the mouse down to charge a
transmission. There's a terminal behind the backtick key.
Some of the page is genuinely live: the ISS position in the corner is real,
the moon over the desk scene is tonight's actual phase, and when it rains in
Delhi it rains in the little line drawings. The evidence panels on the project
cards are real captures from real runs — nothing on this site is a mockup, which
is also why features leave as often as they arrive: anything that turned out to
be decoration without meaning got removed, and the commit history is honest
about every one of those retreats.
Both themes are separate designs, not a recolour — the waterfall inverts
between them the way printed spectrograms always have. There's a print
stylesheet that turns the whole thing into a clean typographic document, and an
llms.txt for AI readers, written plainly, no tricks.
Built by hand in July–August 2026, and still growing.
I crash-tested a 3-billion-parameter model on a simulated ops shift and it did
something I didn't expect: it aced the metric everyone optimises for, and failed
the one nobody talks about.
Across every run, it never missed a single critical ticket — 100% escalation
recall. Sounds like a pass. But its escalation precision on the hardest
shift was 21%: for every ticket that genuinely needed a human, it escalated
roughly four that didn't. Having worked in a real queue, I can tell you exactly
what that is: it's the analyst who forwards everything to the senior team. Nobody
gets fired, nothing improves, and within a month the seniors stop reading the
forwards — which is how the real P0 eventually gets missed.
The model wasn't being safe. It was being unaccountable, and only a
scorecard that prices escalation as spending a human could tell the difference.
The general lesson I took: whatever your agent is for, find the metric that makes
cowardice visible, or your agent will learn to be a coward.
The full numbers are in the case
study; every figure comes from a committed run capture.
The first surprise of building a prompt regression tester: an LLM at
temperature 0 can still give different answers to the identical request. Batching,
hardware, quantisation — the sources vary, but the consequence doesn't: a single
passing run of a prompt test tells you almost nothing.
So PromptDrift
runs every case N times and scores a pass rate against a threshold. A
prompt that works 70% of the time is a different object from one that always
works, and a tool that can't tell them apart will bless flaky prompts all day.
Flakiness itself became a first-class report state: technically green, but on its
way to failing on someone else's machine.
The second surprise was subtler: the baseline must store pass rates, never raw
outputs. Store outputs and every run diffs noisily against the last one, the diff
becomes wallpaper, and people stop reading it — the tool's worst failure mode is
going green while nobody looks. Regression testing non-deterministic systems is
mostly the art of deciding what not to compare.
157 tests, all offline against a deterministic mock — because a
test tool whose own tests flake has no business judging your prompts.
Ask an LLM "how much did I spend on dining in June?" with your transactions
pasted into the prompt and you'll get a confident number that is sometimes right.
Confident-sometimes-right is the worst possible property for anything touching
money.
So Ledgr's
assistant is banned from arithmetic. The model gets four tools —
get_totals, get_spending_by_category,
search_transactions, get_largest_transactions — and its
only job is deciding which to call and how to phrase what comes back. Pandas does
every sum; the model does every sentence. Its answers match the dashboard to the
paisa because they are the dashboard's numbers.
The same split runs through the monthly summary: code computes a stats dict,
the model writes prose around it. I've come to think of this as the honest
division of labour with today's models — they are unreliable calculators and
excellent narrators, and most "AI hallucinated my data" stories come from letting
the narrator hold the calculator.
Bonus defence: the tool loop is capped at five rounds, because an
unbounded loop with a confused model is a bill, not a bug report.
David Jeremie Anand · written by hand, measured by machine ·
home