This is a very interesting introduction to a blog post, but... I'm somehow missing the actual blog post. How does this stuff work in practice? What are some concrete examples? How does one get from JavaScript tokenizing things in a commit hook to validating that the LLM didn't disable tests it didn't agree with, or any other helpful property?

Makes sense, I have had the biggest wins with AI by attacking nondeterminism whenever possible.

BTW, you should probably fix the Beagle link on your homepage: https://replicated.live/beagle/

Basically what I’ve been saying since OldJob forced LLMs down our throats and pegging performance to usage metrics: why the fuck are we handing deterministic processes to probabilistic systems when it should be the other way around (using probabilistic systems to design deterministic ones)?

LLMS should be abstracted out of a process as soon as practicable, replaced with deterministic processes or procedures. Otherwise you’ve built the world’s most fragile process at the mercy of token cost, vendor hostility, geopolitics, and model deprecation.