Why "deletable" is the most important word in LLM systems
You can delete the document. Its shadow stays everywhere. What changed when I stopped trusting the word "deleted."
A customer asked me a plain question: if we leave, can you delete everything the system knows about us? I said yes before I had earned the right to.
Then I went and checked. The source documents, sure, those delete. But the chunks cut from them were still in the index. The embeddings were still sitting in an ANN graph. A cached answer still quoted a paragraph whose document no longer existed. A summary written two weeks earlier still carried a fact I had been asked to forget. The source was gone. Its shadow was in five other places.
The source is gone. The shadows aren't.
That question turned out to be the cleanest test I have for whether an LLM system is actually under control. Not retrieval quality. Not model choice. Can you delete a thing and have it be gone?
If you can't, you don't really know what your system knows. And a system whose knowledge you can't enumerate is one you can't debug, can't reason about, and can't stand behind when someone asks.
Most stacks are very good at adding state. They are terrible at removing it. That asymmetry doesn't announce itself on launch day. It shows up the first time someone asks you to take something back, and you find out how many copies you made without meaning to.
What changed when I made delete a first-class path
A prompt you can't roll back is a prompt you're scared to touch. The same fear applies to the whole system: data you can't cleanly remove is data you stop reasoning about. So I started designing the delete path on day one, next to the write path, instead of discovering it the week a customer leaves.
In practice it lives in four layers, easy to hard.
1. Prompts — you may have solved this already
I argued before that prompts deserve migrations, not vibes: versioned, reviewed, reversible. Deletability is the same argument from the other side. If every prompt version is a checkpoint, then "remove this instruction" is a diff you can review and revert, not a leap of faith. If you've put your prompts under version control, this layer is done. It's the easy one.
2. Documents — the source is the easy 10%
One source document fans out into chunks, embeddings, an index entry, maybe a cached answer that quoted it.
Deleting the document is the easy 10%. Its derivatives are the 90% everyone skips.
The trap is that derived data is a copy you forgot you made. The rule I follow now is dull and load-bearing. Every derived artifact carries a back-reference to the source it came from, and deletion walks those references. Get that right and removal is a clean cascade. Get it wrong and "deleted" means "mostly deleted," which is the worst kind of wrong, because it reads as done.
3. Derived knowledge — the layer that lies
Summaries. Agent memory. A fine-tune. A "facts about this account" cache. This is the hard layer, because by the time a fact reaches a summary, the thread back to its source is usually cut.
I don't have a clean solution here, and I'd distrust anyone selling one. What I have is a constraint: be wary of any process that turns sources into derived knowledge without recording which sources fed it. If you can't answer "what would this summary look like if document X had never existed," you've built something you can't fully retract. That can be an acceptable trade. It should never be a surprise.
4. Tenants — the architecture answers before you do
In a multi-tenant system, "delete everything about this customer" is the whole game, and your isolation model wrote the answer before the request arrived.
Shared tables with a tenant_id column? Deletion is a DELETE ... WHERE tenant_id = ? across every table, index, and cache, and "everything" is only as complete as your memory of where that ID leaked. Database-per-tenant? Deletion is a DROP DATABASE. One statement. Provably total. Easy to explain to an auditor.
I picked database-per-tenant for Clarytia for a few reasons. The one that ages best is that it makes the delete path trivial and complete. The expensive isolation choice on day one is what turns an existential question into a one-line operation later.
Where it bit me
I didn't reason my way to this. I got burned into it.
Early on, a tenant offboarded. I ran the deletes, watched the rows disappear, and called it done. Three days later, testing something unrelated, an answer surfaced a sentence from one of their documents. The document was gone from its table. The answer cache had outlived it. Nobody outside ever saw it. But I had told myself "deleted" when I meant "deleted the part I remembered."
That cache was the cheapest possible version of this failure. The expensive versions involve embeddings, summaries, and a customer's lawyer. I built the cascade after that, not before, which is the honest order these lessons usually arrive in.
Design the delete path first
Here's why I lead with deletion and not with features. The delete path is the most honest audit of your architecture. To remove something cleanly you have to know everywhere it went, and a system whose data flow you can trace backward is one you can actually operate.
You delete what a system knows. You keep the record of what it did.
Confusing those two is its own failure, and immutability has its place: audit logs and provenance should never be erasable. Reversibility isn't "make everything mutable." It's deciding, on purpose, what must be removable, and paying for that instead of for all of it.
So before the exciting questions about retrieval and models, I now ask the dull one first: when this is over, can we make it gone? The answer tells me more about whether I'm in control than any benchmark will.
If you've actually solved layer 3 — removing derived knowledge cleanly, not just the sources — I want to hear how. It's the one I still find hard, and I don't think the field has a real answer yet.