Most of the AI security conversation is about what happens in the moment: a trick email, a bad answer, a mistake you can catch. The quieter danger is what persists. If your AI system keeps knowledge between sessions, in a memory store, a document library, or a retrieval database, then whatever gets written there becomes the context every future session trusts.
OWASP’s 2026 guidance states it flatly: one tainted entry in persistent memory or a retrieval corpus reaches every future session that reads it. The corruption doesn’t announce itself. It sits in your systems wearing the same clothes as your real data.
Why this is an agent problem, not a chatbot problem
A chatbot with no memory forgets you the moment the conversation ends. An agent that accumulates knowledge, project notes, vendor documents, past decisions, internal wikis, is building the very thing an attacker wants to own. OWASP lists Memory & Context Poisoning as its sixth agentic threat (ASI06) in the 2026 Top 10, with real-world cases including memory attacks against Gemini.
The mechanics are simple. Anything that can write to the knowledge base can poison it: an indirect prompt injection that saves a “note,” a compromised integration, one careless import from an outside source. From then on the poison isn’t an incident, it’s part of how the system thinks.
What makes it dangerous in a business setting
Three properties compound. First, persistence: the bad instruction survives across sessions, sometimes triggering weeks later in an unrelated task. Second , trust: retrieval systems surface stored content as authoritative, so the model treats an injected line as company truth. Third, scale: poison one shared library and every employee’s assistant inherits it.
Containment, in order
- Treat knowledge sources as production systems. Who and what can write to them? If the answer is “anything with access,” you don’t have integrity controls, you have hope.
- Change review for knowledge. Additions to shared context deserve the same scrutiny as code deployments, especially anything sourced from outside the company.
- Keep humans out of the write path for sensitive stores, or at minimum behind approval. Fast is the enemy here.
- Version and roll back. The Five Eyes agencies’ 2026 agentic AI guidance recommends versioning and rollback mechanisms so you can revert to known-good behavior when something goes wrong. If you can’t answer “what changed in the knowledge base last week,” you can’t investigate a poisoning, let alone undo one.
- Watch for drift. Compare what the system currently believes against its approved baseline. Sudden confidence about things nobody taught it is a signal.
The private-AI angle
This risk is a core argument for keeping AI on infrastructure you control. Private AI agents, models running on systems you own or rent under your name, with knowledge bases you administer, put the write path inside your perimeter, where your existing change controls already apply. It doesn’t make poisoning impossible. It makes it investigable, and that difference decides whether you clean up in an afternoon or a quarter.
If you’re building an agent that accumulates knowledge, talk to us before the knowledge base becomes load-bearing. Untangling trust after the fact costs far more than designing for it up front.