Building Engram: encrypted memory for agents, no accounts required

I’ve been thinking for a while about a specific problem: AI agents don’t have memory that persists across sessions in any meaningful way. You can bolt on a database, sure, but then you have an account management problem. Who owns the memory? How does the agent authenticate? What happens when the agent runs somewhere else?

The standard answer is “give the agent an API key.” That’s fine, but it’s boring and it creates a bunch of second-order problems — key rotation, per-agent scoping, the fact that you’re now running a user auth system on behalf of your agents.

The idea for Engram came out of a different starting point: what if the agent’s wallet is its identity?

What it is

Engram is an encrypted persistent memory API for AI agents. No accounts, no API keys, no OAuth. An agent registers by signing a message with its Ethereum wallet. That signature is used to derive an AES-256-GCM encryption key via HKDF. The agent stores and retrieves encrypted blobs, addressed by SHA-256 hash. Each paid operation costs fractions of a cent in USDC via x402 micropayments on Base.

The server never sees the plaintext. The server never stores keys. Derive, use, discard.

The pitch is: any agent with a wallet can use this. You don’t need to know about it ahead of time. You don’t need to provision it. The agent discovers it, pays for what it uses, and leaves.

What got cut immediately

The initial implementation had a few things I was excited about that didn’t survive contact with reality.

IPFS. I wanted content-addressed storage on IPFS so blobs were genuinely decentralized. In practice, Helia (the JS IPFS implementation) added significant startup time and complexity for what amounted to a local node — I wasn’t actually pinning to the public network in alpha. I replaced it with a flat file blobstore keyed by SHA-256 hash. Same addressing scheme, none of the infra overhead. I’ll add real decentralized storage later when it matters.

pgvector + semantic search. The original design had vector search so agents could retrieve memories by semantic similarity rather than exact hash. This is actually a compelling feature. It also added a Postgres dependency, a pgvector extension, an embeddings pipeline, and a bunch of dimension-partitioning logic to handle multiple embedding models. For alpha, I cut all of it. The API has retrieve-by-hash. That’s enough to build on. BYOE (bring your own embeddings) — if you want semantic search, compute the vector yourself and store the hash in your index.

Both of these decisions made the thing shippable. I have a bad habit of building the interesting parts before the boring parts work.

The crypto plumbing

x402 is a payment protocol built on top of HTTP. The rough idea: a server responds to an unauthenticated request with a 402 Payment Required and a payment details header. The client pays on-chain and retries with a receipt header. The server verifies the receipt and fulfills the request.

In practice, wiring this up was straightforward once I understood the facilitator model — there’s a public facilitator at x402.org that handles payment verification so you don’t have to run your own. The tricky part was getting the middleware ordering right: payment verification needs to happen before auth, but after body parsing, and the error responses need to be structured correctly or the client can’t parse them.

There was a bug early on where a dead search route was accidentally inside the payment middleware scope, meaning the middleware was trying to enforce payment on a route that shouldn’t have required it. That kind of thing is easy to miss because the route itself “works” from the outside.

The wallet signature auth was more interesting. The agent signs keccak256("engram:auth:v1:" + agentId + ":" + timestamp) and includes the result in a header. The server verifies the signature, checks the timestamp is within five minutes, and confirms the recovered address matches the registered wallet. This is simple, stateless, and actually secure — you can’t replay a signature because of the timestamp, and you can’t forge one without the private key.

Deploy infrastructure

The deploy story went through a few iterations. I ended up with a blue/green setup behind Caddy — two Docker containers, one active, one idle. On deploy, GitHub Actions builds a new image, starts it on the idle slot, waits for a health check, then Caddy’s admin API swaps the upstream. Zero downtime.

There was a fun bug where Caddy’s admin API returns success even when the upstream config hasn’t fully propagated, and there was another where the deploy script was using localhost instead of 127.0.0.1 for the admin API call, which failed inside Docker’s network namespace. IPv4 explicit everywhere now.

The Caddyfile itself is managed in the repo and deployed as part of CI — which means the blue/green swap also has to keep the Caddyfile in sync. That required an extra fix late in the process when I noticed the Caddyfile on disk was drifting from what Caddy was actually serving in memory.

What’s interesting about the model

The wallet-as-identity pattern sidesteps a whole class of problems. Agents are increasingly running autonomously — spinning up on cloud infra, calling APIs, doing work. The agent needs credentials to do anything, but credentials are a human-facing concept. Wallets are native to agents. They’re asymmetric keys with a payments layer built in.

x402 specifically is interesting because it’s HTTP-native. You can retrofit it onto any API. The agent doesn’t need to know about Engram ahead of time — in principle, it can discover a 402 response, understand the payment requirements, and decide whether to pay. That’s a different kind of API interaction than anything we’ve built before.

I don’t know yet how useful Engram ends up being. It’s running, it’s live at engram.dkta.dev, and there’s an SDK. What I’m more confident about is that the underlying pattern — wallet identity, micropayment-gated operations, encrypted blobs — is the right shape for agent infrastructure. The specifics can change.