An assistant processes a conversation inside a trusted execution environment (TEE). The response comes back, the session ends, and the environment disappears. That is a useful model for private inference. It is a poor model for an assistant that is supposed to remember what you said tomorrow.
That gap has been on my mind after reading three recent designs: Google’s plan for secure server-side memory, Meta’s Private Processing architecture for AI glasses, and Amazon’s Bee Private Compute paper. Each starts with a different product. All three have to deal with the same fact: a personal assistant needs context that lasts longer than one inference request.
The TEE is important here, but continuity is not a property the hardware provides. Hardware can isolate code and data while a workload runs. The rest of the system has to decide what to retain, where to put it, who holds the keys, and what must be verified before that memory is opened again.
Three versions of the same problem
Google describes an addition to Private AI Compute that will enable persistent memory across devices. Its earlier architecture was stateless: context was discarded when a task ended. The new design puts memory in dedicated encrypted storage, with unlocking keys held on the user’s devices. When needed, an isolated cloud environment temporarily decrypts the data, processes the request, and encrypts updated context before storing it. Google also describes a public record of server software that devices can check before releasing personal data. This is a technical direction Google has published, not a claim that every Google assistant already works this way.
Meta is approaching the problem through AI glasses. A pair of glasses can be present throughout the day, but the experiences Meta describes, including recall across sessions, require more compute and durable context than the device can keep locally. Its Private Processing design encrypts saved output with user-provided keys before it leaves the TEE. On a later request, the device supplies a key to an attested environment that can retrieve and query the memory. Meta says it built the query engine inside the protected boundary, partly because an ordinary database can reveal access patterns even when its records are encrypted. Anonymous credentials and a third-party OHTTP relay make it harder to identify a user’s request and route that user to a chosen TEE. Separately, a publicly witnessed transparency ledger records deployed software measurements; Meta makes the corresponding binaries available to researchers in its security program under agreement. The glasses experiences are framed as work toward future capabilities.
Bee has a different rhythm. Its wearable assistant is designed to build context and perform background work, including when the user is not in an active chat. Bee’s paper says user devices generate the root secrets and release scoped server-processing keys only after verifying the receiving confidential VM. Long-term content stays encrypted at rest. The system holds scoped server-processing keys inside confidential VMs and Confidential Redis keystores for at most seven days of inactivity. That lets background work continue without a fresh request from the device for every job. If the client stops refreshing those keys, jobs that need plaintext are supposed to fail closed when the keys expire.
These designs have different product requirements and different security boundaries. None establishes that private memory is a solved problem.
Follow a saved memory
With a stateless private request, the trust path is comparatively short. A device verifies the execution environment, sends encrypted input, receives a result, and the service discards session state. Even that path is hard to implement correctly. Persistent memory adds another set of transitions.
First, the assistant decides what should become memory. It might save a preference, summarize a conversation, or update an index used for later retrieval. Encryption does not tell us whether that summary is accurate, whether a private detail should have been retained, or whether information from another person belongs in the user’s long-term context. Those are product and application decisions made by code inside the protected environment.
Next, the memory leaves active compute. If it is stored as ciphertext outside the TEE, the provider may be unable to read the content, but the storage layer can still expose metadata: record counts, timestamps, sizes, and sometimes which records are accessed together. Bee’s own threat-model table explicitly lists metadata that can remain visible. Meta’s decision to put queries inside its TEE boundary addresses part of the access-pattern problem. Neither design makes the general metadata problem disappear.
Later, a new workload needs the memory. The original TEE session is gone. The device now has to decide whether this new instance is allowed to receive a key. Hardware attestation can report a measured runtime, but somebody must define which measurements are acceptable and make changes to that policy visible. A genuine attestation for newly deployed code is useful evidence only if the user has a reason to trust that code’s behavior with old memories.
Finally, the assistant may act on what it recalls. A memory can influence a recommendation, a search, or an external tool call. Confidential processing limits who sees the plaintext along the approved path. It does not establish that the remembered fact is correct or that a downstream action was authorized. That distinction matters as assistants become more proactive.
The security object has changed. It is no longer one request and one response. It is a memory lifecycle that may span devices, software versions, and weeks of interaction.
A reference pattern for the lifecycle, not a diagram of any one company’s implementation.
Where the trust boundary stretches
Key custody is the first question I would ask. “The device holds the key” sounds simple until a person owns a phone, laptop, and glasses, loses one of them, or replaces all three. How are those devices enrolled? Can an old device be revoked? What happens when the client stops refreshing a processing key but the assistant still has background work queued? Bee’s bounded server-processing window is an explicit answer: previously released keys remain available for at most seven days of inactivity, and work that needs plaintext stops after they expire. Other products may choose different windows.
Software changes are the second question. Memory is valuable precisely because it survives upgrades. A new model, storage engine, or retrieval pipeline may see years of accumulated context. Attestation tells a client what is running; transparency records can make unexpected deployments visible. Neither by itself determines whether the new code should be allowed to read every old memory. Key-release policy has to carry that judgment.
Retention and deletion are the third. A user may want to delete a fact the assistant learned, not merely hide it from the interface. That fact might exist in an encrypted record, a derived search index, a summary, or a queued background job. Bee’s paper distinguishes encrypted canonical storage from derived indexes and describes removing certain caches when processing keys become unavailable. The broader question for any design is how a user can tell which copies and derivatives have actually been removed. Expiring a decryption key can make ciphertext inaccessible, but that is not the same operation as deleting every physical copy, and any plaintext already shared with an external service follows that service’s rules.
There is also a difference between operator privacy and complete privacy. These designs aim to keep infrastructure operators and non-attested services away from plaintext. The approved application still has to process the user’s information, and a compromised device or a bug in that application can still matter. Bee states that limitation plainly in its threat model. A TEE narrows who can access data; it does not make the code inside it infallible.
The TEE answers the execution question. The other two boundaries need their own controls and evidence.
What I would want to verify
For a private assistant with durable memory, I would want answers to five concrete questions:
- What becomes memory? Can the user inspect and correct it, and does the system distinguish source material from an AI-generated summary?
- Which code can reopen it? Does the client verify attestation and an independently visible software record before releasing a key?
- How long can the server work without a key refresh? Is the processing window disclosed, bounded, and revocable?
- What leaks outside the protected boundary? The review should cover storage metadata, query patterns, logs, and connected services.
- What does deletion mean? Does it cover canonical records, derived indexes, queued work, and any data already sent elsewhere?
I do not expect every product to answer these questions the same way. A glasses assistant, a cross-device assistant, and a background wearable will make different choices. I do expect the choices to be visible enough that users and independent researchers can evaluate the claims being made.
Confidential computing gave private AI a credible way to protect data during execution. Persistent assistants force us to examine what happens before and after that execution: the memory selected, the key released, the software updated, and the data eventually forgotten. That is the trust boundary I will be watching as these designs move into products.
The views expressed here are my own and do not represent those of my employer.