These are my notes on context and long-term memory. I wanted to connect several questions that are easy to blur together: what a model sees in one call, how progress is preserved as a task continues, and how information is saved, retrieved, and corrected across sessions.
“Longer context windows” and “agents with memory” both help systems handle more information, but implementing them still involves many details. Saving a complete conversation does not mean the next call will read all of it. Writing a fact to a vector database still leaves decisions about whom it belongs to, when it applies, and how it should be used later.
The article first establishes the concepts and how context is constructed for each call, then examines long-running tasks, memory layers, and storage formats before returning to these mechanisms through open-source implementations and evaluation. If you are working through this topic too, I hope you will read along and share your understanding and practical experience.
1 Conceptual Boundaries: Context, Task State, and Long-Term Memory
Current context: the information the model actually processes in this call
The context of a model request is the input actually used for that inference call. It can include system instructions, the current question, historical messages, tool definitions, tool results, and retrieved documents and memories. The host program or server-side session layer organizes this material, and the model uses it to generate an answer or its next action.
The context window sets a capacity boundary for this input and any other content required by the interface. Even before the window is full, repeated, outdated, or contradictory information can interfere with decisions, so context management also involves selection and arrangement. Anthropic’s guide to context engineering discusses this ongoing process of maintaining the information available to a model.
Knowledge in model parameters is a separate source. Ordinary inference uses the current input; saving a document or conversation in a program does not automatically write it into the model’s parameters. This article focuses on external information that a host can read, write, and supply to the model again.
Task state: allowing work to continue
Task state records where the current work stands, such as its goal, constraints, results obtained, outstanding items, and necessary messages. The application maintains it, either only within a process or in persistent storage so that work can resume after an interruption.
State can contain information omitted from the current call. A program can save complete messages and intermediate artifacts while selecting only a summary, current progress, and relevant results for the model. A field’s presence in a state object means only that the application can read it. Whether it enters a model request depends on the assembly logic.
Long-term memory emphasizes future value across sessions, such as stable preferences, confirmed facts, useful past experience, and reusable methods. An unfinished task may last several days while its progress still belongs to the same task state. Retention duration alone cannot determine the category of information.
| Object | Main question it answers | What it may hold | How it is used in the current call |
|---|---|---|---|
| Model request context | What can the model see now? | Instructions, questions, selected history, tools, and material | Directly participates in this inference call |
| Thread or task state | Where does this work stand? | Messages, goals, progress, results, outstanding items, and recovery positions | The application or tools select relevant parts |
| Long-term memory across sessions | What will still be worth using later? | Facts, preferences, event-based experience, procedures, and rules | Read according to the task, or loaded as a suitable always-present subset |
| Original records and working material | Where is the evidence when something needs checking? | Complete conversations, files, tool outputs, and logs | Accessed through references, queries, or file reads |
Original records provide sources and a basis for checking other representations. Some of this material can contribute to long-term memory, but what remains worth using still needs to be selected.
Scope: saving within a thread and using information across sessions
LangGraph’s Persistence and Stores offer a concrete distinction: a checkpointer saves snapshots of a thread’s graph state, while a Store holds application-defined data across threads. In this documentation, “short-term” primarily refers to thread scope. Thread state itself can also be persistent.

Source: LangChain, Memory overview, MIT license. The original image is unchanged. Checkpointer and Store represent thread state and cross-thread memory respectively. Stored information still needs to be selected for model input according to the task.
This distinction also reminds us to examine two questions separately: where data is saved and how the program uses it. An in-memory object can be reused across calls without surviving a process restart. A persistent database can hold extensive content without being queried in the next request.
2 Constructing Context: What the Model Should See in Each Call
Input assembly: from available information to the current request
An agent call can start with the current question and task state. The application verifies identity and the accessible scope of material, reads existing progress, retrieves memories, documents, or tool results as needed, and combines them with suitable instructions and interaction history into a request.
After the model returns tool calls, the host executes them and records new observations and necessary state changes. For the next call, it decides again what should be retained, what can be shortened, and what needs further reading. Context therefore changes as the task develops.

Original workflow diagram: input is selected and assembled for the task in each call, while model and tool interactions produce new observations and state. Writing long-term memory is optional, and saving and reading can occur in different rounds.
Retrieval can be performed by the application before a model call or triggered by the model through tools as needed. The former makes it easier to supply material already known to be necessary; the latter lets the model continue searching based on what it has seen. Anthropic’s discussion of just-in-time retrieval shows how file paths, links, and queries support on-demand reading. These choices can be combined and do not require all material to be vectorized in advance or placed in the window at once.
Token budgets: leaving room for the work ahead
A budget must account for more than conversation text. Tool descriptions and parameters, long logs, retrieved material, and the current task instructions can all occupy input space. Planning also needs to reserve room for output and any other tokens required by the interface.
The following relationship illustrates budget allocation. Actual limits and counting rules depend on the model and interface:
Here, is the window budget used for this plan, and includes output and other necessary reserves. The relationship shows how different parts compete for space. Some interfaces also impose separate limits on input, output, or other content, so adding the parts alone cannot establish that a request is valid.
Limiting history by message count can overlook differences in length. One tool log may be longer than many brief exchanges. Tool sets can also be selected for the task to keep unrelated descriptions from occupying the window indefinitely. Repeated input material can be merged while retaining its source associations.
Prompt caching can reduce computation or costs for repeated prefixes, depending on the service implementation. Cached content may still occupy the context window, and caching does not determine whether the material is relevant. Claude Context Editing also explains that removing old content can affect an existing prefix cache, so benefits need to be assessed against actual costs.
Arranging information: scope and status matter alongside relevance
Material selected for context should let the model distinguish its source and purpose. A user’s latest requirement, a long-term preference, a tool observation, and a third-party statement in a document have different attributions. They can be labeled separately while retaining their applicable project, time, and status.
Recent content is not necessarily the most reliable, and the most similar memory may not apply to the current subject. An explicit requirement in the current task may cover only this interaction, while a long-term default preference applies when no more specific requirement exists. The application needs to preserve these scopes so that temporary instructions do not become permanent rules.
History trimming must also respect the interface’s message protocol. Tool calls and results are associated through identifiers, and unfinished interactions need to be retained intact rather than arbitrarily cut by text length. Externally retrieved content and generated memories should continue to be treated as material. The host enforces the system’s identity, authorization, and execution boundaries.
LangChain’s overview of context engineering organizes common methods as write, select, compress, and isolate: write information outside the window, retrieve it selectively, compress current input, or keep local processes in separate contexts. This is a useful perspective, while the methods still need to work together around the same task.
3 Managing Long-Running Tasks: Trimming, Compression, and External Offloading
Strategy differences: what is shortened and what remains
Messages and tool outputs accumulate as a task continues. Several different operations can help manage them:
| Strategy | Main action | What is retained and lost |
|---|---|---|
| Sliding window | Includes only a recent stretch of interaction | Recent content remains intact; earlier constraints need to be saved separately |
| Rule-based trimming | Removes repeated, outdated, or specified categories of content | Retention rules determine what disappears; associated messages need checking |
| Summarization and compaction | Replaces older history with a shorter representation and assembles subsequent input | Allows the task to continue, but summaries select and omit information |
| Structured task state | Separately maintains goals, constraints, progress, unresolved items, and sources | Key fields are easy to check but still need updates as work actually proceeds |
| External offloading | Keeps source text in files or record storage and includes necessary content and references in the current call | Reduces repeated input but requires the source text to remain findable and readable |
| Subtask isolation | Keeps local exploration in separate contexts and returns relevant findings | The main task receives less process detail, but handoffs may omit conditions |
These strategies can be combined. Keeping a long log outside context while recording failed items and the next step in state is a common pairing. The file location enables rereading; the state explains why that result matters to the current task.
Compression and continuation: preserving enough state to keep working
Compaction usually has two parts: distilling older history and rebuilding subsequent context around the distilled result. Merely appending a summary to the end of complete history leaves the original history occupying space.
A summary should prioritize information that affects the next step: task goals and constraints, confirmed decisions, completion status, unresolved questions, and necessary source references. Recent tool interactions still in progress can be retained unchanged as required by the interface. The Claude Compaction documentation shows application-controlled and threshold-based triggering, along with specific conditions such as retaining recent content.

Original relationship diagram: a summary selectively represents history, while original records can remain outside the window. The next call continues from the summary, key state, and recent interactions, returning to referenced source records when details are needed.
Triggering compression requires room for summary generation, possible tool results, and the next call. Organizing information at task stages can also reduce the risk of losing too much at once. A fixed percentage does not suit every model, interface, and task.
Task state can use YAML like the following. The fields illustrate responsibilities; an application could also use JSON or database records:
thread_id: demo-thread
goal: Complete the document review and report the results
constraints:
- Preserve content explicitly requested by the user
completed:
- Structural checks are complete
open_items:
- Two citations still need checking
next_action: Reread the failed items from the citation check
artifacts:
- ref: logs/reference-check
meaning: Original check results
summary_cursor: message-42summary_cursor marks how far the summary extends, helping connect later messages with the existing summary. completed and open_items preserve the actual work status. Planned actions, completed modifications, and verified results must remain distinct rather than all becoming “done” during compression.
Reading external material again: keeping both location and content usable
The account by the original Manus authors describes saving material in the file system and retrieving it on demand through paths or URLs. For long documents, search results, and logs, the current request can include only summaries, relevant passages, and references for rereading.
Those references also need to lead to the appropriate content. Files may change, URLs may stop working, and old locations may point to new results. For material that affects decisions, source identifiers and necessary content identity can be retained and checked on rereading to ensure that it still corresponds to the original record.
Summaries can also drift through repeated retelling. A fact with conditions may become only a conclusion after several rounds of compression. Saving key constraints in separate fields, preserving original evidence, and rebuilding a summary from source text when needed is more useful than continuing to compress a summary that has already become distorted.
Deletion boundaries: cleaning input, modifying state, and removing records
Omitting old messages from the current input changes only what the model can see now. LangGraph’s memory management guide further distinguishes trimming before a model call from updating graph state with RemoveMessage.
Updating current state also does not mean that historical checkpoints, logs, or other derived records have been removed. Server-side tool result clearing may use placeholders to preserve message associations; such mechanisms need to follow the relevant interface. Freeing window space, modifying state, and deleting stored content each require a clear target.
4 Memory Layers: Working Context, Task State, and Long-Term Memory
Functional layers: how close information is to the current call
From the perspective of an agent’s operation, memory can be divided into three layers according to where it contributes. The first is the context for the current work, the second is the state that maintains continuity within a task, and the third is long-term memory accumulated and used across sessions. This is a functional view for understanding the system, not a requirement to deploy three storage services.

Original layer diagram: the functional layers explain where information participates in work; semantic, episodic, and procedural categories within long-term memory explain what is saved. External information enters the current context after selection, and the same content can have different representations in several locations.
| Functional layer | Main contents | Maintenance | How it is used |
|---|---|---|---|
| Working context | Current question, selected instructions, recent interactions, relevant evidence, and memories | Selection, arrangement, and budget control for every request | Processed directly by the model |
| Thread or task state | Messages, summaries, goals, progress, temporary results, and recovery information | Updated during execution, with checkpoints available | Supplies fields needed by later requests and tools |
| Long-term memory | Facts, experiences, and methods useful across sessions | Selective writing, revision, organization, and removal | Always-loaded content, reading by key, or relevance-based retrieval |
“Always loaded” and “long-term” are not mutually exclusive. A stable, frequently used rule can be saved in long-term memory and also loaded into context every time. Extensive historical experience is better retrieved when a relevant task arises.
When a task ends, its entire state does not automatically move into the third layer. A temporary path, an unconfirmed guess, or unfinished work may matter only to the current task. Experience worth retaining needs selection and associations with the original task and evidence.
Content types: semantic, episodic, and procedural memory
Long-term memory can also be categorized by its use. LangMem’s conceptual guide borrows the terms semantic, episodic, and procedural to explain different purposes. They help with designing storage and reading, but are not fixed API types shared by every framework.
| Type | Core contents | Common representations | What to check when using it |
|---|---|---|---|
| Semantic memory | Facts, preferences, object attributes, and relationships | Profiles, individual fact records, and entity relationships | Subject, scope, source, and whether it remains valid |
| Episodic memory | Past events, task experiences, and related results | Events with times and sources, experience summaries, and success or failure cases | Original conditions, actions, actual outcomes, and applicability to the current task |
| Procedural memory | Reusable steps, rules, strategies, and communication habits | Rule files, workflow descriptions, and behavioral conventions in prompts | Scope, source, execution conditions, and subsequent corrections |
Semantic memory focuses on “what is known.” It can hold a project convention or a default preference explicitly stated by a user. Extraction needs to preserve the subject and conditions. A requirement applying only to the current task must not become a global preference after its scope is omitted.
Episodic memory focuses on “what happened before.” Useful experience often includes the situation, action, and outcome together. A statement that “this method works” without the problem it addressed or the result it produced makes later reuse difficult to judge. Original conversations can serve as sources, while selected experience records can be retrieved as cases to help the model understand similar tasks.
Procedural memory focuses on “how things are usually done.” It can come from user-provided rules or organized experience that has been checked. Generalizing one successful case into a reusable procedure is a new inference. Its conditions need to remain explicit; success once does not establish that every task should follow it. Model-generated rules also need attribution and remain subject to the current task and the system’s permission boundaries.
Organization: how content types relate to storage media
The same type of memory can be organized in different ways. A profile groups related attributes into a relatively complete object, making the current overview easy to read. A collection separates facts or experiences into individual records for independent retrieval and updates, but requires handling duplication, omissions, and conflicts. Hierarchical files can separate always-loaded rules, topic indexes, and details for on-demand reading.
Vector retrieval, keyword indexes, and relationship graphs address organization and lookup. Semantic memory can be stored as JSON or vectorized. Episodic memory can remain in files and also be indexed. “Using a vector database” describes a retrieval implementation; understanding the memory system still requires examining its content, scope, and maintenance logic.
Start by deciding why each item is retained and how it can be located and corrected, then choose the required indexes. For a small set of stable rules, reading files or records with known keys can be sufficient.
5 Memory Management: Storage, Writing, Updates, Retrieval, and Forgetting
Storage formats: connecting primary records, sources, and indexes
Storage first needs to distinguish two objects: the primary record recognized by the application and the indexed representation that helps locate it. The primary record holds content, ownership, sources, and current status. An index can contain vectors, keywords, or relationships, linked back to the primary record by identifiers. They can share a database or be stored separately.
The following formats illustrate common organizations. They are not fixed file conventions from a particular framework, and a system does not need to use all of them:
| Content | Possible formats | Key fields or structure |
|---|---|---|
| Task state and compressed summaries | JSON, YAML, or structured database fields | Task ID, goals, constraints, progress, outstanding items, source locations, and summary coverage |
| Original interactions and event records | JSONL, message tables, or event tables | Event ID, subject, event type, content, recorded time, and source references |
| Reusable facts and preferences | JSON documents, key-value records, or relational tables | Memory ID, type, content, scope, source, applicability period, and status |
| Episodes and cases | Event records or experience summaries | Situation, action, outcome, occurrence time, and related original events |
| Procedures and rules | Markdown, structured workflows, or prompt configurations | Scope, prerequisites, steps, sources, and revision information |
| Retrieval representations | Vector collections, full-text indexes, graph nodes and edges | Primary record ID, indexed content, filter fields, and necessary representation configuration |
JSONL places each record on a separate line and is suitable for appending events individually. For example:
{"event_id":"evt-17","type":"user_message","content":"Documentation for this project uses Simplified Chinese by default","source_ref":"conversation/demo#message-17"}
{"event_id":"evt-18","type":"memory_written","memory_id":"mem-language-1","source_refs":["evt-17"]}These two lines are teaching data. The first preserves the source of the information, and the second records the corresponding memory write. An application can implement the same relationship in a message table. With many events and complex queries, there is no need to insist on a single log file.
A fact formed from this source can be represented as follows:
{
"memory_id": "mem-language-1",
"kind": "semantic",
"scope": {
"user_id": "demo-user",
"project_id": "demo-project"
},
"content": "Documentation for this project uses Simplified Chinese by default",
"source_refs": ["evt-17"],
"recorded_at": "<system recording time>",
"valid_from": null,
"valid_until": null,
"status": "active",
"supersedes": null
}recorded_at indicates when the record was stored, while valid_from and valid_until express its period of applicability. Unknown values can remain empty; the occurrence time should not be inferred from the recording time. source_refs preserves the original evidence, while status and supersedes help express current validity and replacement relationships. Whether every field is needed depends on how the application actually corrects and queries information.
Procedural memory can use readable Markdown:
# Document Citation Checks
Scope: Document review for the current project
Source: Confirmed project rules
1. Check whether key claims are linked to their corresponding sources.
2. Reread the source text when details need confirmation.
3. Record unconfirmed citations without marking them as passed.Rule files also need stable locations and scopes. Sources can be maintained through adjacent metadata or structured fields within the file. Writing content in Markdown does not automatically grant it greater instructional authority.
Writing: turning an interaction into a reusable record
Writing can be understood as “source event → candidate extraction → scope and value assessment → comparison with existing content → saving the result.” Candidates may be facts explicitly stated by a user, tool observations, or model inferences. Those attributions should stay with the content.
Extraction first needs to consider how the information will be used later. Temporary progress usually remains in task state. Preferences, facts, or experiences with an explicit ongoing scope can become long-term candidates. Excessive extraction increases duplication and irrelevant retrieval; insufficient extraction can leave later tasks unable to find needed information.
When comparing existing records, first limit the comparison to the same subject and scope, then determine whether new content is a duplicate, an addition, a change, or a disagreement. Deduplicating identical text is relatively direct. Two semantically similar records may both be valid because their projects, times, or conditions differ. Merging should retain their respective evidence rather than leave an overly broad generalization.

Original management diagram: writing and reading connect through primary records with sources and scopes, while retrieval indexes locate the corresponding content. Addition, revision, invalidation, and deletion are different operations. The application and chosen implementation make the specific decisions.
After a successful save, related retrieval representations still need to correspond to the primary record. Updating content while retrieving old vectors, or leaving an index to present an invalidated record as a current fact, affects the next use. Synchronization, rebuilding, or status-based filtering depends on the storage design, but the object ultimately read back should be checked.
Updates and conflicts: revising content or its scope
When a user corrects a default rule, the application can update the original record directly or retain both old and new records with a replacement relationship. The former makes the current value easy to read; the latter makes historical questions easier to answer. The choice should follow actual needs for traceability and source references.
Updates cannot rely only on the latest write time. Information added today may describe a past state, and two projects may have different rules. Confirm whether the subject, attribute, time, and scope match before deciding to replace a record, retain both, or leave a disagreement unresolved.
For the earlier language preference, if the user asks only to “write this paragraph in English,” the current task can follow that instruction while retaining the long-term default. The long-term record should change only when the ongoing rule is explicitly changed. This avoids interpreting every local choice as a change in long-term preference.
Record status, replacement relationships, and indexes also need to be considered together. Reads following a primary record update should obtain the new content. With asynchronous indexing, retrieved candidates can be checked against the primary record for current applicability. The need is for explicit update and read semantics; not every small application needs a complex versioning system in advance.
Retrieval and use: from storage back into the current context
Reading can start with a known identifier or with search. A small set of fixed rules suits direct loading, a known user or project profile can be read by key, and extensive facts and experiences can use keyword, vector, or relationship retrieval.
First establish accessible scope from a trusted identity, then select candidates according to the current question, state, and time requirements. After retrieval, candidates still need deduplication, validity checks, and selection within the budget. The material ultimately supplied to the model should retain its sources and key conditions.
Reading can also proceed in stages. A topic index or short summary can let the model decide whether to open details, reducing the cost of loading all memories every time. Conversely, repeated searches for a rule needed almost every time can add unnecessary latency; keeping a suitable rule always loaded is more direct.
Retrieving the right content is followed by another question: whether the model uses it correctly. When memory conflicts with current requirements, sources disagree, or evidence is insufficient, the system can explain differences, search further, or seek clarification. Similarity helps locate material; sources and conditions still need checking.
When to write: within the request or in the background
Writing within a request makes newly confirmed information available promptly but adds work to the current path. Background organization can extract and merge information after an interaction, but memory updates may still be unfinished when a quick response is returned.
LangMem’s delayed processing guide demonstrates deferred and combined processing. With background writes, the application needs to distinguish “submitted for processing” from “memory is readable.” If the next step immediately depends on new information, it can remain in current state first and connect to long-term storage later.
Background organization can also find duplicates, split oversized records, or summarize experience. New conclusions from that work still need links to original sources. A fluent model-generated summary does not establish that it is more accurate than the original records. Save failures should also remain in task state so that saying “I’ll remember that” is not mistaken for a completed operation.
Forgetting and deletion: stopping use and removing content completely
Forgetting can use lower priority, expiration, archiving, or consolidation into summaries to improve later use. Infrequent use does not always mean low value: a rarely triggered constraint may be essential to the corresponding task.
Invalidated records can remain as history while being excluded from the selection of current rules. Deletion needs an explicit target: primary records, related indexes, summaries, or other derived content. Removing a memory from current input does not automatically clear saved copies outside the window.
If source conversations will still be processed in the background, deleted content also needs to be kept from being extracted again. Systems using versioned files should distinguish current files from historical records. The application and deletion request determine the actual retention scope; a single unsuccessful search cannot establish that content has disappeared completely.
The relationship with RAG: shared retrieval, ongoing maintenance needs
Long-term memory can use retrieval augmentation: select relevant facts or experiences and give them to the model with the current question. Documents, conversations, and experience can all be retrieval targets, so RAG and memory cannot be distinguished solely by whether material comes from documents or users.
| Perspective | What retrieval augmentation addresses | What memory management must also explain |
|---|---|---|
| Input sources | Finding material relevant to the current task | Selecting content from interactions for continued retention |
| Information organization | Parsing, indexing, filtering, and retrieval | Ownership, sources, applicability periods, and memory types |
| Ongoing maintenance | Changes to indexes after material updates | Preference corrections, experience organization, conflicts, invalidation, and forgetting |
| Model input | Assembling sufficient context | What stays always loaded, what is retrieved on demand, and whether it affects current behavior |
One system can implement these responsibilities together. Retrieval methods address finding information again; memory policies decide what to retain and maintain. Within a framework, their connection still needs to be traced.
6 Implementation Observations: From State Frameworks to Memory Systems
LangGraph / LangMem: separating state persistence, content management, and input selection
LangGraph uses State to connect execution steps in a graph. State can contain messages, summaries, progress, and business fields. During execution, a checkpointer saves snapshots of thread state for continuation, recovery, and review. Persistence
A Store provides data access organized by namespace and key for use across threads. With the corresponding index configured, it can also perform semantic search. Values in the Stores documentation are structured records, and namespaces can express organizational scopes such as users and projects. Trusted identity and access authorization remain application responsibilities, and the example InMemoryStore exists only in memory.
Reading and assembly logic still stand between these objects and model input. A node can inspect State and Store before giving the model only a few of their fields. runtime.context separately passes dependencies such as identity, configuration, or connections for the current run into nodes. Its responsibility differs from both State and model request context, and its entire contents are not automatically injected into the model. Graph API
LangMem adds memory content processing. An application can call managers to extract and organize information or expose management and search as tools for the model to call during an interaction. Memory tools guide
Specifically, create_memory_manager receives a conversation and existing memories and returns extraction or update results, leaving persistence to the application. create_memory_store_manager additionally integrates Store lookup and writing. The semantic memory guide shows these combinations in more detail. Content management, data access, and request assembly can be chosen independently, and their effects also need to be checked separately.
Letta / MemGPT: combining always-loaded memory with information outside the window
MemGPT proposes hierarchical context and proactive reading of external information to support longer interactions. The paper’s main context includes instructions, working content, and a message queue, while functions and management components connect storage outside the window.

Source: Charles Packer et al., Figure 3 of the original MemGPT paper, CC BY 4.0. The original SVG was converted to PNG with the figure’s contents unchanged. Capacity numbers illustrate the original design, and the components show the organization used in that paper.
The diagram shows reading, queueing, and context management between saving external content and letting the model actually read it. This is a concrete implementation for understanding a layered design, not a requirement that every agent use the same components.
Letta’s MemFS documentation further demonstrates file-based organization: some memory enters the always-loaded prompt, while the rest remains in hierarchical files and is loaded on demand through directories and read operations. Git supports file revision and synchronization, but factual conflicts still need judgment by the application and agent. Placing files in MemFS does not establish vector retrieval by default.
Memory configuration and dreaming explain the role of background review and organization, such as merging duplicates or adjusting file organization. The always-loaded scope, file structure, and organization policies jointly affect what the next call actually sees. Understanding this implementation requires tracing file persistence, selective loading, and background maintenance separately rather than summarizing it only as “longer context.”
Mem0 OSS: connecting extraction and retrieval interfaces to an application
Mem0 provides interfaces for operations such as adding and searching memories. An application can pass interactions to add, call search in later tasks, and decide which results to include in model requests. How It Works
Its open-source write implementation includes candidate fact extraction, embedding, deduplication, and storage. The default extraction path uses incremental addition. Correcting existing content and deleting records are handled by the corresponding explicit operations. Update, Delete
This boundary affects the maintenance design in Chapter 5: calling an addition interface does not mean that all conflict revisions have happened automatically. Hash-based deduplication also covers only content actually compared; it does not establish semantic deduplication across the entire memory collection. Platform services and OSS capabilities need to be assessed through their respective interfaces. Graph relationships or temporal features from one approach should not be treated as default configurations for every deployment.
These three groups of implementations show combinations of state and Store, always-loaded and on-demand files, and extraction/retrieval services and applications. When reading the code, follow the same relationships: what is saved, how it is updated, and which part the next request actually uses.
7 Evaluation and Optimization: Checking Retention, Retrieval, and Use Separately
Evaluation material: linking every question back to original evidence
Evaluation needs to cover continuity within the current task and later use across sessions. Examples can retain original interactions, constraints to follow, information to write, subsequent queries, and the expected sources and scopes of the information used.
For compression, examine whether work can continue afterward and whether key constraints and unfinished items remain. For long-term memory, examine whether information becomes the correct records, can be retrieved later, and preserves distinctions between old and new states. Final answers alone are often insufficient to identify which intermediate step failed.
Layered observation: from writing to actual behavior
| Observation point | Question to answer | Metrics or checks |
|---|---|---|
| Context and compression | Are goals, constraints, state, and necessary sources preserved? | Compare key information, subsequent task performance, and input tokens |
| Memory writing | Are content, subject, scope, and attribution correct? | Check extraction accuracy, omissions, duplication, and sources |
| Memory retrieval | Does the correct material enter the candidate set at an appropriate rank? | Recall@k, NDCG@k, and similar metrics, with relevant evidence and granularity fixed |
| Context injection | Does retrieved material enter the model’s actual input? | Inspect final selection, deduplication, and budget trimming |
| Updates and application | Do answers or actions use information currently applicable? | Corrections, cross-session use, temporal reasoning, and task correctness |
| Missing and removed information | Can the system abstain when it does not know, and is removed content still used? | Unanswerable examples, post-deletion use, and source tracing |
| Resource investment | Does additional maintenance provide actual benefits? | Record latency and costs separately for retrieval, writing, background organization, and generation |
LongMemEval organizes evaluation around information extraction, multi-session reasoning, knowledge updates, temporal reasoning, and abstention, examining answers and memory recall separately. These dimensions help investigate cases where a system saves extensive content but still uses it incorrectly later.
The published LoCoMo paper provides long conversations and tasks including evidence localization and question answering. Token-level F1 measures overlap between answers and references but is sensitive to different wording, while model judges have their own judgment errors. Evaluation methods each have boundaries. A single overall score cannot establish reliability across every long-term task.
Diagnosing failures: locating where information changed
If an original requirement disappears after compression, inspect summary and state retention first. If the source is intact but an extracted memory changes the subject, examine writing. If the memory is correct but is not retrieved, focus on scope, queries, and indexes. If retrieval finds it but the budget cuts it from context, inspect input assembly.
If actual input contains the right material but the answer still uses old information, examine conflict handling and generation. If the same fact reappears after deletion, trace its source: was an index left uncleared, was a historical file still read, or was the content extracted again in the background? Separating these stages gives each change a clear target.
When comparing approaches, keep the questions, generation model, input budget, and evaluation rules fixed where possible, and observe a change with a clear purpose. Context management and memory management ultimately return to actual tasks: whether information is retained accurately, can be found when needed, and helps the agent make an appropriate next decision.

