International interdisciplinary project
Student research collective · CS × Quant × Policy × Design

What do machines forget?

We study AI memory decay: how language models lose track of what they were told, and what that means for the people who rely on them. Four tracks, one question, looked at from every side.

FounderJane Magdiel Ebenezar
Co-FounderSai Saramati Kautilya
Tap the logo or any symbol ↗
The research question

Memory is a design choice nobody voted on.

Long conversations make models drift. Early instructions fade, details blur, and the model keeps talking with full confidence. We want to measure that fade, explain it, and decide what should be done about it.

Human forgetting has been charted since Ebbinghaus in 1885: retention falls fast, then levels off, and it falls slower for things that matter. Machine forgetting follows its own rules, set by context windows, attention and training choices.

No single discipline can answer this on its own. An engineer can find where a model drops information. A statistician can model the curve. A policy researcher can ask who is harmed when it happens. A designer can show it so that anyone understands. So we do all four.

Key findings

What the paper found.

1,540

questions tested across five memory designs and three model sizes.

13–15

F1 points: how far Summary RAG fell behind Full Dialogue and Observations RAG at 7B and 14B.

25.6 → 43.9

Raw Truncation’s F1 from the smallest model to the largest, the biggest gain from scaling up.

Mean token-level F1 on the LoCoMo benchmark. Full tables and limitations are in the paper.

Four tracks

One topic, four ways in.

Every member joins a track. Each track studies AI memory decay with its own tools, then the findings are brought together.

Decay lab · interactive

Watch a passage fade.

A toy model of memory decay. Each word gets a strength; important words (in blue) hold on longer. Move the slider to add conversation turns and see what survives.

0
30
Retention curve
–average retention
–key facts kept
–half-life, turns

Illustration only. Real decay depends on model, context length and position; measuring it is our work.

Want to see it happen in a real conversation? The simulator plays out a full chat and shows each memory weaken, go wrong or vanish.

How we work

From a hunch to a finding.

Probe

Track 1 designs recall tests: plant facts early in a long exchange, then ask for them back later.

Measure

Track 2 fits decay curves to the results and checks which differences are real and which are noise.

Question

Track 3 asks where these failures matter, who they affect, and what rules or safeguards could help.

Show

Track 4 turns it all into something people can see, share and argue with.

Game

Can you out-remember a machine?

Meet Remi, our mascot. Remi’s leaves are memories, and they fall when something is forgotten. Help Remi keep the facts that matter in Remi’s Memory Jar, a quick game about AI memory decay.

Remi, a tiny AI who tries hard to remember

Remi’s Memory Jar

Meera’s messages float by and fade away. Tap the important ones to save them in Remi’s jar, then answer Remi’s questions. Easy enough for kids, fun for everyone!

Founders and project leaders

The people steering it.

Founder

Jane Magdiel Ebenezar

Founder, The Polymath Initiative

Co-Founder

Sai Saramati Kautilya

Co-Founder, The Polymath Initiative

PS

Pranjal Sharma

Project leader

A

Anuroop V J

Project leader

Across borders

Our team spans five countries.

One research question, studied by students in India, the United Arab Emirates, Kyrgyzstan, Germany and Pakistan.

India

  • Sai Saramati Kautilya
  • Jane Magdiel Ebenezar
  • Pranjal Sharma
  • Anuroop V J
  • Sakshum Dugar
  • Srishti Jain
  • Shruthi Kalyani S
  • Grihith Santhosh

United Arab Emirates

  • Alisha Faria Ashker Riyas

Kyrgyzstan

  • Anarkhan Taalaibekova

Germany

  • Lea Marie Bleibaum

Pakistan

  • Raja Muhammad Noor ul Arfeen
Reviews

What people are saying.

Visited the site, played the game or read the paper? Leave a review and tell us what you think.

Write a reviewLoading reviews…
Your rating

💬
No reviews yet. Yours could be the first!

Be the first to leave a review!

Recruitment closed

Our team is complete for now.

We're not taking new members at the moment. Follow our work on Instagram, or write to us if you have a question about the research.

Research paper

Read the paper.

Our full research paper, bound as a booklet. Every page is shown whole: click a page, swipe, or use the arrow keys to turn it.

Research paper team
  • Pranjal Sharma
  • Anuroop V J
  • Sai Saramati Kautilya
  • Jane Magdiel Ebenezar
  • Alisha Faria Ashker Riyas
  • Lea Marie Bleibaum
  • Shruthi Kalyani S
  • Srishti Jain
  • Grihith Santhosh
Summary

The paper in plain language.

The question

Chat assistants need to remember what people told them earlier, and smaller models struggle most. Memory systems are usually tested as complete packages, so it is hard to tell how much the memory design itself matters.

What we did

We held everything else fixed and changed two things: five memory designs and three Qwen2.5 model sizes (1.5B, 7B and 14B). Each setup answered 1,540 questions from LoCoMo, a benchmark of 10 very long conversations, scored with token-level F1.

What we found

The best design depends on model size. At 1.5B, Full Dialogue and Observations RAG beat Raw Truncation by about 7 and 9 points. At 7B and 14B, Summary RAG fell 13 to 15 points behind them, which suggests summaries can lose useful facts. Paged Memory scored lowest at every size.

Why it matters

A bigger model alone does not guarantee better recall. The information that reaches the model matters too.

Limitations

The study covers one model family at three sizes and only ten conversations, so statistical power is limited. Answers were capped at 32 tokens, and the 7B Raw Truncation run finished only 9 of 10 conversations, so it was left out of the statistical comparisons.

Model sizeRaw TruncationFull Dialogue RAGSummary RAGObservations RAGPaged Memory
1.5B25.6332.8623.5734.8811.82
7B40.8741.6626.6040.9017.22
14B43.9140.6225.4038.7523.45

Mean F1 by model size and memory design (Table 4). Higher is better, and the top score at each size is in bold. Close scores may not differ significantly. The 7B Raw Truncation figure covers 9 of 10 conversations.

Cover
Read the full textThe complete paper as plain text, for easier reading on any screen and with screen readers.

A Controlled Evaluation Of Memory Architecture For Long-Horizons Factual Retention

Pranjal Sharma, Anuroop VJ, Jane Magdiel Ebenezar, Sai Saramati Kautilya, Grihith Santhosh, Shruti Kalyani S , Srishti Jain, Alisha Faria Ashker Riyas, Lea Marie Bleibaum

I. Abstract

Long-context language modelling has increasingly shifted from simply enlarging the model context window towards systems that selectively compress, preserve, and retrieve information from long inputs. We address this gap with a controlled comparison of five memory architectures across Qwen2.5 models of 1.5B, 7B, and 14B parameters on the LoCoMo benchmark. Using token-level F1 across 1,540 questions, we find that memory architecture significantly influences factual retention, but its effect varies with model capacity. Summary RAG consistently underperformed Full Dialogue and Observations RAG at larger model sizes, while Paged Memory produced the lowest F1 at every evaluated size. Raw Truncation showed the largest increase with model scaling, from 25.64 F1 at 1.5B to 43.91 at 14B. These findings indicate that merely scaling up model capacity cannot necessarily ensure superior factual recall and highlight the significance of information availability.

II. Introduction

Conversational agents are increasingly expected to work with users over weeks or months. To do this, they must remember what a user said in earlier sessions and use it later. Language models handle this poorly. When relevant details are spread across a long, multi-session history, accuracy drops as the history grows [4][7]. This matters most for small models, which are cheaper, can run on a user's own device, and keep personal conversations off third-party servers.

Researchers have proposed many external memory mechanisms to address this problem, including recursive summarization, retrieval over past dialogue, extracted facts, and hierarchical or graph-structured memory [1][3][14]. However, these methods are usually introduced as complete systems. They differ in backbone model, prompting, and retrieval settings, so a difference in results cannot be traced to the memory design alone [1][2]. To our knowledge, few studies compare memory architectures under matched conditions, and fewer still do so for small and mid-sized open-weight models [3][10][11][12][1]. We address this gap with a controlled comparison. We fix the model family (Qwen2.5), inference settings, benchmark, and scoring procedure. We vary only two factors: the memory architecture and the model size. We evaluate 5 architectures (truncation, summary-based memory, observation-based memory, full-dialogue-based memory and paged memory on three model sizes (1.5B, 7B, and 14B parameters). We use LoCoMo [4], a benchmark of very long conversations, each averaging roughly 300 turns across up to 35 sessions. We define factual retention as the model's F1 score on LoCoMo questions whose answers depend on facts stated in earlier sessions.
We ask two research questions:

RQ1. How does memory architecture affect factual retention on long, multi-session conversations when model family and inference setup are held constant?

H1. Full Dialogue RAG will achieve higher F1 than Raw Truncation, Summary RAG, and Observations RAG because it preserves the original conversational text while retaining retrieval-based access.RQ2. How does model size affect factual retention, and does it change which architecture performs best?

H2: Increasing model size will improve factual retention, although the magnitude of this effect will vary across memory architectures
This paper makes one contribution: a controlled, matched-condition comparison of 5 memory architectures across three Qwen2.5 model sizes on LoCoMo, which isolates the effect of memory design from the effect of the surrounding system.

III. Related Work

Long-context language modelling has increasingly shifted from simply enlarging the model context window towards systems that selectively compress, preserve, and retrieve information from long inputs. Retrieval Augmented Generation (RAG) provides a general mechanism for giving LLMs relevant information that may or may not be in their training data, while recent long-context approaches have explored how the information presented to a model can be organized and represented more efficiently.

ReadAgent [1] approaches this issue by mimicking human reading behaviour: long documents are split into pages, and each page is compressed into a location-linked “gist memory”. The original text is retrieved when detailed information is required for answering the given question. Evaluations on QuALITY, NarrativeQA, and QMSum showed that ReadAgent outperformed the evaluated retrieval, original-context, and gist-memory baselines. 

LongMemEval [7] introduced a benchmark specifically designed to measure the long-term memory capabilities of chat assistants across information extraction, multi-session reasoning, temporal reasoning, and knowledge updates. The benchmark contains 500 questions embedded within scalable user-assistant conversation histories and frames memory systems as a pipeline consisting of indexing, retrieval, and reading.

[3] Evaluated Mem0 against results from the LoCoMo benchmark and found that it achieves the highest scores in single- and multi-hop questions. Mem0ᵍ and Mem0 also scored higher than typical RAG since selective memory extraction avoids retrieval of irrelevant conversational chunks. Mem0 outperformed OpenAI’s built-in memory architecture. Base Mem0 stored memories in approximately 7,000 tokens per conversation, whereas Zep consumed more than 600,000 tokens owing to the caching of the full abstractive summaries at every node in graph memory.  

[4] Proposes the LoCoMo benchmark that aims to test whether a model can retain facts, maintain coherence, and reason over long conversations. They developed LoCoMo using a hybrid machine-human pipeline where they use different LLMs instructed to abide by a specified persona to generate synthetic conversations, which are then modified, corrected, and reviewed by humans. The LoCoMo benchmark primarily uses QA tasks based on each of the total 10 conversations to evaluate the agent. LoCoMo introduced a benchmark specifically designed to evaluate long-term conversational memory in LLMs. It consists of 10 long, multi-session conversations accompanied by QA pairs that test different forms of memory and reasoning. The authors primarily evaluate models of sizes greater than 14B (one 7B model is present); they are models from different model families and vendors. The authors don't study the size scaling effect or the effect of memory architecture on QA performance.

IV. Methods

We evaluated models from the Qwen2.5 family because it provides multiple model sizes while maintaining a consistent underlying model architecture and training framework. This allowed us to treat the model size as the primary independent variable. This enabled us to alter the model size while minimizing variation attributable to differences between model families, allowing model size to be evaluated as an independent variable.
The other variables of the study are outlined in Table 1.

Experimental variables
Independent variablesControlled variables
Model size
Model architecture
Temperature
Top_p
Inference provider
Table 1

We measure the token-level F1 for each tested condition. The evaluation follows the same evaluation procedure used by the LoCoMo implementation. We implement a 3 x 5 experimental design that varies the Qwen2.5 model size across 1.5B, 7B, and 14B parameters and memory architecture across 5 conditions. All experiments were conducted using identical prompts, decoding configuration, and evaluation metric across all conditions. Condition 1, referred to as Raw Truncation, involves placing conversation turns directly into the model context window without performing any preprocessing. Whenever excessive amounts of historical context are available, truncation occurs either by removing the oldest or newest conversation turns according to the condition definition. No retrieval process is performed, and Raw Truncation acts as the baseline condition. The second condition, Full Dialogue RAG, involves storing conversation turns separately and individually as retrievable memory entries. To retrieve memories to form prompts for answering the query, the system searches for closest vectors based on calculated cosine similarities of conversation turn vectors with the query using pre-generated DRAGON embeddings. In the third condition, Summary RAG, pre-computed summaries of conversations are utilized instead of conversation turns. Again, close memories are selected based on vector similarity to a particular query in the same way as in Condition 2. In the fourth condition, Observation RAG, observations, snippets of factual information obtained through LoCoMo or the memory pipeline, are used. Condition 5, Paged Memory, was inspired by ReadAgent [1]. First, conversations are divided into pages by a deterministic strategy; subsequently, the gist of each page is computed. Thus, essentially, Condition 5 offers all gist vectors simultaneously to prompt generation, where only specific pages will be retrieved based on the requirements of the model at that moment.

Models for experimentation
ModelsParametersRole
Qwen2.5-1.5B-Instruct1.5BSmall
Qwen2.5-7B-Instruct7BMedium
Qwen2.5-14B-Instruct14BUpper bound
Table 2
Experimental conditions
Memory storageMemory architecture
Raw conversation turnsTruncated
Raw conversation turnsTruncated, Vector Retrieval
SummariesVector Retrieval
ObservationsVector Retrieval
Summary, Raw conversation turnsPaged Memory
Table 3

The primary task in the benchmark is Long-Context Question Answering, in which the model under evaluation is fed the complete long-term conversation history (or uses a memory/RAG pipeline) and must answer 1,540 diagnostic questions (excluding category 5). We omitted the adversarial QA category (category 5) as this is not part of the evaluation. Each test sample is formatted by appending the evaluation prompt after the conversation context. To mitigate the ambiguity of free-form open-ended text evaluation, reference answers are constructed as verbatim phrases extracted directly from the dialogue text wherever possible. The evaluation prompt explicitly instructs the evaluated model to replicate exact wording from the dialogue text instead of generating verbose explanations. 

We modified the original implementation to support Qwen models via the NVIDIA API. We created a new module for the NVIDIA API that matched the implementation pattern of functions in the original OpenAI API code.

We implemented the paged memory based on the ReadAgent paper. [1] The adapted implementation is aimed at being a minimal, testable version of the original ReadAgent implementation. We keep the ReadAgent source repository and commit hash recorded in the experiment metadata when using its exact prompt templates. The original ReadAgent implementation uses documents as the unit of retrieval, which was adapted to use the multi-session conversations from the LoCoMo dataset. Deterministic page boundaries were used for controlled comparison.

V. Results

The study examines how memory architecture and model scale impact each other on the LoCoMo benchmark based on token-level F1 scores across four classes of answerable questions: Multi-hop, Temporal, Open-Domain, and Single-hop. Both conditions were assessed based on the full set of 1,540 questions spread across 10 conversations. The 7B Raw Truncation condition had only 9 out of 10 conversations that got completed and hence could not be used for matched statistical comparisons. [4]

F1 mean
Model sizeRaw TruncationFull Dialogue RAGSummary RAGObservations RAGPaged Memory
1.5B25.6332.8623.5734.8811.82
7B40.8741.6626.6040.9017.22
14B43.9140.6225.4038.7523.45
Table 4
Bar chart: overall mean F1 by model size and memory architecture, using the values in Table 4.
Overall mean F1 by model and memory
Box plot: question-level F1 by question category and memory architecture.
QA-level F1 by category and memory

Conclusions drawn from these results indicate that scaling up model size does not necessarily lead to consistent gains across all memory architectures. For instance, Raw Truncation F1 increases from 25.64 to 43.91 when the model size scales up from 1.5B to 14B. Conversely, Summary RAG F1 values remain consistently lower irrespective of model size (23.57-26.60). Lastly, Paged Memory F1 scores scale up from 11.82 to 23.45 but remain the poorest-performing memory architecture. 

The pairwise comparison of memory architectures was carried out using conversation-level F1 scores as the independent variable. In each case, we used comparable architecture sizes while comparing two different architectures. Pairwise comparison was done for F1 scores calculated on the same set of conversations in both cases to maintain the paired structure of our benchmark. The difference values are expressed in terms of percentage points (pp), denoting the average paired difference in F1 scores. Statistical inference about whether this average difference value significantly differs from zero was made using two-sided Wilcoxon signed-rank tests. Effect sizes were estimated through rank biserial correlations. Because multiple comparisons were done under each model size, adjustments to the p-values were made based on the Holm procedure. We considered p < α=0.05 statistically significant.

VI. Discussions

In general, the pairwise comparisons show that the impact of memory architecture changed depending on the model size. At 1.5B Full Dialogue and Observations RAG both had better F1 scores than Raw Truncation, with gains of 7.22 and 9.25 percentage points respectively (both Holm-adjusted p =0.0117). At 7B Full Dialogue and Observations RAG were not very different from each other (Holm-adjusted p =0.4316). Both did much better than Summary RAG by 15.06 and 14.30 percentage points respectively (both Holm-adjusted p =0.0059). At 14B Summary RAG was worse than Dialogue and Observations RAG by 15.23 and 13.36 percentage points respectively (both Holm-adjusted p =0.0117). Full Dialogue and Observations RAG never had a difference at any model size tested. Comparisons, with 7B Raw Truncation were not included because that setup was not complete. Finally, at 7B, there was no statistical difference observed between Full Dialogue and Observations RAG. The experiments reveal a significant impact of memory architecture on factual recall, and the impact differs across different capacities. The summary RAG underperforms full dialogue and observations RAG with 7B and 14B capacities in F1 score. Specifically, summary RAG lags behind full dialogue and observations RAG in F1 by 15.06 points and 14.30 points in the case of 7B, and 15.23 points and 13.36 points for 14B capacity. Such findings imply the potential loss of useful facts during summarization, and merely enhancing the model capacity does not necessarily improve factual recall performance. Full Dialogue and Observations RAG exhibited no significant difference detected F1s across all three sizes, with F1 of 32.86, 41.66, and 40.62 achieved by Full Dialogue and 34.88, 40.90, and 38.75 achieved by Observations RAG at 1.5B, 7B, and 14B respectively. The confidence intervals around these differences do not include margins exceeding approximately 5. Neither of the two improves monotonically as the parameter size increases since they increase 6 to 9 points when going from 1.5B to 7B but drop slightly at 14B. Paged Memory produced the lowest overall F1 at every evaluated model size: 11.82 at 1.5B, 17.22 at 7B, and 23.45 at 14B. Although its performance increased with model size, it remained substantially below the other evaluated memory architectures. This suggests that the additional memory-access mechanism implemented in the Paged Memory condition did not translate into effective factual retention under the tested configuration. The experimental results partially validated our hypotheses. In relation to H1, while Retrieval Memory (Full Dialogue RAG and Observations RAG) only surpassed Raw Truncation at 1.5B with improvements of 7.22 F1 and 9.24 F1 respectively, Raw Truncation was comparable or even superior to the retrieval-based conditions at 7B and 14B. Hence, H1 cannot be verified except for the smallest model size. Nevertheless, Summary RAG consistently lagged behind all retrieval-based architectures at all size points. For H2, F1 did not strictly grow alongside increasing model sizes across all architectures. For instance, Full Dialogue, Observations RAG, and Summary RAG reached maximum F1 values at 7B whereas Raw Truncation and Paged Memory had increasing scores as we moved up along model capacities. Also supported was the assumption that retrievals offered more benefits compared to other architectures at smaller scales but decreased as models became more capable. Thus, we can conclude that model capacity played an important role in how memory architecture impacted recall.

In particular, it can be observed that summary-based memory does not equal richer representation in terms of recall or performance in this particular LoCoMo benchmark. On the contrary, there seems to be empirical evidence suggesting that a less complex version of the dialogue, Observations RAG, performs comparably well to the most sophisticated Full Dialogue approach. These results must be viewed in light of the experimental conditions in this study. 

VII. Limitations

This paper employs Qwen2.5 series across three model sizes on the LoCoMo benchmark. LoCoMo includes merely ten conversations; thus, there are no sufficient statistics. All responses are limited up to 32 tokens, cutting off around 2% to 33% of responses contingent upon the experimental condition. Exclusion of such items failed to alter the directionality of any pairwise differences between non-Paged conditions. Deterministic decoding parameters are applied across all configurations. In contrast to ReadAgent, the Paged Memory condition implements ReadAgent with modifications in lookup prompt format, active page size, and maximum number of pages recalled simultaneously.

VIII. References

  1. Lee, K.-H., Chen, X.-Y., Furuta, H., Canny, J. F., & Fischer, I. (2024). A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts. ArXiv, abs/2402.09727. https://doi.org/10.48550/arxiv.2402.09727

  2. Wei, T., Sachdeva, N., Coleman, B., He, Z., Bei, Y., Ning, X., Ai, M.-T., Li, Y.-Z., He, J., Chi, E., Wang, C., Chen, S., Pereira, F., Kang, W.-C., & Cheng, D. (2025). Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory. ArXiv, abs/2511.20857. https://doi.org/10.48550/arxiv.2511.20857

  3. Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025). Mem0: Building Production-Ready           AI Agents with Scalable Long-Term Memory. 2993-3000. https://doi.org/10.48550/arxiv.2504.19413

  4. Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024). Evaluating Very Long-Term Conversational Memory of LLM Agents. ArXiv, abs/2402.17753. https://doi.org/10.48550/arxiv.2402.17753

  5. Wu, Y., Lin, T., Zhou, Y.-L., Zhang, F., Guo, Q., Zhou, X., Wang, S., Liu, X., Y., & Fang, Y. (2026). Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework. ArXiv, abs/2604.01707. https://doi.org/10.48550/arxiv.2604.01707

  6. Patel, D., & Patel, S. (2025). ENGRAM: Effective, Lightweight Memory Orchestration for Conversational Agents. ArXiv, abs/2511.12960. https://doi.org/10.48550/arxiv.2511.12960

  7. Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., & Yu, D. (2024). LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ArXiv, abs/2410.10813. https://doi.org/10.48550/arxiv.2410.10813 

  8. Hao, X., Zhang, Z., Lin, Z., Sun, Y., Guo, Z., Zhang, X.-C., Liang, Y., Xiong, F., & Li, Z. (2026). MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations. ArXiv, abs/2607.12893. https://doi.org/10.48550/arxiv.2607.12893 

  9. Wang, S., Yu, E., Love, O., Zhang, T., Wong, T., Scargall, S., & Fan, C. (2026). MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agents. ArXiv, abs/2604.04853. https://doi.org/10.48550/arxiv.2604.04853

  10. Zhang, J.-Q., Zhang, C., Chen, S., Huang, Z., Zheng, P.-C., Wang, Z., Guo, P., Mo, F., Bae, S.-H., Zou, J., Wei, J., & Yang, Y. (2026). Lightweight LLM Agent Memory with Small Language Models. 12914-12929. https://doi.org/10.48550/arxiv.2604.07798

  11. Ahn, K. (2025). HEMA : A Hippocampus-Inspired Extended Memory Architecture for Long-Context AI Conversations. ArXiv, abs/2504.16754. https://doi.org/10.48550/arxiv.2504.16754 

  12. Zhang, Z., Dai, Q., Bo, X., C., Li, R., Chen, X., Zhu, J., & Dong, Z. (2024). A Survey on the Memory Mechanism of Large Language Model-based Agents. ACM Transactions on Information Systems, 43, 1 - 47. https://doi.org/10.1145/3748302 

  13. Kang, J., Ji, M., Zhao, Z., & Bai, T. (2025). Memory OS of AI Agent. 25961-25970. https://doi.org/10.48550/arxiv.2506.06326

  14. Hu, S., Wei, Y., Ran, J., Yao, Z., Han, X., Wang, H., Chen, R.-H., & Zou, L. (2026). Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory. ArXiv, abs/2601.01280. https://doi.org/10.48550/arxiv.2601.01280

See it for yourself

Watch memory decay happen.

The simulator plays out a full conversation and shows each memory weaken, go wrong or vanish.

Credits

The people behind the Polymath Initiative.

Everyone who has worked on this initiative, across research, analysis, policy, design and this website.

Founders
  • Jane Magdiel EbenezarFounder, The Polymath Initiative
  • Sai Saramati KautilyaCo-Founder, The Polymath Initiative
Overall project leaders
  • Jane Magdiel EbenezarProject leader
  • Sai Saramati KautilyaProject leader
  • Pranjal SharmaProject leader
  • Anuroop V JProject leader
The four tracks
TRACK_01 / TECH

Tech & AI

  • Anuroop V J
  • Pranjal Sharma
  • Sai Saramati Kautilya
  • Jane Magdiel Ebenezar
  • Shruthi Kalyani S
  • Grihith Santhosh
  • Raja Muhammad Noor ul Arfeen
TRACK_02 / QUANT

Quant & Analytics

  • Jane Magdiel Ebenezar
  • Srishti Jain
  • Lea Marie Bleibaum
  • Sai Saramati Kautilya
  • Anuroop V J
TRACK_03 / POLICY

Investigative Research & Policy

  • Alisha Faria Ashker Riyas
  • Anuroop V J
  • Pranjal Sharma
  • Sai Saramati Kautilya
  • Jane Magdiel Ebenezar
  • Lea Marie Bleibaum
  • Raja Muhammad Noor ul Arfeen
  • Sakshum Dugar
TRACK_04 / DESIGN

Media & Design

  • Sai Saramati Kautilya
  • Lea Marie Bleibaum
  • Anarkhan Taalaibekova
  • Jane Magdiel Ebenezar
  • Anuroop V J
  • Pranjal Sharma
Research paper team

Research paper

  • Pranjal Sharma
  • Anuroop V J
  • Sai Saramati Kautilya
  • Jane Magdiel Ebenezar
  • Alisha Faria Ashker Riyas
  • Lea Marie Bleibaum
  • Shruthi Kalyani S
  • Srishti Jain
  • Grihith Santhosh
Website
Designed and built by
Sai Saramati Kautilya
In charge of development, design and publishing of the website and domain
With help fromRaja Muhammad Noor ul Arfeen
Special thanks

To everyone who read our work, tried the simulator and followed us on Instagram.

Certificates

Certificates of Participation

Awarded to every member of The Polymath Initiative for their part in our interdisciplinary research on AI memory decay. Presented on 1st October 2026 and signed by our founders.

View only. These certificates are protected and can't be downloaded, copied or printed.

Tap any certificate to view it larger. Certificates hide automatically when you leave this window.

Game · Remi’s Memory Jar

Remi’s Memory Jar

Meera is chatting with Remi. Her messages float by and fade away, just like old messages fade from an AI’s memory. Save the important ones in Remi’s jar, then help Remi answer the questions!

Hi! I’m Remi. Will you help me remember?

How to play

  1. WatchMeera’s messages float across the screen. They slowly fade away.
  2. Tap to saveTap the important messages to drop them into Remi’s jar. The jar only has a few spaces!
  3. Skip chit-chatMessages like “lol” or “hi” are not important. Let them float away.
  4. AnswerRemi asks questions about Meera. Use the jar to pick the right answer and win stars!
Level1/ 3
Stars0⭐
Leaves🌿🌿🌿
Meera

Ready to help Remi remember?

Why this matters: AI chatbots also “forget” messages from early in a long chat. Our research studies how much they forget and how to help them remember.

Simulator · AI memory decay

Watch a model forget.

A user tells a small AI model six facts, then keeps chatting. Every few turns she checks what it still remembers. Watch each memory stay strong, fade, get confidently wrong, or vanish.

Model size
Turn 0/ 18
ConversationUser Model Outside context window
6

facts will be planted in turn 1. Then come 17 more turns of ordinary chat and memory checks.

Press Start simulation to begin, or Auto-play to watch it run.

What the model remembers
–overall recall
Strong0
Fading0
Distorted0
Lost0
Retention over the conversationAll facts Allergy (safety-critical)
Context window0 / 0 tokens

Each block is one turn. Anything left of the frame has scrolled out of the model's context window.

What's happening

The model starts with an empty memory. In turn 1 the user will tell it six facts about herself.

Lesson 1

Distance hurts

The further back a fact is in the conversation, the weaker the model's hold on it.

Lesson 2

Recall refreshes

When a fact is repeated, it is back in recent context, so it gets stronger again.

Lesson 3

Wrong beats blank

A fading model often guesses instead of admitting it forgot, and says the guess with full confidence.

Lesson 4

Windows end

Once the turn that held a fact leaves the context window, the model can no longer see it at all.

This is a teaching simulation with a simplified decay model, not a live AI. Measuring how real models forget is the work of our four tracks.