Introducing SIM
A new model architecture built around memory that carries forward.
Enter the researchPersistent state
Remember. Build on it. Keep going.
SIM carries useful information forward in a persistent internal state. It uses no KV cache, keeps memory per token constant, and does not need to revisit every earlier token.
A different architecture
Memory that moves with the model.
Transformer context becomes more expensive as a sequence grows. SIM updates an internal state as new information arrives, keeping the cost of the next token independent of the sequence behind it.
Early evidence
Remembering across longer distances.
We compared matched 150M parameter models using the same tokenizer, data, and 8B token training schedule. SIM had lower cross entropy at every tested distance between mentions of a name.
Less catastrophic forgetting.
When the model learned a new domain, SIM's protection system reduced catastrophic forgetting by 72%. This is an early result toward learning new things while retaining earlier knowledge.
Built to scale differently
Testing how far memory can go.
SIM's state does not grow with the sequence. That gives us a different way to explore larger models, longer memory, and continued learning without an expanding cache.
Our 254 million parameter program has a 100 billion token training plan. We are testing whether this approach can help models learn for longer, remember across greater distances, and keep adapting.
A new foundation
A different foundation for what comes next.
We are working toward AI that carries state, learns continuously, and remembers more of what it has learned.