Cross-Session Memory System for AI Agents, Fully Local
AI products · Creator, sole engineer · active
Giving an AI agent memory is easy to do badly. Dumping every past observation into context hoards noise, retrieval by embedding alone misses exact identifiers and error strings, and memory written from tool output is an untrusted input channel that persists.
Code: https://github.com/vasu-devs/Forge
The detail
This consolidates rather than accumulates. Every candidate memory is decided against the existing set - add a genuinely new fact, update one that supersedes an old one, delete one that has been invalidated - which is what keeps the store small enough to be useful at session start.
Retrieval is a hand-built hybrid of semantic similarity, keyword ranking and entity matching, deliberately without an approximate-nearest-neighbour index: at personal-memory scale that index adds a dependency and a failure mode for no measurable speed gain. That trade-off is documented rather than hidden.
Memory carries separate timestamps for when something was true and when it was recorded, so stale facts cannot silently outrank current ones, and importance weighting keeps durable decisions from decaying just because they are old.
The store is treated as untrusted input against prompt injection. No telemetry, no network calls, local storage only.
What I did
- Designed the consolidation model that adds, updates or deletes rather than accumulating
- Built the hybrid retriever combining semantic, lexical and entity matching by hand
- Deliberately omitted an approximate-nearest-neighbour index and documented why at this scale
- Implemented the dual-timestamp model separating when a fact was true from when it was recorded
- Added importance weighting so durable decisions resist decay
- Treated the memory store as untrusted input against prompt injection
Stack
Node.js, AI Agent Development, Machine Learning, Vector Database, BM25, ONNX, Hybrid Search, Open Source
Case study: https://www.siddhvasudev.com/projects/cross-session-memory-system-for-ai