Hermes Agent ships with a deliberately tiny memory: about 2,200 characters of curated notes, 1,375 characters of user profile, and a full-text search index over past sessions. That is enough for a personal assistant — and not nearly enough for everything else. Here is how the built-in layer works, how six external memory systems compare, and which one I actually run.
What Hermes Agent remembers by default
Hermes keeps memory in two layers, and the split is intentional.
| Layer | Carrier | Size | Purpose |
|---|---|---|---|
| Curated persistent memory (core) | MEMORY.md |
~2,200 characters | Environment facts, project conventions, lessons from past tasks |
USER.md |
~1,375 characters | User identity, preferences, communication habits | |
| Full-text session search (auxiliary) | SQLite + FTS5 | Unlimited | Retrieve details from historical conversations on demand |
The strengths are real:
- Zero dependencies and full local control
- Token overhead is fixed and very low
- Writes are reviewable and auditable
- An external memory backend can be plugged in through the
memory.providersetting, so you are not locked in
So are the limits:
- Capacity is tiny — it cannot absorb large volumes of knowledge automatically
- No knowledge graph and no semantic association
- Unsuited to heavy reasoning or synthesis across many memories
Why add an external memory system
The built-in layer is light and controllable, but it has a hard ceiling. An external system buys you:
| Dimension | What an external system adds |
|---|---|
| Capacity | Effectively unlimited; handles large knowledge bases |
| Retrieval | Semantic search, keyword search, graph traversal and temporal reasoning running in parallel |
| Automation | Automatic extraction, deduplication and synthesis — no manual curation |
| Structure | Knowledge graphs, hypergraphs, virtual filesystems |
| Collaboration | Cloud options support multi-device sync and team sharing |
| Explainability | Some systems expose memory decay, contradiction flags and supersession relationships |
Six external memory systems compared
| System | Core architecture | Main strengths | Main limits |
|---|---|---|---|
| Mem0 | Vector store + knowledge graph + server-side LLM extraction | Highly automated; roughly 91.6% on the LoCoMo benchmark; claims over 90% token cost savings | Depends on cloud LLM calls, so privacy is a concern; knowledge is decomposed into triples, which can lose conversational context |
| OpenViking | viking:// virtual filesystem with L0/L1/L2 tiered loading |
One place for memory, RAG and skills; structured browsing that feels like a filesystem; around 80–83% memory accuracy in official evaluations | AGPLv3 licensing may not suit closed-source commercial use; self-hosted and architecturally heavy |
| RetainDB | Cloud API + local write-behind queue | Cross-session semantic search, user profiling and context synthesis; the local queue keeps writes reliable | Core processing and storage stay in the cloud; retrieval depends on network round-trips, so there is latency |
| Hindsight | Knowledge graph + four parallel retrieval strategies (semantic, keyword, graph traversal, temporal) | The most complete retrieval model, with synthesis across memories; the broadest integration ecosystem (40+ official integrations) | Complex architecture; self-hosting has a higher setup bar |
| HyperRecall | Hypergraph + spreading activation retrieval | Stores whole experiences as units, preserving the full shape of an event; supports memory decay, contradiction flags and supersession; $0 ingest cost with a portable format | Relatively niche; ecosystem maturity is unknown |
| Holographic | HRR (holographic reduced representations) algebraic memory + SQLite | Fully local with no external dependencies; trust scoring for self-correction; very low retrieval latency | Simpler feature set; no cross-device sync or other complex scenarios |
Deployment and cost
| System | API key | LLM calls | Deployment and cost | Best for |
|---|---|---|---|---|
| OpenViking | Optional server-side; embedding/VLM models required | Required (memory extraction and vectorization) | Self-hosted at zero software cost if you bring your own models; open source (AGPL-3.0) | Teams that want full data control and already run local models |
| Mem0 | Required (Mem0 Cloud) | Required (server-side extraction) | Managed cloud on subscription (from about $19/month); free tier available | Fast integration with no infrastructure to manage, if you accept cloud costs |
| RetainDB | Required for Cloud; not for Local | Local can use local models; Cloud is server-side | Cloud from $20/month; Local self-hosted at zero cost | Local for individual developers; Cloud for teams and hosted infrastructure |
| Hindsight | Required (Cloud, or an API key for a local LLM) | Required (knowledge graph construction) | Cloud is paid with a free trial; the local version is zero-cost with your own LLM API | Knowledge graph and cross-memory synthesis, if you are willing to configure a local LLM API |
| Holographic | None | None | Fully local, zero cost (SQLite, no external dependencies) | Maximum lightness, privacy and zero cost with modest feature needs |
| HyperRecall | None | None | $0 ingest cost, portable format | Memory completeness and brain-like traits such as forgetting and contradiction handling |
Can it run entirely on your own machine?
| System | Local deployment | Requirements | Cost |
|---|---|---|---|
| Holographic | ✅ Fully local | None — built on SQLite | Zero |
| OpenViking | ✅ Possible | Bring your own embedding/VLM models | Software free, model cost separate |
| RetainDB Local | ✅ Possible | Configure a local model | Software free, model cost separate |
| Hindsight (local) | ✅ Possible | Bring your own LLM API key | Software free, API cost separate |
| Mem0 | ❌ Cloud only | Mem0 Cloud required | Subscription (free tier available) |
| RetainDB Cloud | ❌ Cloud only | Cloud API required | $20/month |
| Hindsight Cloud | ❌ Cloud only | Cloud service required | Paid, with a free trial |
What I chose, and why
I have used both Mem0 and Hindsight. Both offer free tiers, and Mem0’s free tier is roughly enough for light use. But they are cloud services, and I wanted the whole system running on my own hardware. I looked at OpenViking, but even the local deployment still calls an LLM.
So I went with a local RetainDB deployment. It ships with a tiny model that runs on CPU at zero cost, which is exactly the trade-off I wanted: local, cheap, and good enough.
Deploying it is simple. The main piece is the @retaindb/local npm package, built as a runtime with no external database dependency and aimed squarely at local use. The most direct way to start it is npx, which downloads and runs the latest version:
npx -y @retaindb/localThat starts an HTTP service on local port 3111 and stores data under ~/.retaindb/. You can hand the whole deployment to your Hermes agent and let it do the work.
The bottom line
When you pick an external memory system for Hermes Agent, weigh four things together: privacy requirements, budget, deployment capability and feature needs. I settled on locally deployed RetainDB. If you want something even lighter and still zero-cost, Holographic is worth a look. And whichever you choose, memory.provider lets you swap backends later without rewriting anything else.