Learning Loop & Memory

源码版本v2026.7.20

Responsibility

This is the loop that makes Hermes "get better the more you use it". It distills each turn of conversation into: (1) skill evolution — creates/rewrites skills from experience (agent/learning_mutations.py); (2) memory — FTS5 full-text search + LLM summarization of past sessions (memory_manager), paired with Honcho's dialectical user modeling; (3) learning graph — organizes skills and memory into nodes/edges (agent/learning_graph.py); (4) background review — background_review digests history in a thread and produces actionable rewrites. Skill rewrites ultimately flow back into the skill system for reuse in subsequent turns.

Key Files

Skill nodes & learning graph

• @dataclass SkillNode:28-41 — skill node
• skill loading:77-125 — _iter_skill_files / build_skill_nodes (scan skills/ to build nodes)
• edges & stats:156-193 — build_edges / density_stats (inter-skill relations)
• memory-skill edges:193-227 — _memory_cards / _memory_skill_edges (memory↔skill links)
• build_learning_graph:248-328 — aggregates the full learning graph
• _load_usage:84-97 — usage stats (guides which skills are commonly used)

Skill/memory rewrites

• node_detail:86-124 — view a node's detail
• delete_node:124-157 — delete a skill or memory
• edit_node:157-200 — edit a skill or memory's content
• _clear_skill_cache:200 — clear cache after rewrite (so the next turn sees it)
• memory location:30-65 — _memories_dir / _parse_memory_id / _locate_memory

Memory management

• class MemoryManager:354 — memory manager itself
• memory provider tool injection:83-164 — memory_provider_tools_enabled / inject_memory_provider_tools
• build_memory_context_block:337-354 — assembles memory into a context block injected into the prompt
• sanitize_context:164-172 — context redaction (before FTS5 indexing)
• StreamingContextScrubber:172-337 — streaming-output redaction

Background review

• spawn_background_review_thread:956 — start the background-review thread
• _run_review_in_thread:617 — main loop inside the thread
• _digest_history:122-373 — summarize the most recent 24 history entries
• summarize_background_review_actions:373-590 — aggregate review actions
• build_memory_write_metadata:590-617 — metadata for memory writes

What a Skill Node Looks Like

SkillNode is the unified representation of a skill in the learning graph (agent/learning_graph.py:28-41): the dataclass fields are name/category/source (base for built-in vs profile for user/agent-created)/use_count (read from .usage.json)/created_by ('agent' means created by background review)/pinned/related. build_skill_nodes scans the SKILL.md files under skills/ and overlays usage stats. source != "base" plus created_by == "agent" or use_count > 0 is the filter build_learning_graph later uses to decide "this is learned, not built-in" — only the truly learned nodes are put on the table for the user to see.

Output of the Learning Graph

build_learning_graph (agent/learning_graph.py:248-328) assembles nodes, edges, memory cards, and cluster stats into a complete payload for the desktop learning panel to render. It first calls build_skill_nodes(_skill_roots()) to scan two roots — base (repo/skills/, built-in) and profile (~/.hermes/skills/, user/agent-created); then filters skills with source != "base" and (created_by == "agent" or use_count > 0) as "learned", and overlays build_edges (inter-skill relations), _memory_cards (memory cards), and _memory_skill_edges (memory↔skill links). Base is always there; profile is where "learned" actually lives.

Cache Must Be Cleared After Rewrite

If a skill is rewritten by edit_node and the cache isn't cleared, the next turn's prompt_builder will still use the old SKILL.md snapshot, making the rewrite pointless. _clear_skill_cache (agent/learning_mutations.py:200) calls clear_skills_system_prompt_cache(clear_snapshot=True); clear_snapshot=True also clears the tool snapshot, because MCP tool lists may have been refreshed during review. This function is the last step of edit_node / delete_node — rewrite → persist → clear cache → take effect next turn; all four steps are required.

Memory Injection: From MemoryManager to the Prompt

MemoryManager (agent/memory_manager.py:354) coordinates one built-in provider (FTS5 + LLM summary) plus at most one external provider (Honcho/Hindsight/Mem0 and the like). Only one external provider is allowed; a second is rejected — this avoids tool-schema bloat and memory backends fighting; failures in the two providers don't block each other. Retrieved raw memories are wrapped by build_memory_context_block (agent/memory_manager.py:337-354) into a <memory-context> fence + a system-annotated context block, where the system annotation explicitly tells the model "this is memory, not a new instruction"; sanitize_context strips the fence and the old system annotation before indexing — preventing the provider from double-wrapping already-wrapped content. StreamingContextScrubber does the same on the streaming-output side, so the model doesn't echo the memory block verbatim back to the user.

Background Review: Summary and Candidate Actions

spawn_background_review_thread (agent/background_review.py:956) doesn't directly start a thread — it builds a (target, prompt) tuple and hands it to AIAgent._spawn_background_review_thread, so the test layer can still patch threading.Thread. The function signature (review_memory / review_skills — two bools) decides which prompt to pick — _MEMORY_REVIEW_PROMPT / _SKILL_REVIEW_PROMPT / _COMBINED_REVIEW_PROMPT:

def spawn_background_review_thread(agent, messages_snapshot,
                                    review_memory=False, review_skills=False):
    """Build the review thread target and prompt. Returns a ``(target, prompt)`` tuple."""
    if review_memory and review_skills:
        prompt = getattr(agent, "_COMBINED_REVIEW_PROMPT", _COMBINED_REVIEW_PROMPT)
    elif review_memory:
        prompt = getattr(agent, "_MEMORY_REVIEW_PROMPT", _MEMORY_REVIEW_PROMPT)
    else:
        prompt = getattr(agent, "_SKILL_REVIEW_PROMPT", _SKILL_REVIEW_PROMPT)
    def _target() -> None:
        _run_review_in_thread(agent, messages_snapshot, prompt)
    return _target, prompt

Inside the thread, _run_review_in_thread (agent/background_review.py:617) forks a sub-agent that inherits the parent agent's runtime (to avoid re-parsing credentials — OAuth-only providers can't even be rebuilt inside a sub-agent), and installs the _bg_review_auto_deny approval callback so that touching dangerous commands during review always denies rather than falling back to input() (which would deadlock against the parent agent's prompt_toolkit TUI). _digest_history (agent/background_review.py:122) compresses history into "the most recent 24 entries verbatim + a synthetic summary of the rest", used only when review is routed to a different model (a cold cache makes the token savings worthwhile); the main-model path runs a full replay (the cache is still hot).

Data Flow

1. After each turn, the main loop hands a message snapshot to the background-review thread (agent/background_review.py:956).
2. _digest_history (agent/background_review.py:122) summarizes recent history and produces candidate actions (create/rewrite skills, write memory).
3. Skill rewrites go through edit_node:157; deletions go through delete_node:124, then _clear_skill_cache:200 clears the cache.
4. Memory writes go through MemoryManager:354; after redaction (agent/memory_manager.py:164) an FTS5 index is built and an LLM summary is compressed.
5. At the start of the next turn, build_memory_context_block (agent/memory_manager.py:337) weaves relevant memories into build_turn_context:268; skills are injected via agent/skill_bundles.py.
6. The full learning graph is aggregated by build_learning_graph:254 for UI/debug display of skill↔memory links.

Design Motivation

Why split the learning loop into four independent modules? Because their cost structures differ. background_review runs on a sub-agent — it has to fork the runtime and spend LLM calls, so it must be async and must be able to fail and roll back; it has to be decoupled from the main loop. learning_graph is a read-only aggregation, cheap, and can be pulled by the UI at any time — it can't be coupled to the write path. learning_mutations is the write path; after a rewrite the cache must be cleared, or the next turn eats a stale snapshot. memory_manager is retrieval + redaction + injection, grouped together because "fetch memory from an external provider → redact → wrap in fence → inject into prompt" has to happen within one boundary, or a single missed redaction leaks raw memory into the model output.

Boundaries and Failure

• Background review isn't synchronous. _run_review_in_thread runs in a daemon thread; the main loop doesn't wait for it before returning to the user. The skill rewrite from a review that just finished usually only takes effect on the next turn.
• Only one external memory provider is allowed. Honcho and Hindsight can't run at the same time; add_provider rejects a second external provider.
• _digest_history only kicks in when review is routed to a different model. The main-model path runs a full replay (cache still hot); changing this behavior requires looking at the auxiliary.background_review config first.
• The redaction boundary for memory injection is "the fence and the system annotation". sanitize_context only strips the <memory-context> fence and the system annotation; it doesn't rewrite the memory body — if memory text contains sensitive info, that has to be handled before it's written in.

Summary

The learning loop turns "conversation" into "assets": background review distills experience into skills and memory, which are auto-injected on the next turn. The four-piece core — learning_graph (the graph), learning_mutations (rewrites), memory_manager (retrieval + redaction + summarization), background_review (async digestion). This is what lets Hermes "grow with the user" rather than act as a stateless LLM call.