Context Compression & MoA

源码版本v2026.7.20

Responsibility

Two things: (1) when history messages approach the provider's context limit, summarize/trim old messages to free up budget (context_compressor); (2) optionally use a Mixture-of-Agents path to aggregate outputs from multiple models (moa_loop). Both run inside the main loop — one keeps long conversations from overflowing the context window, the other aggregates multiple models for better answers.

Key Files

• context_compressor module — compression main entry (~3700 lines)
• budget estimate:190-242 — _estimate_msg_budget_tokens / _serialized_length_for_budget
• message sanitization:136-155 — _fresh_compaction_message_copy / _strip_persistence_markers
• tool call & path mention extraction:337-360 — _extract_tool_call_name_and_args / _collect_path_mentions
• content trimming:472-515 — _append_text_to_content / _strip_image_parts_from_parts
• moa_loop — Mixture-of-Agents aggregation
• IterationBudget — compression refund and budget coupling

Data Flow

1. Each turn the main loop evaluates whether compression is needed (see _should_run_preflight_estimate:213).
2. When the threshold is hit, context_compressor is called:
   • estimate each message's budget cost (agent/context_compressor.py:419)
   • make sanitized copies of old messages (agent/context_compressor.py:136), stripping persistence markers (agent/context_compressor.py:155)
   • preserve tool-call names/arguments and path mentions so summarization doesn't drop key operations (agent/context_compressor.py:337)
   • trim image parts and overlong content (agent/context_compressor.py:472)
3. The "new space" freed by compression is partially refunded via iteration_budget.refund() (agent/iteration_budget.py), letting the main loop run a few more turns.
4. If MoA is enabled, moa_loop calls multiple providers in parallel this turn and aggregates them as an enhancement or arbitration of the main answer.

Budget estimation is the foundation of the compression decision. Here's the core of agent/context_compressor.py:419-437:

def _estimate_msg_budget_tokens(msg: dict) -> int:
    """Counts the full ``tool_call`` envelope, not just ``function.arguments``
    — counting only the arguments undercounted parallel-tool turns by 2-15x
    (#28053). Also counts provider replay fields (``codex_reasoning_items``)."""
    content_len = _content_length_for_budget(msg.get("content") or "")
    tokens = content_len // _CHARS_PER_TOKEN + 10  # +10 for role/key overhead
    for tc in msg.get("tool_calls") or []:
        if isinstance(tc, dict):
            tokens += len(str(tc)) // _CHARS_PER_TOKEN
    for key in _REPLAY_BUDGET_KEYS:
        tokens += _serialized_length_for_budget(msg.get(key)) // _CHARS_PER_TOKEN
    return tokens

Two traps we hit are documented in the comments: (1) counting only the function.arguments string undercounts by 2-15x in parallel-tool-call turns; (2) skipping replay fields like codex_reasoning_items leaves "looks small but is actually large" assistant messages wrongly protected — compression finishes still near the ceiling and then keeps re-compressing (#55572).

Sanitization and persistence-marker stripping live at agent/context_compressor.py:136-160:

def _fresh_compaction_message_copy(msg: Dict[str, Any]) -> Dict[str, Any]:
    """Shallow .copy() propagates ``_db_persisted``; new child session then
    skips every row (#57491). Strip at every copy site."""
    fresh = msg.copy()
    fresh.pop(_DB_PERSISTED_MARKER, None)
    return fresh


def _strip_persistence_markers(messages: List[Dict[str, Any]]) -> None:
    """Terminal sweep — invariant holds regardless of intermediate copy sites."""
    for msg in messages:
        if isinstance(msg, dict):
            msg.pop(_DB_PERSISTED_MARKER, None)

Two lines of defense: _fresh_compaction_message_copy strips at each copy site close to the action, and _strip_persistence_markers does a full terminal sweep at the end of compress(). The first is intent-clear; the second is structural backstop. Image trimming (see _strip_image_parts_from_parts at agent/context_compressor.py:497-515) follows similar logic — replace image/image_url/input_image parts with text placeholders, and return None when there are no images so the caller skips the swap, avoiding meaningless new lists on every message.

Design Motivation

Why estimation instead of exact token counts? Provider tokenizers aren't public (especially the hidden fields on reasoning models), and tiktoken is both slow and inaccurate. The module uses a char-based estimate with a fixed +10-character overhead formula — it's consistently on the conservative side, which is enough for the "should we compress" and "how do we protect the tail" decisions. It doesn't need to be exact, just measured with the same ruler as the preflight estimate (see _should_run_preflight_estimate in turn_context), otherwise you get a deadlock where "preflight says no need to compress" but "compressor says it can't compress."

Why preserve tool-call names and path mentions? When an LLM summarizes old messages it tends to swallow key operations like "called write_file(/tmp/foo.py)". The next turn the model reads "we discussed a file" but doesn't know which path was written, so it creates duplicates or reads the wrong location. _extract_tool_call_name_and_args / _collect_path_mentions paste the hard facts next to the summary — cost is a few extra tokens, payoff is that subsequent tool calls don't go amnesic.

Refund goes to iteration_budget rather than directly adding budget, so max_iterations doesn't become a dead letter — every compression turn adding a few turns would let long conversations run forever. Refund only returns the one or two tool-call iterations that compression itself saved; "the ceiling stays the ceiling."

Boundaries and Failure

• Compression stalls: _compression_made_progress requires a token reduction >5% to count as progress. A turn that only saves 2% is judged "no progress" to prevent spinning across turns; the main loop then switches to an auto-reset and opens a new session.
• Persistence marker residue: If a compression product carries _db_persisted, the new child session's state.db skips it and the compacted transcript is lost from the DB entirely (#57491). A terminal sweep at the end of compress() forcibly scans everything as a backstop.
• Summary mistaken for a new instruction: Weak models treat "## Active Task" inside the summary as a fresh user request to answer (#11475, #14521). _SUMMARY_END_MARKER adds an explicit separator; in alternation edge cases, _MERGED_PRIOR_CONTEXT_HEADER / _MERGED_SUMMARY_DELIMITER double-wrap to avoid ambiguity.
• Historical prefix revived: _HISTORICAL_SUMMARY_PREFIXES keeps old summary prefixes. When resuming into a new lineage, the old prefix is stripped along with everything else, preventing stale directives embedded in the body from continuing to hijack replies.

Summary

The compressor is what keeps long conversations going; the core tradeoff is "what to drop, what to keep" — it prioritizes tool calls and path mentions, because losing them makes subsequent turns "amnesic". MoA is an orthogonal enhancement that can be toggled. Neither changes the main-loop structure; they only cut in at the right moment.