LLM Providers

源码版本v2026.7.20

Responsibility

Turn "a provider name (nvidia, kimi, openai, openrouter...)" into "a concrete hostname, message preprocessing, extra_body, model list". Hermes does not use an inheritance hierarchy for provider abstraction; instead it uses a ProviderProfile dataclass + a set of hooks. Profiles are registered as plugins — "declare to integrate". Nous Portal, OpenRouter, OpenAI, and custom endpoints are supported.

Design Motivation

Why a ProviderProfile dataclass + hooks instead of class OpenAIProvider(BaseProvider)? hermes integrates 30+ providers, and most differences are just base_url, auth header, and a few special fields. An inheritance hierarchy forces every subclass to override a pile of methods, and 90% of subclasses are just empty stubs. Dataclass + hooks means "declare 7 fields + override 1 hook" finishes integration; complex logic (like where Kimi puts the reasoning field) goes into a hook override.

Profiles live in plugins/model-providers/<name>/ rather than providers/<name>.py: the plugin directory carries a plugin.yaml manifest and submodules, and is overridable by the same-shaped tree under user $HERMES_HOME/plugins/ (last-writer-wins), so you can patch a built-in provider without forking the repo. The single-file providers/*.py is a back-compat legacy.

Key Files

• OMIT_TEMPERATURE:21 — sentinel object meaning "do not send temperature" (Kimi lets the server manage it):
• @dataclass ProviderProfile:38-88 — provider metadata (name, aliases, base_url, auth, temperature strategy...):

OMIT_TEMPERATURE = object()

@dataclass
class ProviderProfile:
    name: str
    api_mode: str = "chat_completions"
    aliases: tuple = ()
    env_vars: tuple = ()
    base_url: str = ""
    auth_type: str = "api_key"
    supports_vision: bool = False
    supports_vision_tool_messages: bool = True
    fallback_models: tuple = ()
    fixed_temperature: Any = None

fixed_temperature is tri-state: None uses the caller default, OMIT_TEMPERATURE does not send the field, and a concrete value overrides. supports_vision_tool_messages defaults to True; Xiaomi MiMo accepts multimodal user messages but rejects list-type tool content, so it must be set to False — otherwise you get a 400 "text is not set".

• get_hostname / prepare_messages:98-118 — hooks: which host the request goes to, how messages are preprocessed
• build_extra_body:119-175 — hook: provider-specific request body fields:

def build_extra_body(self, *, session_id: str | None = None, **context: Any) -> dict[str, Any]:
    """Merged into the API kwargs extra_body. Default: empty dict."""
    return {}

def build_api_kwargs_extras(self, *, reasoning_config=None, **context) -> tuple[dict, dict]:
    """Returns (extra_body_additions, top_level_kwargs).
    OpenRouter: reasoning in extra_body. Kimi: reasoning_effort is top-level."""
    return {}, {}

build_extra_body defaults to an empty dict; subclasses override as needed. build_api_kwargs_extras splits fields into extra_body vs top-level api_kwargs: OpenRouter's reasoning goes into extra_body, Kimi's reasoning_effort is top-level.

• fetch_models:175-232 — hook: pull the list of available models for this provider. _profile_user_agent() returns hermes-cli/<version> instead of Python-urllib/<ver>, because some providers sit behind a WAF that 403s the default UA.
• module docstring:9-28 — describes the lazy-discovery mechanism (only scans plugins on first call)
• register_provider:53-65 — register a profile:

_REGISTRY: dict[str, ProviderProfile] = {}
_ALIASES: dict[str, str] = {}

def register_provider(profile: ProviderProfile) -> None:
    """Later registrations with the same name replace earlier ones —
    user plugins can override bundled profiles without editing repo code."""
    _REGISTRY[profile.name] = profile
    for alias in profile.aliases:
        _ALIASES[alias] = profile.name

def get_provider_profile(name: str) -> ProviderProfile | None:
    if not _discovered:
        _discover_providers()
    canonical = _ALIASES.get(name, name)
    return _REGISTRY.get(canonical)

Lazy discovery: the first get_provider_profile triggers _discover_providers(), which scans bundled + user plugins + legacy single-file locations. The alias table maps synonyms like kimi / moonshot to the same profile.

• get_provider_profile:65-76 — lookup by name or alias
• list_providers:76-91 — list all profiles
• plugins/model-providers/ directory — actual profile definitions (one subdirectory per provider)

Data Flow

1. init_agent:276 takes the user-configured provider name.
2. It calls get_provider_profile:65 (triggering lazy discovery on first call: scan plugins/model-providers/<name>/, import it, and register_provider).
3. Once it has the ProviderProfile:
   • get_hostname (providers/base.py:98) fixes the request host
   • prepare_messages (providers/base.py:111) preprocesses the api_messages produced by build_turn_context:268
   • build_extra_body (providers/base.py:119) adds provider-specific fields
   • temperature follows the profile strategy (None / OMIT_TEMPERATURE / a concrete value)
4. The main loop calls the provider API with the assembled request (retry sub-loop:1227).
5. Merging of a custom provider's extra_body happens in _merge_custom_provider_extra_body:257.

Boundaries and Failure

• Profile missing: if config names an unknown provider, get_provider_profile returns None, the main loop falls back to a generic chat_completions provider, and the picker shows an empty list because it cannot get fallback_models.
• Vision unsupported: when supports_vision=False, tool_result images are not pushed as-is into the tool message; when supports_vision_tool_messages=False, using list-type content would 400 — the image must be downgraded or converted to base64.
• fetch_models failure: some providers' /models endpoint sits behind a WAF (the default Python-urllib UA gets 403). fetch_models uses the hermes-cli/<version> UA to bypass; on failure it falls back to fallback_models — only agentic models should be listed here, non-tool-calling ones would mislead the picker.
• User plugin overrides bundled: register_provider is last-writer-wins, so a user plugin with the same name overrides the bundled one. A legacy providers/<name>.py is imported last and can also override — footgun; new code should use the plugin directory.

Summary

The provider abstraction is "profile + hooks", not "inheritance". Adding a provider only requires creating a subdirectory in plugins/model-providers/, declaring a ProviderProfile, and calling register_provider — no need to touch base. Lazy discovery keeps startup fast and loads on demand.