{
  "tool": {
    "name": "Hermes Agent",
    "repo": "https://github.com/NousResearch/hermes-agent",
    "version_at_verification": "v0.18.2",
    "version_date": "2026-07-07",
    "last_verified": "2026-07-09",
    "dossier_version": "2026-07-09",
    "decay_class": "volatile"
  },
  "scorecard": {
    "claims_sampled": 14,
    "as_labeled": 5,
    "partial": 5,
    "not_as_labeled": 2,
    "not_implemented": 2,
    "open_deltas": 6,
    "verified_closed": 5,
    "closed_partial": 3,
    "closure_trend": "high release velocity; vendor closes surface bugs (Telegram pool drain, Slack channel IDs) while structural deltas (skill auto-invocation, single event loop, 13.9K overhead floor) remain open through v0.18.2"
  },
  "verdict": "A fast-moving, well-connected personal agent chassis whose headline self-improvement claim is not functional in any released version.",
  "claims": [
    {
      "id": "1",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "probe": "probes/hermes-agent/claim-1/",
      "claimed": "Self-improving agent — learns skills from experience and applies them autonomously",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "The loop is broken at both ends: the LLM ignores `skill_view()` auto-invocation even when a relevant skill exists (#4589); `skills_guard` treats its \"ask\" verdict as a hard block, silently rejecting agent-created skills (#13686); and background skill creation can deadlock the entire gateway (#10138). Skills work — manually, invoked by name, by a human. UPDATE 2026-06-28 (v0.17.0): #13686 (skills_guard) closed not-planned/implemented-on-main and #10138 (gateway deadlock) closed via PR #39737; the core loop bug #4589 (skill_view auto-invocation) remains open — the self-improvement loop still has no completion path. UPDATE 2026-07-09 (v0.18.2): #4589 re-confirmed OPEN (last activity 2026-06-30); v0.18.x shipped `/learn` and `/journey` as *manual* skill distillation/timeline surfaces, not auto-invocation — falsification criterion for autonomous skill application still not met.",
      "status": "not_implemented",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/4589",
        "https://github.com/NousResearch/hermes-agent/issues/13686",
        "https://github.com/NousResearch/hermes-agent/issues/10138"
      ]
    },
    {
      "id": "2",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "claimed": "Continuous self-improvement powered by GEPA (ICLR 2026)",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "The GEPA *algorithm* is real and independently validated (arXiv:2507.19457, ICLR 2026 Oral). The *product* (hermes-agent-self-evolution) remains Phase 1 of 5 (DSPy skill optimization); Phases 2–5 are still \"Planned\" with no timeline per README as of 2026-07-09. Companion repo last push 2026-06-17 (config fix; 8 commits total historically) — no longer multi-month stale, but still no Hermes-specific before/after benchmark of self-improvement. v0.18.0's `/learn`/`/journey` are manual distillation/timeline UX, not GEPA Phases 2–5. Research credential is still being spent on a product claim it does not cover.",
      "status": "not_implemented",
      "pattern_labels": [
        "credential_transfer"
      ],
      "evidence": [
        "https://arxiv.org/abs/2507.19457"
      ]
    },
    {
      "id": "3",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "claimed": "40–90% token reduction vs naive implementations",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Hermes carries a 13.9K-token fixed overhead per API call (8.7K tool definitions + 5.2K system prompt). In a measured 3-session evening: 2.8M of 3.9M tokens (73%) were overhead (#4379). v0.15.0's genuine 47% per-turn call reduction addresses variable cost only — the release notes themselves confirm the floor is unchanged in v0.15.1. UPDATE 2026-07-09 (v0.18.2): #4379 re-confirmed OPEN (updated 2026-06-30; clustered with related agent-overhead issues #20717/#36656/#49673/#54256). No re-measurement on v0.18.x in this pass — floor claim stands pending fresh session measure.",
      "status": "not_as_labeled",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/4379"
      ]
    },
    {
      "id": "4",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "probe": "probes/hermes-agent/claim-4/",
      "claimed": "Unified gateway across 22 messaging platforms",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "The platform count has exceeded the marketed \"22\" (Photon iMessage, WhatsApp Business Cloud API, SimpleX, Raft added in v0.17.0); exact recount not done this pass. Six adapters are natively maintained (rest community, no SLA), and all platforms still share a single event loop — one network flake can cascade (#21026, OPEN). Confirmed history: Slack cross-channel send fixed (PR #16198 / #15927); Discord non-response (#16744) closed not-planned/implemented-on-main. UPDATE 2026-07-09 (v0.18.2): #21026 still OPEN (stale since 2026-05-08); Telegram pool-timeout silent-drop path addressed by PR #54121 (#53524 closed completed) but the structural single-loop root cause remains. Status remains partial.",
      "status": "partial",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/21026",
        "https://github.com/NousResearch/hermes-agent/issues/15927",
        "https://github.com/NousResearch/hermes-agent/issues/16744",
        "https://github.com/NousResearch/hermes-agent/releases/tag/v2026.6.19",
        "https://github.com/NousResearch/hermes-agent/issues/53524",
        "https://github.com/NousResearch/hermes-agent/pull/54121",
        "https://github.com/NousResearch/hermes-agent/releases/tag/v2026.7.7.2"
      ]
    },
    {
      "id": "5",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "probe": "probes/hermes-agent/claim-5/",
      "claimed": "iMessage support",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Dual path: BlueBubbles (always-on Mac bridge) and Photon managed relay (v0.17.0 \"Reach\" release, no Mac relay required). Photon was broken at launch: sidecar pinned to spectrum-ts@3.1.0 dialing decommissioned gRPC host fusor.spectrum.photon.codes (NXDOMAIN) — #50918. UPDATE 2026-07-09 (v0.18.2): #50918 re-confirmed OPEN; Photon still not a reliable production iMessage path. Status remains partial.",
      "status": "partial",
      "evidence": [
        "https://hermes-agent.nousresearch.com/",
        "https://github.com/NousResearch/hermes-agent/issues/50918"
      ]
    },
    {
      "id": "6",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "probe": "probes/hermes-agent/claim-6/",
      "claimed": "MCP as a core feature — OAuth 2.1 PKCE, OSV malware scanning, dynamic discovery, SSE transport",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Largely verified in code: SSE transport confirmed, OSV scanning confirmed as a runtime check before spawning servers (depth-verified 2026-06-08). But SSE *discovery* is not supported (#17244), and MCP startup could hang up to 120s on a synchronous `urllib` call on the event loop thread (#29184). UPDATE 2026-06-28: the 120s MCP startup hang (#29184) closed via PR #52147. UPDATE 2026-07-09 (v0.18.2): #17244 re-confirmed OPEN — SSE discovery gap still partializes the \"MCP as core feature\" claim.",
      "status": "partial",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/17244",
        "https://github.com/NousResearch/hermes-agent/issues/29184",
        "https://github.com/NousResearch/hermes-agent/pull/52147"
      ]
    },
    {
      "id": "7",
      "claimed": "Three-layer persistent memory (prompt / session / external)",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "The layers exist as described. Under-documented semantics: MEMORY.md is a frozen snapshot loaded at session start — in-session writes are invisible to the running agent until the next session, and short sessions may retain nothing. Of 8 documented external providers, only 2 (Hindsight, Holographic) have meaningful public evidence of real-world use; the benchmark-leading memory systems (Mastra OM, Letta, OMEGA, Zep) have no Hermes integration path at all.",
      "status": "partial",
      "evidence": [
        "https://hermes-agent.nousresearch.com/"
      ]
    },
    {
      "id": "8",
      "claimed": "Runs on a $5/mo VPS (orchestration separated from inference)",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Confirmed as the documented reference architecture: agent on cheap VPS, inference offloaded to OpenRouter/Nous Portal/Ollama host.",
      "status": "as_labeled",
      "evidence": [
        "https://hermes-agent.nousresearch.com/"
      ]
    },
    {
      "id": "9",
      "claimed": "Pluggable inference providers — LiteLLM, vLLM, Ollama, Llamafile",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Confirmed; vLLM ships as an official bundled skill; provider system formalized in v0.13.0.",
      "status": "as_labeled",
      "evidence": [
        "https://hermes-agent.nousresearch.com/"
      ]
    },
    {
      "id": "10",
      "claimed": "Home Assistant built-in",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Confirmed first-party: four tools auto-activate on two env vars, WebSocket real-time state subscriptions verified in docs.",
      "status": "as_labeled",
      "evidence": [
        "https://hermes-agent.nousresearch.com/"
      ]
    },
    {
      "id": "11",
      "claimed": "Profiles: isolated multi-instance agents on one machine",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Confirmed as documented, including the honest caveats in their own docs (no filesystem sandboxing; one exclusive bot token per profile).",
      "status": "as_labeled",
      "evidence": [
        "https://hermes-agent.nousresearch.com/"
      ]
    },
    {
      "id": "12",
      "claimed": "Secure defaults (v0.13.0): secret redaction on, WhatsApp rejects unknown senders",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Release-verified; both moved from opt-in to default. Genuine security-positive movement.",
      "status": "as_labeled",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/releases"
      ]
    },
    {
      "id": "13",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "probe": "probes/hermes-agent/claim-13/",
      "claimed": "Configurable context compression (`threshold`, `protect_last`)",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Compression config works as documented for static models, but `ContextCompressor.threshold_percent` is not updated on runtime model switch (`/model`, fallback activation, or primary restore) — `context_length` updates while `threshold_percent` retains the previous model's value, so compression triggers at the wrong token threshold (#18617, OPEN). Earlier dossier text that described integer `threshold: 75` being silently read as \"~75 tokens\" is NOT the bug #18617 tracks; that integer-coercion mechanism remains unverified. UPDATE 2026-07-09 (v0.18.2): #18617 re-confirmed OPEN (title and body match the per-model threshold_percent sync failure). Status remains partial.",
      "status": "partial",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/18617"
      ]
    },
    {
      "id": "14",
      "verified_at": "2026-07-09",
      "verified_by": "session",
      "claimed": "Cost/token analytics (ecosystem: hermes-workspace)",
      "claimed_source_url": "https://hermes-agent.nousresearch.com/",
      "observed": "Retroactive SQLite token counting only — no budgets, no alerts. `delegate_task` subagent costs are reported as never flushed to the database (one documented measurement: $15.57 actual vs $3.37 recorded, 79% undercount) — but no primary GitHub issue URL has been located as of 2026-07-09 (search for delegate_task billing/state.db returned unrelated open issues only). UPDATE 2026-06-28: v0.17.0 adds delegate_task(background=true) but subagent billing persistence is still unfixed per prior measurement. UPDATE 2026-07-09: still no primary-source issue for the 79% undercount; claim retained as not_as_labeled on the \"analytics exist and are trustworthy\" marketing implication, with the undercount sub-claim flagged as secondary-only.",
      "status": "not_as_labeled",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent"
      ]
    }
  ],
  "deltas": [
    {
      "id": "HM-01",
      "title": "Skill auto-invocation non-functional — self-improvement loop has no completion path",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/4589",
        "Re-verified open 2026-06-28 against v0.17.0",
        "Re-verified OPEN 2026-07-09 against v0.18.2 (state=open, last activity 2026-06-30)"
      ],
      "opened": "≤2026-05",
      "status": "open",
      "falsification_criterion": "`skill_view()` auto-triggers reliably across N conversations, re-verified"
    },
    {
      "id": "HM-02",
      "title": "`skills_guard` \"ask\" verdict hard-blocks agent-created skills",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/13686"
      ],
      "opened": "2026-05",
      "status": "closed_partial",
      "falsification_criterion": "\"ask\" surfaces to user instead of blocking; re-verified",
      "closed_by": "Closed not-planned + implemented-on-main 2026-06 (#13686); behavior change not TD-re-verified"
    },
    {
      "id": "HM-03",
      "title": "Gateway deadlock on background skill creation, no recovery path",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/10138",
        "https://github.com/NousResearch/hermes-agent/pull/39737"
      ],
      "opened": "≤2026-05",
      "status": "closed_verified",
      "falsification_criterion": "Nested `register_mcp_servers` invocation completes or fails safely",
      "closed_by": "PR #39737 merged 2026-06 (#10138)",
      "reverified_date": "2026-06-28"
    },
    {
      "id": "HM-04",
      "title": "Single event loop is a systemic root cause — originally 7 P1/P2 blocking bugs; 5 of 7 now closed (#29184/#20525 via PRs, #28300/#20269 sweeper, #25351 not-planned), #13281 and root-cause #21026 remain open",
      "evidence": [
        "All 7 confirmed open 2026-06-08",
        "Re-verified open 2026-06-28 against v0.17.0",
        "Re-verified 2026-07-09 against v0.18.2: #28300/#20525/#29184/#20269/#25351 CLOSED (June 2026); #13281 and root-cause #21026 still OPEN — falsification (all 7 closed OR architecture change) not met",
        "https://github.com/NousResearch/hermes-agent/issues/21026",
        "https://github.com/NousResearch/hermes-agent/issues/13281"
      ],
      "opened": "Apr–May 2026",
      "status": "open",
      "falsification_criterion": "All 7 issues closed and re-verified, or architecture change ships"
    },
    {
      "id": "HM-05",
      "title": "13.9K-token fixed overhead floor contradicts marketed savings",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/4379",
        "Re-verified open 2026-06-28 against v0.17.0",
        "Re-verified OPEN 2026-07-09 against v0.18.2 (state=open, updated 2026-06-30)"
      ],
      "opened": "2026-04",
      "status": "open",
      "falsification_criterion": "Overhead floor materially reduced, re-measured on a real session"
    },
    {
      "id": "HM-06",
      "title": "Telegram FD/pool exhaustion wedges gateway while reporting connected — vendor pool-drain fix shipped; TD soak not re-run",
      "evidence": [
        "~280 sockets/reconnect cycle (prior measurement)",
        "https://github.com/NousResearch/hermes-agent/issues/53524",
        "https://github.com/NousResearch/hermes-agent/pull/54121",
        "UPDATE 2026-07-09: #53524 CLOSED completed via merged PR #54121 (fix(telegram): drain general send pool on pool timeout before retry, 2026-06-28) — vendor fix landed; 48h soak falsification not TD-executed"
      ],
      "opened": "2026-05",
      "status": "closed_partial",
      "falsification_criterion": "Pool cleanup ships; 48h soak test passes",
      "closed_by": "PR #54121 merged 2026-06-28 (#53524 closed completed); pool-timeout drain ships, but 48h soak criterion not TD-re-verified → closed_partial",
      "reverified_date": "2026-07-09"
    },
    {
      "id": "HM-07",
      "title": "Slack cross-channel send silently fails (`isdigit()` on alphanumeric channel IDs)",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/15927",
        "https://github.com/NousResearch/hermes-agent/pull/16198"
      ],
      "opened": "2026-05",
      "status": "closed_verified",
      "falsification_criterion": "Channel ID validation fixed; cross-channel send verified",
      "closed_by": "PR #16198 merged (#15927) — channel ID validation fixed",
      "reverified_date": "2026-06-28"
    },
    {
      "id": "HM-08",
      "title": "Compression threshold_percent not updated on model switch/fallback (wrong compression trigger after `/model`)",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/18617",
        "Re-verified open 2026-06-28 against v0.17.0",
        "Re-verified OPEN 2026-07-09 against v0.18.2; corrected mechanism: issue is per-model threshold_percent sync on model switch, NOT integer 75→75-token coercion (that earlier characterization is unverified)"
      ],
      "opened": "2026-05",
      "status": "open",
      "falsification_criterion": "`threshold_percent` recalculates (or is re-read from per-model config) on every model switch, fallback activation, and primary restore; re-verified"
    },
    {
      "id": "HM-09",
      "title": "`delegate_task` costs never persisted — 79% billing undercount",
      "evidence": [
        "unverified source, single measurement (no primary issue URL as of 2026-07-09)",
        "Re-verified open 2026-06-28 against v0.17.0",
        "Re-check 2026-07-09 against v0.18.2: gh search for delegate_task cost/billing/state.db found no matching issue; related open delegate_task bugs exist (#59386 schema, #52484 recursion, etc.) but not the 79% undercount. Delta retained open; primary-source gap remains."
      ],
      "opened": "2026-05",
      "status": "open",
      "falsification_criterion": "Subagent costs flush to state.db; recorded ≈ actual"
    },
    {
      "id": "HM-10",
      "title": "Env override re-enables explicitly disabled platforms",
      "evidence": [
        "https://github.com/NousResearch/hermes-agent/issues/55113",
        "https://github.com/NousResearch/hermes-agent/issues/47804",
        "Prior 2026-06-28 note citing #53697 was a mis-attribution (#53697 is Telegram streaming default bypass, not platform enable override)",
        "Re-verified OPEN 2026-07-09 against v0.18.2: #55113 (TELEGRAM_BOT_TOKEN forces platform on despite platforms.telegram.enabled: false) OPEN; #47804 (Feishu env forces enable) OPEN; #48820 (Windows env override ignores yaml) OPEN"
      ],
      "opened": "2026-05-23",
      "status": "open",
      "falsification_criterion": "`platforms.X.enabled: false` survives presence of the platform's bot-token env var (Telegram/Feishu and peers); re-verified"
    },
    {
      "id": "HM-C1",
      "title": "Docker `--insecure` was default",
      "evidence": [
        "release-reviewed 2026-05-29"
      ],
      "status": "closed_verified",
      "closed_by": "v0.15.1 — now explicit env opt-in",
      "reverified_date": "2026-05-29"
    },
    {
      "id": "HM-C2",
      "title": "WhatsApp accepted all senders by default",
      "evidence": [
        "release-reviewed 2026-05-07"
      ],
      "status": "closed_verified",
      "closed_by": "v0.13.0 — unknown senders rejected by default",
      "reverified_date": "2026-05-07"
    },
    {
      "id": "HM-C3",
      "title": "Secret redaction was opt-in",
      "evidence": [
        "release-reviewed 2026-05-07"
      ],
      "status": "closed_verified",
      "closed_by": "v0.13.0 — on by default",
      "reverified_date": "2026-05-07"
    },
    {
      "id": "HM-C4",
      "title": "Per-turn token cost (variable portion)",
      "description": "Partial — floor (HM-05) explicitly unchanged",
      "evidence": [
        "Partial — floor (HM-05) explicitly unchanged"
      ],
      "status": "closed_partial",
      "closed_by": "v0.15.0 — 47% reduction (399K→213K, 31-turn chat)"
    }
  ],
  "methodology": {
    "sampling_rule_ref": "Label-check methodology (D0086) — material-claims rule: a claim is sampled if it appears in the product's own description of itself and would plausibly influence an adoption decision. Claims sampled from official docs (hermes-agent.nousresearch.com), release notes, and the project README.",
    "verification_depth": "source-reviewed",
    "verification_statement": "Evidence is drawn from the project's own code, issue tracker, release notes, and production reports filed by third parties; select claims depth-verified against source on 2026-06-08; re-verified against the issue tracker and v0.17.0 releases on 2026-06-28; re-verified again on 2026-07-09 against v0.18.2 (tag v2026.7.7.2) via `gh issue view` / `gh api` on every open-delta backing issue and key claim issues (#4589, #21026, #13281, #4379, #18617, #17244, #50918, #53524/#54121, #55113, #47804) plus hermes-agent-self-evolution README/push date. Theory Delta has not executed Hermes in its own environment for this dossier. Star counts (~212K as of 2026-07-09) are recorded in the backing block only and remain excluded from the adoption assessment.",
    "backing_confidence": "secondary-research",
    "executed_by_td": false,
    "strongest_case_against": "Hermes's release cadence is fast enough that any specific issue may close within weeks; v0.15–v0.18 refactors and the PR #54121 Telegram pool drain may have changed runtime behavior not fully reflected in older open threads. The self-improvement bugs may be model-size or configuration-sensitive rather than universal. HM-09 (delegate_task billing undercount) still lacks a primary issue URL. Re-verification cadence for this tool should remain high (decay_class: volatile). Note 2026-06-28: prior delta HM-11 (claimed P0 session-auth exposure) was removed because no primary-source URL was ever attached."
  },
  "provenance": {
    "source_block": "theorydelta-blocks/blocks/hermes-agent.md (depth-verified 2026-06-08; block claims freshness pass 2026-07-04)",
    "source_blocks": [
      "theorydelta-blocks/blocks/hermes-agent.md"
    ],
    "additional_sources": [
      "published/hermes-agent-self-improvement-non-functional.md"
    ],
    "derived_date": "2026-07-09"
  }
}