Integrations/Hermes

AgentMetrics for Hermes

Add full observability to any Hermes agent with a single decorator. AgentMetrics tracks every run, tool call, memory access, token usage, and cost automatically.

Quickstart

pip install agentmetrics

Wrap your agent:

import agentmetrics
from hermes import Agent

@agentmetrics.track()
class MyAgent(Agent):
    def run(self, task: str) -> str:
        # Your agent logic here
        ...

What AgentMetrics tracks for Hermes agents

The @agentmetrics.track() decorator instruments your Hermes agent's entire execution lifecycle. Here's exactly what gets captured:

Agent runs. Start timestamp, end timestamp, wall-clock duration, and final status. A failed run includes the exception type and message.
Tool calls. Tool name, inputs, outputs, latency, and whether the call succeeded. Captured for every tool invoked during the run.
Memory reads and writes. When your agent reads from or writes to its memory store, these operations are captured as structured events with the memory key and byte size.
Token usage. Input and output token counts per LLM call, aggregated at the run level. Supports all major model providers.
Cost. Per-run cost calculated from token usage against current model pricing. Visible in the dashboard broken down by run and by day.
Nested calls. If your Hermes agent spawns subagents or calls other tracked functions, the call tree is preserved in the trace.

Full install walkthrough

Step 1. Install the AgentMetrics SDK:

pip install agentmetrics

Step 2. Set your API key. Get it from Settings in the AgentMetrics dashboard:

export AGENTMETRICS_API_KEY=am_live_xxxxxxxxxxxxxxxxxxxx

Step 3. Add the @agentmetrics.track() decorator to your agent class:

import agentmetrics
from hermes import Agent, tool

@agentmetrics.track()
class ResearchAgent(Agent):
    """An agent that researches topics and summarizes findings."""

    @tool
    def search_web(self, query: str) -> str:
        """Search the web and return results."""
        ...

    @tool
    def read_page(self, url: str) -> str:
        """Fetch and extract text from a URL."""
        ...

    def run(self, task: str) -> str:
        # AgentMetrics captures the full run automatically
        results = self.search_web(task)
        content = self.read_page(results[0].url)
        return self.summarize(content)

Step 4. Run your agent. Each invocation of agent.run() creates one run in the dashboard:

agent = ResearchAgent()
result = agent.run("What are the latest developments in AI safety?")
# This run is now visible in your AgentMetrics dashboard

Adding run metadata

Pass metadata to @agentmetrics.track() to tag runs with context that's useful for filtering in the dashboard:

@agentmetrics.track(
    name="research_agent",
    tags={"environment": "production", "version": "2.1"},
)
class ResearchAgent(Agent):
    ...

You can also set per-run context dynamically using agentmetrics.set_run_context() at the start of a run:

def run(self, task: str, user_id: str) -> str:
    agentmetrics.set_run_context({"user_id": user_id, "task": task[:100]})
    # rest of agent logic

Using the decorator on functions

You don't have to use a class-based agent. The decorator works on any Python function too:

import agentmetrics

@agentmetrics.track()
def run_research_task(query: str) -> str:
    # All LLM calls and tool calls within this function
    # are captured as part of a single run
    ...

Each call to the decorated function creates one run in the dashboard.

Troubleshooting

Runs not appearing. Confirm your AGENTMETRICS_API_KEY is set in the same Python environment where your code is running. Check with import os; print(os.getenv("AGENTMETRICS_API_KEY")).

Token counts showing zero. AgentMetrics reads token usage from the model response object. If you're using a custom LLM wrapper that doesn't expose usage metadata, token tracking will be unavailable. Open a GitHub issue and we'll add support.

For other issues, email support@agentmetrics.dev.

Start tracking your Hermes agents

Free plan includes 10,000 events per month, no credit card required.

Get started free View docs