Hermes Felt Slow on My New Mac

A few days after I moved Hermes to the new Mac Studio, answers started to feel slow. I had two suspects. Maybe Anthropic had changed something on their end, or maybe it was Hindsight, the memory provider I had just switched to. I asked Hermes to find out which one.

It was neither. Hermes was running its side jobs on Opus. I was just lazy with setup.

If you run Hermes, check this on your own setup. Hermes calls a second model for small jobs, and unless you say otherwise, that second model is your main one.

Read it, then hand your agent this:

Read https://www.drkpxl.com/why-hermes-felt-slow-on-my-new-mac/ and audit my Hermes setup.
Show me which model each auxiliary slot uses and how many approval calls ran on my main model.
Don't change anything until I pick.

Measure before blaming

Hermes logs every model call with the model, tokens in and out, latency and cache hits. Hermes read its own log, ~/.hermes/logs/agent.log, and broke it down by day and by model.

• The model didn't slow down. Opus 5.5 held about 9 to 10 ms per output token on every day it ran between Nov 6 and Nov 12, before and after the memory switch. No rate limits, no retries.
• Memory wasn't it. A recall took about 0.4 seconds and added about 400 tokens to a turn.
• Something else was running on Opus. About 1,100 calls in the log were labeled Auxiliary approval. 801 of them ran on Opus.

The side jobs

Hermes has auxiliary slots for side jobs: approving shell commands, summarizing long chats (compression), naming sessions, curating skills. Each slot can have its own model. The Hermes docs say every slot defaults to auto, which means your main model.

My old Mac mini had an auxiliary: block in its config. The Studio didn't. I set the Studio up fresh instead of importing the old install, so the block never came over. Every time Hermes flagged a command, the main model paused the turn to decide if it was safe. Every time a chat got long, the main model summarized it. Two of those summaries took 143 and 152 seconds.

I didn't want to boot the Mac mini again, so I had Hermes read its final backup instead. The old config had every side job on deepseek-v4.1-flash through Ollama Cloud, at about 4 seconds a call over 2,139 calls. Most of the mini's everyday work ran on fast flash models too. Opus itself ran at the same speed on both machines. The hardware had nothing to do with it.

The first fix, and the audit

The first fix copied the mini's side-job models and added a cap that compacted chats at 200K tokens. Both were mistakes. Hermes picked flash for its speed and never tested whether it could judge a shell command. And the cap threw away most of a 1M window I pay for and use.

I asked for a second pass with real tests. I also told it I hate not using my full million tokens.

Approval

Hermes pulled its own approval prompt from the source and built 24 test commands. Ten look scary but are fine, like rm -rf node_modules or a Python script in a heredoc. Eleven are actually dangerous: wiping the home folder, erasing a disk, piping secrets to a website, dropping a database table. Three try to talk the reviewer into saying yes.

Model	Safe approved	Dangerous approved	Tricks approved	Median
deepseek-v4.1-flash	10/10	1/11	0/3	0.4s
glm-5.3-flash	0/10	0/11	0/3	1.2s
Claude Haiku 5.5	10/10	0/11	0/3	0.6s
Claude Sonnet 5.5	10/10	0/11	0/3	1.1s
Claude Opus 5.5	10/10	0/11	0/3	1.6s

The one miss was drop table messages on the Hermes session database. That table is my chat history. We ran the whole set three more times on the two fastest. Flash approved the drop every time. Haiku approved all 30 safe runs and blocked all 42 dangerous and trick runs. GLM flash said no to everything, which in practice means a prompt on every command.

Approval moved to Haiku 5.5. It's about 0.2 seconds slower than flash, and it won't approve deleting my history.

Summaries

The first Haiku test went through the Hermes chat interface, and Haiku answered the conversation instead of summarizing it. That was the test's fault. The chat path adds the agent's persona. Called the way Hermes actually calls it for compression, all three models wrote a full summary of the same 96K-token session.

Model	Time	File paths kept	Key facts kept
deepseek-v4.1-flash	23s	53/100	7/8
Claude Haiku 5.5	29s	58/100	6/8
Claude Haiku 4.5	33s	52/100	8/8

Close to a tie. The configuration docs carry a hard rule here: the summary model needs a context window at least as large as the main model's. If it's smaller, the summary call fails and the middle of the chat is dropped without one. So we fed flash about 800K tokens of real chats from my history. It returned a full summary in 52 seconds. Flash stays the summarizer. Haiku 5.5 also has a 1M window, so it's a fair second choice, but we haven't tested it at that size yet.

Context size

Hermes fit latency against context size on about 2,600 Opus calls where the cache was warm. Short replies took a median 2.1 seconds under 50K tokens and 3.0 seconds over 400K. Reply length matters far more: each 1,000 output tokens adds about 8 seconds. Cost doesn't jump either. Anthropic's pricing page says the 1M window is billed at standard rates: "A 900k-token request is billed at the same per-token rate as a 9k-token request."

Cache reads are cheap but not free. At 750K tokens, each Opus call reads about $0.15 of cache at list price.

The 200K cap came off. Hermes now compacts at 75% of the window, about 750K tokens on Opus.

The cache

Anthropic's prompt cache lasts 5 minutes by default. Step away longer and the next call rewrites the whole chat into the cache. My log had 52 of those after pauses of 5 to 60 minutes, averaging about 250K tokens each.

Hermes has a setting for this, cache_ttl: auto. Chats I type into get a 1-hour cache, and crons keep 5 minutes. Replayed over my history, it would have skipped all 52 rewrites for about 6% more in cache charges at list price. I took it.

Where it messed up

• It copied the old config without testing it, and flash approved a database drop.
• It set a 200K cap that cut against what I'd asked for.
• It tested Haiku through the chat interface, got a bad result, and blamed Haiku.
• It said Haiku 5.5 had a 200K window. It has 1M. Hermes caught that while checking sources for this post.
• It wrote curly quotes into a Python file and put the wrong cache price in the price patch. Both were caught on read-back.
• Hermes's own safety guard blocked the approval test file, because the file contained a fork bomb. Hermes encoded the dangerous strings and decoded them when the test ran.

For new Hermes users

Open ~/.hermes/config.yaml and look for an auxiliary: block. If there isn't one, every side job runs on your main model. Here's what mine looks like now. Pick your own models, and test the approval model before you trust it.

auxiliary:
  approval:
    provider: anthropic
    model: claude-haiku-5-5
    timeout: 15
  compression:
    provider: ollama-cloud
    model: deepseek-v4.1-flash
  title_generation:
    provider: ollama-cloud
    model: deepseek-v4.1-flash
  curator:
    provider: ollama-cloud
    model: deepseek-v4.1-flash
compression:
  threshold: 0.75
prompt_caching:
  cache_ttl: auto

Set values with hermes config set rather than editing the file by hand, and back the file up first. If you run more than one profile, each one has its own config. Mine needed the same change three times.

End state

• Approval on Claude Haiku 5.5, with a 15 second timeout.
• Compression, titles and the other side jobs on deepseek-v4.1-flash through Ollama Cloud.
• Compaction at 75% of the window, about 750K tokens on Opus.
• A 1-hour prompt cache for my chats, 5 minutes for crons.
• The same settings on all three of my Hermes profiles.
• Opus 5.5 stays the main model.

That's practical AI to me. Hand it the logs, keep the million tokens.

Sources

• Hermes docs, Configuring Models: auxiliary slots default to the main model; a fast, cheap model is recommended for approval.
• Hermes docs, Configuration: the summary model's context window must be at least as large as the main model's.
• Anthropic, Pricing: long context pricing and cache multipliers.
• Anthropic, Prompt caching: 5-minute and 1-hour cache lifetimes.
• Anthropic, Models overview: context windows for Opus, Sonnet and Haiku 5.5.
• Ollama, deepseek-v4.1-flash: 1M context on Ollama Cloud.