Why I stopped using Hermes and OpenClaw (and what I built instead).

How I built my own personal AI Chief of Staff without all the complexity.

I love the idea of an AI Chief of Staff. An agent that works for you 24/7, helps you stay organised, stay focused, and on track to hit your goals.

There are some brilliant open source projects that promise exactly this. OpenClaw and Hermes Agent are the two most popular. They are incredible projects and they have genuinely transformed the way we think about AI personal assistants. They invented the category.

But after running Hermes Agent for a few months as my daily assistant, I got frustrated. The projects felt too complicated, and some of the headline (and frankly overhyped) features were plain annoying. So I built my own solution.

In this article I’ll cover what these tools actually do under the covers, which features I found genuinely useful, which ones I had to turn off, and how I ended up building something much simpler on top of my existing coding agents. Everything here comes from my video on the same topic, which you can watch here.

Start with the problem, not the technology

The biggest mistake I see with this topic is people leading with the technology instead of the problem. They install Hermes or OpenClaw, get excited, and then have no idea what to actually use it for.

Go and read the community use cases and some of them are frankly ridiculous. One person uses OpenClaw to remind them to drink water. That’s a problem looking for a solution. The Reminders app on your phone does that already, for free, with no virtual machine involved.

Here’s how I think about it instead. Hermes and OpenClaw are implementations. The reusable idea is the personal assistant pattern. The value comes from writing clear jobs and running them reliably in the background with the simplest capable agent.

For me, a personal assistant does organisational work. It triages my email, keeps my to-do list honest, prepares my weekly review, identifies things I’d otherwise miss, and keeps me pointed at my actual goals. Anything that looks like a coding task, I just give to a coding agent directly.

How these tools actually work

Under the covers, Hermes and OpenClaw are simpler than you might think. The main innovation was the messaging gateway.

A traditional coding agent runs on your machine. You send it a prompt, it does some work, you get a response. What OpenClaw did was put a gateway in front. The gateway runs 24/7 and accepts messages from Telegram, email, SMS, whatever you like. That’s what gives these assistants the feeling of being always available. When a message comes in, the gateway dispatches it to an agent runtime, the agent does the work, and the response comes back to your phone.

On top of that you get scheduled tasks (essentially cron jobs), session storage in SQLite so you can search past conversations, and a memory system.

Here’s the thing though. Over the last few months, the big coding agents have absorbed most of this. You can now message Claude Code from your phone. Same with Codex. Both can remember things about you. Both can run background cron tasks.

So the honest question becomes:

What do Hermes and OpenClaw give you that your existing coding agent doesn’t?

In my opinion: not that much. And the features that remain were the ones I liked least (memory, self improving skills).

The memory problem

The big promise of Hermes Agent is that it self-learns and improves over time. In practice, I found this to be a negative.

The memory system is simple. After each conversation turn, the agent decides whether to save any facts about you to a memory.md file. That file gets reloaded on the next turn, so the assistant “remembers” you.

Sounds great. But when I actually read my memory.md, a lot of it was false or misleading. One saved fact: my current stack is Rails and PostgreSQL. Not true. I said it once as an off-hand comment months ago, and Hermes promoted it to a permanent fact. Another entry referenced a CLI tool I’d deprecated ages ago. Every one of these bad facts gets fed back into the agent and quietly makes it worse.

The skill creation feature was the same story. Hermes watches what you do and creates reusable skills on the fly. People on YouTube rave about this. In reality, the agent would do something slightly wrong, save it as a skill, and then keep generating junk skills for random one-off tasks. The more skills you accumulate, the worse the agent performs, because every run it has to decide which skills to load and it starts getting confused. I ended up disabling skill creation entirely.

An agent writing markdown files isn’t really learning. It’s automatic documentation. I’d rather have control over that process.

What I built instead

I built Push, a single Rust binary that acts as a gateway and dispatches tasks to your existing coding agent. That’s it. We take the one genuinely valuable piece of the Hermes architecture, the gateway, and we don’t rebuild anything else. The agent runtime is configurable: Codex, Claude Code, Pi, whichever you prefer.

A few design decisions that matter:

No open ports. Push polls Telegram for new messages rather than accepting inbound connections. You can run it on a virtual machine in a private network with no inbound internet connectivity at all. No exposed webhook endpoints, no public dashboard.

Let the coding agent handle integrations. This is the underrated reason to prefer this approach. If you want Gmail working in Hermes Agent, you’re off to Google Cloud to configure OAuth credentials yourself, then managing tokens and refresh flows on your own host. It’s a nightmare to set up and a security risk. With Codex, Gmail is a one-click connection and the credential management is handled by OpenAI. Same for GitHub, Google Drive, Notion, your calendar. You get security best practices for free because these are public consumer products.

Opinionated and minimal. Hermes and OpenClaw are extremely unopinionated. Endless options, loads of messaging platforms, so much to configure. I wanted the opposite. Push has two channel options (Telegram and iMessage for now), one agent setting, and an assistant root directory. Run push init and it creates the whole repository for you.

The three things that actually matter

When you strip everything else away, I think a personal assistant setup only has three parts.

1. Context. What does the agent know about you? In my assistant root I keep a plan.md with my current focus, priorities, and a decision filter. The agent reads this on every run, which means every recommendation it makes is anchored to what I’m actually trying to achieve. Here’s mine:

# Plan

## North star
Grow AI Engineer to $50,000 MRR by making it the best practical program for
people learning to build and profit from software with AI.

AI Engineer is the one offer.

## Current priorities
1. Make AI Engineer exceptional through better courses, workshops, community,
   software, and resources that improve student outcomes.
2. Massively grow qualified traffic and turn that attention into members.
3. Produce successful students and credible proof that compounds growth.

## Decision filter
...

The “Not now” section is the part people skip and shouldn’t. An assistant that knows what you’ve decided not to do is far better at challenging distractions.

2. Jobs. These are the recurring tasks, and each one is literally just a prompt plus a schedule in a single file. Here’s my daily AI news brief job in full:

+++
version = 1
timeout = "12m"
workdir = "/Users/owainlewis/.cos/work"
backend = "codex"

[[triggers]]
id = "daily-morning"
kind = "cron"
schedule = "0 8 * * *"
timezone = "Europe/London"
enabled = true
+++

Prepare Owain's daily AI engineering news brief.

Find material developments that help an AI engineer build software more
effectively:
- new coding models or meaningful model updates;
- Claude Code and Codex features, releases, and breaking changes;
- important changes to agentic development tools and workflows;
- new engineering techniques supported by credible primary evidence.

...

A few things worth noticing in there. The frontmatter is just a cron schedule and a backend, nothing clever. The prompt tells the agent to treat everything it reads on the web as untrusted data, not instructions, which matters when an agent is browsing autonomously every morning. And it checks its own previous run to avoid repeating stories, so the brief stays fresh without any fancy memory system.

The whole jobs system lives in plain files, which means it’s reviewable, version-controlled, and lives in GitHub like everything else. You can add anything you want here: daily focus, email triage, weekly reviews, research digests.

3. Soul. A SOUL file that defines how the assistant should act. Here’s mine in full:

# SOUL

You are Owain's chief of staff for his work as an AI engineer, educator,
creator, and open-source maintainer.

Your job is to turn a noisy stream of tasks, messages, meetings, and ideas into
clear decisions and useful action. Protect his attention. Keep the important
work moving. Reduce operational drag without taking control away from him.

## Working style
- Be calm, direct, practical, and honest.
- Lead with what matters now.
- Make a recommendation and explain the deciding reason.
- Challenge distractions and false urgency.
- Prefer simple systems that will still work next month.
- Ask when a decision is important and genuinely unclear.
- Protect private information and confirm before consequential external
  actions.
- Keep answers concise, but include enough evidence to trust them.

## What success looks like
- AI Engineer becomes the clearest, most practical program for software
  builders who want to ship real products with AI.
- YouTube grows to 100,000 subscribers because the work is consistently useful,
  clear, and distinctive.
- Email, sponsorships, tasks, calendar, and project maintenance stay under
  control without dominating the week.

Notice there’s nothing technical in here. It’s a job description. The more clearly you can describe what a great chief of staff would do for you, the better this file gets.

Alongside that, an AGENTS.md file tells the coding agent what tools are available. Mine lists Todoist via the td CLI for tasks, email and calendar connections, Passage for research documents, GitHub, and Push itself so the agent can query its own jobs and memory.

What this looks like day to day

A few patterns I use constantly:

Email triage into the to-do list. I ask the agent to go through the last 24 hours of email and add anything needing action to Todoist. Last run it reviewed 120 emails and created four tasks, with due dates. My task list becomes the control plane: I either pick a task up myself or delegate it back to the agent.

Delegating replies. For a routine sponsorship email, the agent already knows my preferences, so I just tell it to draft the response.

Research pushed into my workspace. This one I find really useful. I’ll ask for something like the last 48 hours of AI news relevant to an AI engineer, saved into a Passage document I can read on my phone. The agent doesn’t just do the research, it pushes the output into the tools I actually use, so there’s no disconnect between the agent and my workspace.

Voice both ways. I can send a voice note from Telegram and get a voice reply back. Genuinely handy when I’m away from my desk.

Where I’ve landed

I’d still recommend trying Hermes Agent and OpenClaw. They’re great projects and they’ve given us the ideas this whole category is built on. If the self-learning stuff works for you, brilliant.

But for me, most of what they offer is now a commodity, and the distinctive features, the memory and the self-improvement, made my assistant worse, not better. Doing dynamic self-learning well is exceptionally hard, and the trade-offs are real.

Sometimes you just need to ignore the hype.

The pattern that survived is simple: a small gateway, clear context files, reviewable jobs, and the heavy lifting done by a coding agent that already handles security, integrations, and credentials better than I ever would myself.

If you want to see the full setup running, including the live email triage and switching the runtime between Codex and Claude Code, the video walks through all of it:

Have an awesome day,

Owain