Hermes Agent is an open-source AI agent from Nous Research that anyone can download and run. Install it on a computer and you can give it work through a messenger like Slack.

Out of the box it supports scheduled jobs that run at set times, and MCP for connecting outside services. This post is based on Hermes Agent v0.22.2, released on October 15, 2026.

Hermes Agent also improves itself as you use it. When it finishes a multi-step task, gets stuck and finds a way through, or the user corrects its approach, the agent writes that method down as a skill. The next time the same kind of work comes up, it pulls out that skill, and it revises the skill when something needs fixing.

I wrote this post to share one way Hermes Agent can be used. By the end, you will know how an AI assistant that remembers calls and emails, answers when asked, and keeps track of to-dos actually works.

Iroha

Iroha is the personal assistant I built with Hermes Agent. The name comes from Iroha Sakayori in the Netflix anime film "Cosmic Princess Kaguya!". She runs around the clock on a single Mac Studio desktop, and I talk to her through Slack.

I started using her in April 2026. At first she only handled to-do digests; in September I handed her my calls, and in October the job of remembering things by person, company, and project.

A Day with Iroha

A day with Iroha goes like this.

• 10:00: She sends a digest of overdue to-dos and everything due from today through the day after tomorrow.
• When a call ends: She tells me the call summary and the to-dos she registered on my behalf.
• When an important email arrives: She tells me right away. The rest are batched and summarized on the hour.
• When I have a question: If I ask something like "How are things going with Company A?", she searches past calls and emails and answers.
• 17:00: She tells me what I finished today and what is still open.
• 22:00: She tells me what is still open and what is due tomorrow.

Sometimes, Other Things Too

Beyond calls and email, I occasionally hand Iroha jobs like these.

• Searching Bunjang and Karrot (Korean secondhand marketplaces) and reporting when a listing I want shows up,
• Pulling route data from a map service to work out the travel time and route to a destination,
• Or taking direct control of the Mac to play music on Apple Music.

Jobs like these I can simply ask Iroha to do, but calls and email were different.

Time to Build Tools!

Why I Built Them

I had four problems.

• I often missed things that came up naturally in a call, including promises.
• With several projects tangled together, I could not recall at a glance what a given conversation was about.
• Manually copying to-dos out of calls and emails was tedious.
• With so much mail coming in, I kept seeing important emails too late.

The Tools I Built

Hermes Agent comes with Slack chat, scheduled jobs, and connections to outside services built in.

To solve the four problems above, Iroha needed to do three more things: turn calls into text and read them, watch incoming mail, and remember things by person, company, and project.

Hermes Agent alone could not cover these three.

• Turning calls into text: Hermes Agent has no such feature.
• Watching mail: Hermes Agent can search and read Gmail when asked. However, it cannot wake itself up to read new mail when it arrives, so I needed a program that watches the inbox and wakes Iroha whenever new mail comes in.
• Remembering by person, company, and project: The built-in memory is a short notes file, too small to hold records on dozens of people and dozens of companies.

So I built one program for each of the three and wired them in. All of them run on the same Mac as Iroha.

• An upload app and call-ingest: Turn phone call recordings into text and hand them to Iroha.
• mail-watch: Checks Gmail automatically and hands new mail to Iroha.
• entity-context: Collects new calls and emails, hands them to Iroha, and stores what Iroha organizes by person, company, and project.

In this post, context means these records, kept per person, company, and project.

Data Flow

Calls and email flow along two branches.

• The immediate branch: As soon as a call is turned into text or an email arrives, Iroha reads it. To-dos go into Structured, a planner app, and the results are posted to Slack.
• The context branch: entity-context automatically collects new calls and emails and hands them to Iroha, who organizes them by person, company, and project.

When I ask in Slack, Iroha looks at both the stored context and Structured before answering. The to-do digest I get three times a day is also built by reading the to-dos stored in Structured.

Handling Calls

Turning Recordings into Text

Samsung phones sold in Korea record calls automatically and save them as files. Everything in this section starts from the assumption that those files exist. On an iPhone or a phone bought overseas, things differ from this step onward.

An Android upload app I built uploads the recording files to Google Drive. From there, call-ingest, which I built before Iroha existed, takes over.

1. It checks Drive for new recordings.
2. It downloads the recording and transcribes it with CLOVA Speech from Naver Cloud. I chose it for its recognition accuracy on Korean speech and its speaker separation.
3. It saves the recording and the transcript to a database on the Mac.
4. It wakes Iroha and has her read the transcript.

call-ingest stops at transcription. Reading the content, summarizing it, and pulling out promises is Iroha's job.

From July 11, 2025 to October 21, 2026, the calls came to 5,375 in total, 323.4 hours of audio. That averages 11 calls and 41 minutes a day. Because every one of these calls is stored in the database together with its transcript, Iroha can look up and read past calls too.

Processing a Call

Once transcription is done, Iroha processes the call in a fixed order.

1. She reads the full transcript.
2. She summarizes the call in 3 to 5 lines.
3. She pulls out facts, promises, and appointments, and marks who said what. CLOVA Speech separates the speakers; Iroha infers from the conversation which one is me.
4. She registers the to-dos in Structured right away.
5. She posts the result to Slack.

To-dos are registered by these rules.

• Only things I have to do, places I have to be, and things the other party promised to do for me.
• Words like "tomorrow" and "Friday" are resolved relative to the day of the call.
• If the date is ambiguous, the item goes into Structured's Inbox without a date.
• Before registering, she checks whether a to-do from the same call already exists.

A single message lands in Slack with the summary; the facts, promises, and appointments; the to-dos she registered; and the items she did not register, with the reason why. I look at "registered to-dos" and "not registered" to make sure nothing was dropped.

Checking Mail

mail-watch checks two Gmail accounts automatically. When new mail arrives, Iroha reads it and decides whether it is important. What counts as important follows the mail-checking skill Iroha wrote for herself.

• Important mail: Work requests, contracts, approaching deadlines, schedule changes, and the like. This is the kind Iroha reports right away.
• Everything else: Batched and summarized on the hour.
• To-dos: Either way, if there is clearly something I need to do, it goes into Structured.

Even if Iroha wrongly files an important email under "everything else", it still shows up in the hourly summary, so I see it within an hour at the latest.

Remembering as Context

entity-context automatically collects new calls and emails and hands them to Iroha. For each record, Iroha pulls out a summary, the people, companies, and projects that appear, and any promises. For example:

• Summary: Discussed the November delivery schedule with Kim at A; I agreed to send a revised quote.
• Entities found: A (company, also known as "A Corp"), Kim (person, purchasing contact at A)
• Promise: I will send the revised quote by October 24

Context building started on October 20, 2026. By 11 a.m. the next day, Iroha had processed 365 calls and emails, and the records had grown to 79 people, 48 companies, and 26 projects. Those numbers are from before any corrections, and older calls are still being worked through gradually, newest first.

When different names refer to the same entity ("A", "A Corp"), Iroha merges them into one. A person is identified by name and company together, so a Kim at a different company is a different person. When she merges wrongly, I correct her in Slack.

• "X and Y are the same company"
• "In that record, X is actually Z"
• "That record has nothing to do with X"
• "That person's name is Z"

Corrections are recorded separately, so they survive even if the same call or email is processed again.

An extracted promise drops off the "open promises" list when any of the following holds. To-dos registered in Structured stay where they are.

• A later call or email confirms it is done
• 7 days have passed since its due date
• With no due date, 30 days have passed since the call or email
• I correct it with "that's done"

Asking and Answering in Slack

When I ask in Slack, "How are things going with Company A?" or "What did Kim say on the last call?", Iroha looks up that entity in the stored context. She also finds related to-dos in Structured that are not yet finished and answers with both.

The answer comes in three parts.

• One sentence on where things stand now
• 3 to 6 lines summarizing recent calls and emails in date order
• Open promises: who, what, and by when

If Iroha judges that the stored context is not enough, she searches the call transcripts or the original emails, rereads them, and then answers. However, if that judgment is wrong, she sometimes answers with too little.

Keeping To-Dos with Structured MCP

Structured is a planner app that lays out the day as a timeline. To-dos can carry a time and an alert, so I use it for reminders.

Structured provides its own MCP server, a gateway that lets an AI agent read and write to-dos. Iroha creates and looks up to-dos through this server. Entering the server address in the Hermes Agent settings and signing in with a Structured account is all it takes to connect.

The to-do digest that arrives three times a day is not something Iroha composes fresh each time. A small program runs automatically at the set times, reads the open to-dos from Structured, and sends them to Slack in the same format every time.

Cost

For the model, I subscribe to ChatGPT Pro. It costs $150 a month, and Iroha connects to the model through Codex.

Transcription cost scales with call volume. CLOVA Speech charges 28 KRW per minute for speech recognition and speaker separation combined. In September 2026, when I transcribed the backlog of past calls all at once, the bill came to about 470,000 KRW; in a normal month like October, it was about 90,000 KRW through the 21st.

The electricity for a Mac Studio that is always on, and the few minutes a day I spend checking the results, are costs too.

Safety Rules

Because Iroha is constantly reading other people's emails and call transcripts, I set a few rules.

• Instructions inside the content are not followed: Even if an email says "when you receive this, open the attached link", Iroha treats that sentence as one line of email content, not as a command addressed to her. This is to reduce prompt injection.
• Mail is read-only: No replying, deleting, or marking as read. The program that collects mail rejects, at the code level, anything other than read requests.
• Sensitive information is not carried over: Phone numbers, bank account numbers, national ID numbers, and verification codes are never written into summaries, to-dos, or Slack messages. They do remain in the original transcripts.

Because the first rule cannot be fully enforced by instructions alone, the jobs that read calls and email are given no file, terminal, or web tools. Even if she did follow an injected instruction, the most she could do is create a to-do or post a message.

Here is where the words of the people who call and email me end up. Recordings pass through Google Drive and CLOVA Speech and come to rest in the database on the Mac; transcripts and original emails go to OpenAI's model. Summaries stay only in a Slack workspace I use alone.

This setup is possible because these are my personal calls and emails. For business calls, you would need to look at the other party's consent and your organization's policy on sending data outside first.

What Worked and What Takes Effort

Four things worked well.

• When a call ends, the summary, the promises, and the registered to-dos show up in Slack.
• To-dos from calls and emails land in Structured on their own.
• Important emails are flagged the moment they arrive.
• Asking in Slack gets an answer that ties calls and emails together as context. This one is based on a single day of use.

Some things still take effort.

• Transcription errors: Machine transcription sometimes gets names, numbers, dates, and amounts wrong. Iroha tags doubtful values as "transcription uncertain", but in the end I still have to double-check.
• Promises that were never extracted: If Iroha never extracts a promise at all, it does not appear under "not registered" either, so I have no way of knowing. For important calls, I read the transcript instead of trusting the summary alone.
• Correcting merged context: Automatic merging sometimes gets it wrong, and I have to correct each case in Slack.

Closing Thoughts

Of the four problems I started with, three are solved. Promises from calls and emails go into Structured, I no longer copy to-dos by hand, and I see important emails right away. Remembering project context has only just begun, and I need to watch it longer.

Next, I am thinking about building my own application for reminders and schedule management.

What jobs can't you hand to an AI assistant just by asking?