How to integrate FutureAGI MCP with Hermes

Hermes is a 24/7 autonomous agent that lives on your computer or server — it remembers what it learns and evolves as your usage grows. This guide explains the easiest and most robust way to connect your FutureAGI account to Hermes. You can do this through either Composio Connect CLI or Composio Connect MCP. For personal use we recommend the CLI, but you won't go wrong with MCP either.

FutureAGI logoFutureAGI
Api Key

FutureAGI is an AI evaluation and observability platform for testing, tracing, simulating, and improving AI systems. Use it to measure model quality, debug failures, manage datasets, and close the loop with annotation workflows.

9 Tools

Introduction

Hermes is a 24/7 autonomous agent that lives on your computer or server — it remembers what it learns and evolves as your usage grows.

This guide explains the easiest and most robust way to connect your FutureAGI account to Hermes. You can do this through either Composio Connect CLI or Composio Connect MCP. For personal use we recommend the CLI, but you won't go wrong with MCP either.

Also integrate FutureAGI with

What is Composio Connect?

Composio Connect is a consumer offering that lets anyone plug 1,000+ applications directly into their agent harness — including Hermes. It can:

  • Search and load tools from relevant toolkits on-demand, reducing context usage.
  • Chain multiple tools to accomplish complex workflows via a remote workbench, without excessive back-and-forth with the LLM.
  • Manage app authentication end-to-end with zero manual overhead.

Integrating FutureAGI with Hermes

Using Composio Connect CLI

1. Install the Composio CLI

Run the install script directly, or paste https://composio.dev/hermes into your Hermes chat box to have it installed for you.

bash
curl -fsSL https://composio.dev/install | bash
Hermes authenticating with Composio

2. Authenticate

Once the CLI is installed, ask Hermes to authenticate with Composio.

3. Connect to FutureAGI

Ask your agent to connect to FutureAGI, or simply request any FutureAGI-related task. Hermes will prompt you to authenticate and authorize access.

4. Done. You're all set with a new FutureAGI connection.


Using Composio Connect MCP

1. Get your MCP URL and API Key

Go to dashboard.composio.dev and copy your Connect MCP URL and API key.

Copy MCP URL and API key from Composio dashboard

2. Open the Hermes config file

bash
nano ~/.hermes/config.yaml

3. Add the Composio Connect MCP server

bash
mcp_servers:
  composio:
    url: "https://connect.composio.dev/mcp"
    headers:
      x-consumer-api-key: "YOUR_COMPOSIO_API_KEY"
    connect_timeout: 60
    timeout: 180

Save with Ctrl + O, Enter, then exit with Ctrl + X.

4. Restart your Hermes agent

Once restarted, ask your agent to connect to FutureAGI or request any FutureAGI-related task. It will prompt you to authenticate and authorize access.

5. Done!

What is the FutureAGI MCP server, and what's possible with it?

The FutureAGI MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your FutureAGI account. It provides structured and secure access to your evaluations, datasets, annotation workflows, agent definitions, and simulation resources, so your agent can review evaluation tasks, find datasets, inspect annotation work, browse agents, and explore simulations on your behalf.

  • Evaluation setup discovery: Have your agent find evaluation tasks and custom evaluation configurations for a specific project or task name.
  • Dataset and label review: Let the agent search your datasets and review annotation labels by name, type, project, or dataset.
  • Annotation queue oversight: Direct your agent to list annotation queues by name or status and include item counts when needed.
  • Agent and test inspection: Instruct your agent to find agent definitions and review configured run tests by name, source type, or prompt template without running them.
  • Simulation resource exploration: Have your agent browse personas and scenarios using agent type, simulation mode, agent definition, or search text.

Way Forward

With FutureAGI connected, Hermes can now act on your behalf whenever it detects a relevant task or you ask it to.

From here, you can extend Hermes further:

  • Connect more apps: Calendar, Slack, Notion, Linear, and hundreds of others are available through the same Composio Connect setup. Each new integration compounds what Hermes can do for you.
  • Build workflows across tools: Once multiple apps are connected, Hermes can chain actions together — turn an email into a calendar invite, a Slack message into a Linear ticket, or a meeting note into a follow-up draft.
  • Let it learn your patterns: The more you use Hermes, the better it gets at anticipating how you'd handle recurring tasks. Give it feedback on drafts and decisions, and it will adapt.

If you run into trouble or want to share what you've built, join the community or check out the Docs for deeper configuration options.

TOOLS

Supported Tools

Every FutureAGI action and event your agent gets out of the box.

List Agent Definitions

List one page of agent definitions in the connected FutureAGI organization, optionally filtering by type, name, assistant ID, or agent ID.

List Annotation Labels

List one page of FutureAGI annotation labels by type, name, project, or dataset, optionally including usage counts.

List Annotation Queues

List one page of FutureAGI annotation queues by status or name, optionally including item counts.

List Custom Eval Configs

List custom evaluation configurations, optionally restricted to a FutureAGI project or evaluation task.

List Datasets

List one page of datasets in the connected FutureAGI organization, optionally searching by name.

List Eval Tasks

List one page of FutureAGI evaluation tasks, optionally filtering by project ID or task name.

List Personas

List one page of built-in and workspace personas for FutureAGI simulations, with type, simulation-mode, and text filters.

List Run Tests

List one page of configured FutureAGI run tests without executing them, optionally filtering by name, source type, or prompt template.

List Scenarios

List one page of FutureAGI simulation scenarios, optionally filtering by agent definition, agent type, or search text.

FAQ

Frequently asked questions

With a standalone FutureAGI MCP server, the agents and LLMs can only access a fixed set of FutureAGI tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from FutureAGI and many other apps based on the task at hand, all through a single MCP endpoint.

Yes, you can. Hermes fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right FutureAGI tools.

Yes, absolutely. You can configure which FutureAGI scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do.

All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your FutureAGI data and credentials are handled as safely as possible.

Start with FutureAGI.It takes 30 seconds.

Managed auth, hosted MCP servers, and every FutureAGI tool your agent needs.Free to start.

Start building