How to integrate MrScraper MCP with Hermes

Hermes is a 24/7 autonomous agent that lives on your computer or server — it remembers what it learns and evolves as your usage grows. This guide explains the easiest and most robust way to connect your MrScraper account to Hermes. You can do this through either Composio Connect CLI or Composio Connect MCP. For personal use we recommend the CLI, but you won't go wrong with MCP either.

MrScraper logoMrScraper
Api Key

MrScraper is a web scraping API platform for structured extraction, search results, marketplace data, travel data, and web unblocking. Use it to turn messy web pages into clean, usable data without managing scraper infrastructure.

11 Tools

Introduction

Hermes is a 24/7 autonomous agent that lives on your computer or server — it remembers what it learns and evolves as your usage grows.

This guide explains the easiest and most robust way to connect your MrScraper account to Hermes. You can do this through either Composio Connect CLI or Composio Connect MCP. For personal use we recommend the CLI, but you won't go wrong with MCP either.

Also integrate MrScraper with

What is Composio Connect?

Composio Connect is a consumer offering that lets anyone plug 1,000+ applications directly into their agent harness — including Hermes. It can:

  • Search and load tools from relevant toolkits on-demand, reducing context usage.
  • Chain multiple tools to accomplish complex workflows via a remote workbench, without excessive back-and-forth with the LLM.
  • Manage app authentication end-to-end with zero manual overhead.

Integrating MrScraper with Hermes

Using Composio Connect CLI

1. Install the Composio CLI

Run the install script directly, or paste https://composio.dev/hermes into your Hermes chat box to have it installed for you.

bash
curl -fsSL https://composio.dev/install | bash
Hermes authenticating with Composio

2. Authenticate

Once the CLI is installed, ask Hermes to authenticate with Composio.

3. Connect to MrScraper

Ask your agent to connect to MrScraper, or simply request any MrScraper-related task. Hermes will prompt you to authenticate and authorize access.

4. Done. You're all set with a new MrScraper connection.


Using Composio Connect MCP

1. Get your MCP URL and API Key

Go to dashboard.composio.dev and copy your Connect MCP URL and API key.

Copy MCP URL and API key from Composio dashboard

2. Open the Hermes config file

bash
nano ~/.hermes/config.yaml

3. Add the Composio Connect MCP server

bash
mcp_servers:
  composio:
    url: "https://connect.composio.dev/mcp"
    headers:
      x-consumer-api-key: "YOUR_COMPOSIO_API_KEY"
    connect_timeout: 60
    timeout: 180

Save with Ctrl + O, Enter, then exit with Ctrl + X.

4. Restart your Hermes agent

Once restarted, ask your agent to connect to MrScraper or request any MrScraper-related task. It will prompt you to authenticate and authorize access.

5. Done!

What is the MrScraper MCP server, and what's possible with it?

The MrScraper MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your MrScraper account. It provides structured and secure access to your scraper workflows, results, usage, and analytics, so your agent can create AI scrapers, rerun saved scrapers, retrieve results, repair manual workflows, and review scraping activity on your behalf.

  • AI scraper creation: Have your agent create and run general, listing, or map AI scrapers against websites.
  • Single and bulk scraper runs: Direct your agent to rerun saved AI or manual scrapers against one website or multiple websites at once.
  • Result tracking and retrieval: Let the agent list past scrape results, filter and sort them, then retrieve the current status and output of a selected run.
  • Manual scraper repair: Instruct your agent to repair a saved manual scraper when changes to its target website cause the workflow to stop matching.
  • Usage and performance monitoring: Have your agent review plan usage, token consumption, concurrency, rate limits, success rates, request volume, latency, and scraping trends over time.

Way Forward

With MrScraper connected, Hermes can now act on your behalf whenever it detects a relevant task or you ask it to.

From here, you can extend Hermes further:

  • Connect more apps: Calendar, Slack, Notion, Linear, and hundreds of others are available through the same Composio Connect setup. Each new integration compounds what Hermes can do for you.
  • Build workflows across tools: Once multiple apps are connected, Hermes can chain actions together — turn an email into a calendar invite, a Slack message into a Linear ticket, or a meeting note into a follow-up draft.
  • Let it learn your patterns: The more you use Hermes, the better it gets at anticipating how you'd handle recurring tasks. Give it feedback on drafts and decisions, and it will adapt.

If you run into trouble or want to share what you've built, join the community or check out the Docs for deeper configuration options.

TOOLS

Supported Tools

Every MrScraper action and event your agent gets out of the box.

Bulk Rerun AI Scraper

Run an existing General AI scraper configuration against multiple URLs in one request.

Bulk Rerun Manual Scraper

Run an existing saved manual scraper workflow against multiple URLs in one request.

Create AI Scraper

Create and run a general, listing, or map AI scraper against a website.

Get Account Usage

Return the connected MrScraper account's plan, token usage, concurrency allowance, and rate limits without exposing user identity or token details.

Get Scrape Status Analytics

Return aggregate scrape counts, success rate, request rate, latency, and status totals for a domain and action over a time range.

Get Scrape Analytics Timeline

Return time-series scrape activity and success metrics for a domain and action over a time range.

Get Scrape Result

Retrieve the current status and output for one accessible scrape result by ID.

Heal Manual Scraper

Ask MrScraper to repair an existing manual scraper workflow that no longer matches its target site.

List Scrape Results

List scrape results for the connected account with filtering, sorting, and agent-controlled pagination.

Rerun AI Scraper

Run an existing AI scraper configuration against a new URL, with optional map crawl controls.

Rerun Manual Scraper

Run an existing saved manual scraper workflow against a new URL.

FAQ

Frequently asked questions

With a standalone MrScraper MCP server, the agents and LLMs can only access a fixed set of MrScraper tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from MrScraper and many other apps based on the task at hand, all through a single MCP endpoint.

Yes, you can. Hermes fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right MrScraper tools.

Yes, absolutely. You can configure which MrScraper scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do.

All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your MrScraper data and credentials are handled as safely as possible.

Start with MrScraper.It takes 30 seconds.

Managed auth, hosted MCP servers, and every MrScraper tool your agent needs.Free to start.

Start building