    servers/
        web-scraping/
        atalaya-ai-crawler-check
  

AA

Atalaya AI crawler check

Unverified Claim & verify → Web Scraping

Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

Visit website ↗ Try Remote Server

Overview Installation

Overview

Atalaya AI crawler check is a Model Context Protocol server that reveals which AI crawlers—GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot—a website permits and explains why. Any MCP-compatible AI assistant can invoke its tools directly during conversations and agent runs, letting you audit crawler access policies programmatically.

The server runs as a remote, hosted endpoint at https://atalaya-ai.com/mcp with no local installation required and no authentication needed. Version v1.1.0 is the latest release. Full documentation and setup details are available at https://atalaya-ai.com/mcp-server/.

Atalaya AI crawler check belongs in the Web Scraping category on mcp.site and integrates with all MCP clients including Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, Zed and others. Maintainers at atalaya-ai.com are welcome to claim this listing to verify ownership, display the Verified badge, and ensure the information stays accurate.

▸Hosted endpoint — Streamable HTTP · https://atalaya-ai.com/mcp

Installation

Settings → Connectors → Add custom connector
Name: Atalaya AI crawler check
URL:  https://atalaya-ai.com/mcp

Related servers

Gatetest

Web Scraping

120-module QA gate. Give Claude eyes (screenshots), ears (Sentry errors), and hands (verify fixes).

        Added 2 months ago
    →
  

Nimrod Research

Web Scraping Remote

Web research for agents: quality-scored Google search, webpage extraction, and deep research.

          ★ 256
        Added 2 months ago
    →
  

SH

Shiplight

Web Scraping

An MCP server that provides browser automation.

        Added 2 months ago
    →
  

Scrapegraphai Scrapegraph Mcp

Web Scraping Remote

Enable language models to perform advanced AI-powered web scraping with enterprise-grade reliabili….

          ★ 100
        Added 2 months ago
    →
  

Pinkpixel Dev Web Scout Mcp

Web Scraping Remote

Search the web and extract clean, readable text from webpages. Process multiple URLs at once to sp….

          ★ 132
        Added 2 months ago
    →
  

VE

Verant

Web Scraping Remote

AI proofreading for public web pages and full sites, with AI-verified corrections. Paid plans only.

        Added 2 months ago
    →
  