    servers/
        web-scraping/
        ai-crawler-access-check
  

AC

AI Crawler Access Check

Unverified Claim & verify → Web Scraping

Can GPTBot, ClaudeBot and PerplexityBot reach a site? Real fetch per crawler, firewall/CDN blocks.

Visit website ↗ Try Remote Server

Overview Installation

Overview

AI Crawler Access Check is a Model Context Protocol server that tests whether GPTBot, ClaudeBot, and PerplexityBot can reach your website. It performs real fetches as each crawler and reports firewall or CDN blocks that prevent access. Any MCP-compatible AI assistant can call its tools directly during conversations or agent runs.

This is a hosted remote server. Connect your client to https://check.citant.ai/mcp?src=registry with no local installation required. No authentication is needed. The latest release is v1.0.0, available from https://check.citant.ai/.

AI Crawler Access Check appears in the Web Scraping category on mcp.site and works with MCP clients including Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, Zed, and others. Maintainers can claim this listing to verify ownership, earn a Verified badge, and keep information up to date.

▸Hosted endpoint — Streamable HTTP · https://check.citant.ai/mcp?src=registry

Installation

Settings → Connectors → Add custom connector
Name: AI Crawler Access Check
URL:  https://check.citant.ai/mcp?src=registry

Related servers

PM

Playwright MCP

Web Scraping

Browser automation, page inspection, screenshots, and web interaction from Claude, Cursor, and other AI agents.

          ★ 21.3k
        Updated 5h ago
    →
  

FM

Firecrawl MCP

Web Scraping Remote

Powerful web scraping and search capabilities for LLM clients like Cursor and Claude.

          ★ 12.4k
        Updated 3h ago
    →
  

BM

BitBrowser MCP

Web Scraping

Control BitBrowser profiles and automate multi-account flows.

        Added 23 hours ago
    →
  

Gatetest

Web Scraping

120-module QA gate. Give Claude eyes (screenshots), ears (Sentry errors), and hands (verify fixes).

        Added 1 month ago
    →
  

Nimrod Research

Web Scraping Remote

Web research for agents: quality-scored Google search, webpage extraction, and deep research.

          ★ 256
        Added 1 month ago
    →
  

SH

Shiplight

Web Scraping

An MCP server that provides browser automation.

        Added 1 month ago
    →
  