AI Crawler Access Checker

Overview

AI Crawler Access Checker is a Model Context Protocol server that determines whether GPTBot, ClaudeBot, PerplexityBot and other AI crawlers can access and read websites in your list. Any MCP-compatible AI assistant can invoke its tools directly during conversations or agent runs, letting you audit crawler permissions across multiple domains at once.

This is a hosted remote server accessible at https://mcp.apify.com/?tools=nerolabs/ai-crawler-access-checker, requiring only an API key for authentication. No local installation needed. The latest release is v0.1.0, maintained by Nero-Engine and available through Apify.

AI Crawler Access Checker sits in the Web Scraping category on mcp.site and integrates with MCP clients including Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, Zed and others. Maintainers are welcome to claim this listing to verify ownership and keep information up to date.

Installation

Settings → Connectors → Add custom connector
Name: AI Crawler Access Checker
URL:  https://mcp.apify.com/?tools=nerolabs/ai-crawler-access-checker