    servers/
        web-scraping/
        ai-crawler-readiness
  

AC

Ai Crawler Readiness

Unverified Claim & verify → Web Scraping

Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index.

Visit website ↗ Try Remote Server

Overview Installation

Overview

Ai Crawler Readiness checks whether AI crawlers can access your website by providing GPTBot and ClaudeBot verdicts, analyzing robots.txt compliance, detecting edge blocking, and drawing from a 47k-domain index. Use it to understand your site's AI crawler readiness and optimize accordingly.

This is a remote hosted server running at https://shop.lumnika.com/ai-readiness/mcp with no authentication required. Version v1.0.0 is currently available. Simply point your MCP client to the endpoint and start querying—no local installation needed.

Ai Crawler Readiness belongs in the Web Scraping category on mcp.site and works with any MCP client including Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, and Zed. If you maintain this server for lumnika.com, claim this listing to verify ownership and keep information current.

▸Hosted endpoint — Streamable HTTP · https://shop.lumnika.com/ai-readiness/mcp

Installation

Settings → Connectors → Add custom connector
Name: Ai Crawler Readiness
URL:  https://shop.lumnika.com/ai-readiness/mcp

Related servers

PM

Playwright MCP

Web Scraping

Browser automation, page inspection, screenshots, and web interaction from Claude, Cursor, and other AI agents.

          ★ 21.3k
        Updated 5h ago
    →
  

FM

Firecrawl MCP

Web Scraping Remote

Powerful web scraping and search capabilities for LLM clients like Cursor and Claude.

          ★ 12.4k
        Updated 3h ago
    →
  

BM

BitBrowser MCP

Web Scraping

Control BitBrowser profiles and automate multi-account flows.

        Added 23 hours ago
    →
  

Gatetest

Web Scraping

120-module QA gate. Give Claude eyes (screenshots), ears (Sentry errors), and hands (verify fixes).

        Added 1 month ago
    →
  

Nimrod Research

Web Scraping Remote

Web research for agents: quality-scored Google search, webpage extraction, and deep research.

          ★ 256
        Added 1 month ago
    →
  

SH

Shiplight

Web Scraping

An MCP server that provides browser automation.

        Added 1 month ago
    →
  