agent-web — URL to LLM-ready markdown: a polite, robots-respecting web

Overview

agent-web is a Model Context Protocol server that converts web URLs into clean, LLM-ready markdown. It respects robots.txt and works without requiring any API key, making it free and straightforward to use. By speaking the Model Context Protocol, it lets AI assistants call web-reading tools directly instead of requiring manual copy-and-paste workflows.

The server runs as a hosted remote endpoint at https://agent-web.foomworks.workers.dev/mcp, so there's nothing to install locally. Version v0.2.1 is available under the MIT license, with source code at foomworks/agent-web on GitHub.

agent-web belongs in the AI & Agents category on mcp.site and works with any MCP client, including Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, and Zed. If you maintain agent-web, claim this listing to verify ownership and keep information up to date.

Installation

Settings → Connectors → Add custom connector
Name: agent-web — URL to LLM-ready markdown: a polite, robots-respecting web
URL:  https://agent-web.foomworks.workers.dev/mcp