service / x402-robots-txt-ai-check-60efdeed

Robots Txt AI Check API

Fetches a domain's robots.txt and parses its Content-Signal and AI-preference directives (search, ai-input, ai-train) into structured JSON plus a plain-English summary of what's allowed — the "can this domain be crawled by an AI agent" check. Run before scraping content or building a RAG index, ahead of the industry's move toward stricter bot-gating defaults later this year.

uncertifiedrail · x402

Call card

https://x402.forgemesh.io/robots-txt-ai-check price $0.005 per call

Your agent can call this

The slug below is exactly what invoke accepts. Addendpoint (a route_cards[].path from service_detail) to pick a route, and body / queryfor the route's inputs.

# MCP (any connected agent) tools/call nitrograph_invoke_service { "service_slug": "x402-robots-txt-ai-check-60efdeed", "body": { /* route inputs */ } } # HTTP curl -X POST https://api.nitrograph.com/v1/invoke \ -H "Authorization: Bearer $NITROGRAPH_API_KEY" -H "content-type: application/json" \ -d '{"service_slug": "x402-robots-txt-ai-check-60efdeed", "endpoint": "<route_cards[].path>", "body": {}}'

Reliability

uptime 30d-

refund rate 7d-

settled calls0

failed calls0

certified since-

Daily probe success ratio, last 30 days. Last probed 2026-08-26.

Try it

Run this service right now, on us - 3 free probes a day, validated and refunded like any paid call. Query params optional.