Check a path against robots.txt rules I hold
One step with Delx. We can help you get access.
Look something up
Check a path against robots.txt rules I hold
with Delx
Use Delx: its Delx Commerce — services for autonomous agents X402 API lets you check a path against robots.txt rules I hold in a single step (POST /api/v1/x402/robots-allow-path). capabilities.work can help you get signed up and connected.
- Given a robots.txt I already have, is a path allowed for my crawler?
- Can I test a specific user agent against robots rules without fetching the site?
Say it to your AI assistant
- Check whether {path} is allowed by these robots rules: {robots_txt}.
- Test {path} for user agent {user_agent} against the robots.txt text {robots_txt}.
For your developer
POST /api/v1/x402/robots-allow-path in Delx Commerce — services for autonomous agents X402 API. Full spec on APIs.io.