Crawl a site and scrape many pages to Markdown
One step with Context.dev. We can help you get access.
Look something up
Crawl a site and scrape many pages to Markdown
with Context.dev
Works with AI assistants
Use Context.dev: its Context Web Scraping API lets you crawl a site and scrape many pages to Markdown in a single step (POST /web/crawl). Context.dev also has an MCP server, so an AI assistant like Claude or ChatGPT can do it for you. capabilities.work can help you get signed up and connected.
- Can I crawl a whole docs site from one starting URL and get every page back as Markdown?
- What's the maximum number of pages a multi-page crawl will scrape?
Say it to your AI assistant
- Crawl starting at {url} and return every page as Markdown.
- Crawl {url} up to {maxPages} pages and depth {maxDepth}, scraping each to Markdown.
For your developer
POST /web/crawl in Context Web Scraping API. Full spec on APIs.io.