Crawl specific URLs and index them
One step with Algolia. We can help you get access.
Get something done
Crawl specific URLs and index them
with Algolia
Use Algolia: its Crawler API lets you crawl specific URLs and index them in a single step (POST /1/crawlers/{id}/urls/crawl). capabilities.work can help you get signed up and connected.
- How do I recrawl just a handful of pages that changed?
- What's the rate limit on crawling individual URLs?
Say it to your AI assistant
- Crawl these URLs with crawler {id}: {urls}.
- Recrawl {urls} using crawler {id} and add them to its extra URLs.
For your developer
POST /1/crawlers/{id}/urls/crawl in Crawler API. Full spec on APIs.io.