Scrapfly for AI agents

Agents read Scrapfly at once. Changes and sends are held for a person's approval by default.

About Scrapfly

Scrapfly is a web scraping API that enables developers to extract data from websites efficiently, offering features like JavaScript rendering, anti-bot protection bypass, and proxy rotation.

What an agent can do in Scrapfly

Every Scrapfly action is classed by the platform as a read, a change or a send. Reads run at once. Changes and sends are held for a person by default.

Runs at once (reads)

Looks things up in Scrapfly without asking first. 8 actions.

  • Extract Structured DataTool to extract structured data from HTML or other content using AI models, LLM prompts, or custom templates. Use when you need to parse web pages or documents into structured JSON data. Supports pred
  • Get Scrapfly Account InformationTool to retrieve Scrapfly account information. Use after authenticating to get API credit balance and usage stats. Returns comprehensive account data including subscription plan, usage statistics, bil
  • Get Crawler StatusTool to get the current status of a crawler including progress, pages crawled, and completion state. Use for polling workflow to monitor crawl progress.
  • Scrapfly ScrapeTool to perform a web scraping request. Use when you need to fetch a page with custom configuration like JS rendering, proxies, and extraction.
  • Scrapfly Scrape POSTTool to scrape web pages using POST method to send data in the request body. Use when you need to scrape endpoints that require POST requests, such as form submissions or APIs that expect data payload
  • Scrape With PUTTool to scrape web pages using PUT method with body payload. Use when the target API requires PUT requests with data in the request body. Forwards PUT request with custom body to the target URL. If no
  • Get Crawler ContentsTool to retrieve extracted content from crawled pages. Supports multiple output formats including markdown, text, HTML, and JSON. Use when you need to access the actual content extracted during a craw
  • Get Crawler URLsTool to retrieve the list of discovered and crawled URLs from a crawler. Use when you need to get all URLs found during a crawl or filter by status to analyze failed URLs with error codes. Supports pa

Held for approval by default (changes and sends)

Held for a person's approval by default. The approval is recorded and the action is sent once, never twice, with 10 seconds to undo. 2 actions.

  • Create Scrapfly CrawlerTool to create a new web crawler to recursively crawl an entire website. Returns a crawler UUID for tracking progress. Use when you need to crawl multiple pages from a website with configurable limits
  • Capture Screenshot Metadata (HEAD)Tool to capture screenshot metadata without downloading the image body. Use this for async screenshot workflows where you need the URL to retrieve the image later. Returns the screenshot URL in respon

Changes wait in Needs you

Changes and sends in Scrapfly are held for a person's approval by default. The approval is recorded, and the action is sent once, never twice, with 10 seconds to undo.

How approvals work

Scrapfly questions

Can an agent change data in Scrapfly on its own?

Not by default. Changes and sends in Scrapfly are held for a person's approval. The approval is recorded, and the action is sent once, never twice, with 10 seconds to undo.

What can an agent read in Scrapfly?

This connector lists 8 read actions. Reads run at once, without waiting for approval.

How does Scrapfly connect?

It connects with API key. Connectors are built and maintained by third-party developers; availability is confirmed during scoping.

Bring Scrapfly to your agents.