Crawl and extract a website
Crawl a website and extract page content with path/domain filters. Needs url. Served by Tavily.
Inputs
- URL
- The root URL to begin the crawlRequired · url
- Instructions
- Natural language instructions for the crawlerOptional · instructions
- Chunks per source
- Chunks are short content snippets (about 800 characters each) pulled directly from the sourceOptional · chunks_per_source
- Max depth
- Max depth of the crawlOptional · max_depth
- Max breadth
- Max number of links to follow per level of the tree (i.e., per page)Optional · max_breadth
- Limit
- Total number of links the crawler will process before stoppingOptional · limit
- Select paths
- Regex patterns to select only URLs with specific path patterns (e.g., `/docs/.*`, `/api/v1.*`)Optional · select_paths
- Select domains
- Regex patterns to select crawling to specific domains or subdomains (e.g., `^docs\.example\.com$`)Optional · select_domains
- Exclude paths
- Regex patterns to exclude URLs with specific path patterns (e.g., `/private/.*`, `/admin/.*`)Optional · exclude_paths
- Exclude domains
- Regex patterns to exclude specific domains or subdomains from crawling (e.g., `^private\.example\.com$`)Optional · exclude_domains
- Allow external
- Whether to include external domain links in the final results listOptional · allow_external
- Include images
- Whether to include images in the crawl resultsOptional · include_images
- Extract depth
- Advanced extraction retrieves more data, including tables and embedded content, with higher success but may increase… (basic or advanced)Optional · extract_depth
- Format
- The format of the extracted web page content (markdown or text)Optional · format
- Include favicon
- Whether to include the favicon URL for each resultOptional · include_favicon
- Timeout
- Maximum time in seconds to wait for the crawl operation before timing outOptional · timeout
- Include usage
- Credit usage is always requested for actual-cost billingOptional · include_usage
Pricing
- Price
- Usage based, at the cost the provider reports for the call
- Billing
- Billed at the cost the provider reports for the call, never above $0.48. Your agent sees the maximum before running.
- Price checked
- 3 November 2026
Call
Your agent reads the schema and the price with inspect_tool before it runs the tool.
- MCP call
- cURL command
- TypeScript
- Tool ID
Alternatives
- Create a Firecrawl monitorFirecrawlUsage based
- Delete a Firecrawl monitorFirecrawlFree
- Firecrawl crawl errorsFirecrawlFree
- Firecrawl monitor check detailsFirecrawlFree
- Firecrawl monitor checksFirecrawlFree
- Firecrawl monitor detailsFirecrawlFree
- Firecrawl website crawlFirecrawlUsage based
- Firecrawl website mapFirecrawlfrom $0.0025
- Compare all 12
Connect your agent to 3,000+ tools for free
Sign in with Google or GitHub. No card required.