Own the stack
Inspect the code, self-host the infrastructure, and keep control of your data, providers, and deployment.
Search, scrape, crawl, and extract clean context from the live web.
// Why BeeCrawl
From one page to an entire website, BeeCrawl gives your agents clean, structured web context through an open and familiar API.
Inspect the code, self-host the infrastructure, and keep control of your data, providers, and deployment.
Point supported Firecrawl workflows at BeeCrawl and keep the integration your application already understands.
Turn web pages into clean Markdown, links, metadata, and structured JSON ready for agents, RAG, and automation.
A lean Rust core handles everyday extraction, while browser rendering steps in for dynamic websites.
// Capabilities
Use one operation or compose the whole path. Each endpoint returns a useful building block for the system you are already building.
Find the right sources
curl -X POST https://api.beecrawl.dev/v2/search \
-H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"query":"web context infrastructure","limit":5}'Make a page useful
curl -X POST https://api.beecrawl.dev/v2/scrape \
-H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","formats":["markdown","links"]}'Traverse a whole site
curl -X POST https://api.beecrawl.dev/v2/crawl \
-H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","limit":100,"maxDepth":2}'Shape the answer
curl -X POST https://api.beecrawl.dev/v2/extract \
-H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","schema":{"title":"Page title"}}'// Evidence
BeeCrawl's performance story should be earned in a public, repeatable run. The protocol is ready before the numbers are.
Use the same question set, target pages, model, and output contract.
Record end-to-end latency, retrieval quality, and extraction correctness.
Include runtime conditions, date, raw results, and the commands to reproduce it.
Same dataset. Same model. Same runtime conditions. Any future result belongs beside its methodology.
// Start in minutes
Bring the web into your application with the SDK or HTTP client you already use.
from beecrawl_sdk import BeeCrawlClient
client = BeeCrawlClient(
api_key="YOUR_API_KEY",
base_url="https://api.beecrawl.dev",
)
page = client.scrape(
"https://example.com",
formats=["markdown", "links"],
use_browser="auto",
)
print(page["markdown"])import { BeeCrawlClient } from "beecrawl-sdk";
const client = new BeeCrawlClient({
apiKey: "YOUR_API_KEY",
baseUrl: "https://api.beecrawl.dev",
});
const page = await client.scrape(
"https://example.com",
{ formats: ["markdown", "links"] },
);
console.log(page.markdown);use beecrawl_sdk::BeeCrawlClient;
use serde_json::json;
#[tokio::main]
async fn main() -> beecrawl_sdk::Result<()> {
let client = BeeCrawlClient::builder(
"https://api.beecrawl.dev",
).api_key("YOUR_API_KEY").build()?;
let page = client.scrape(
"https://example.com",
json!({"formats": ["markdown", "links"]}),
).await?;
println!("{}", page["markdown"]);
Ok(())
}curl -X POST https://api.beecrawl.dev/scrape \
-H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"formats": ["markdown", "links"],
"use_browser": "auto"
}'# Example Domain
Example Domain is for use in illustrative
examples in documents and applications.
## Links
- https://www.iana.org/domains/example
---
source: https://example.com
status: 200// Compatibility
BeeCrawl exposes overlapping v2-compatible routes for supported scrape, map, crawl, extract, and search workflows. Change the base URL, check the supported surface, and keep the integration your application already knows.
// Built for
Use the same open foundation for agents, retrieval, research, and infrastructure you control.
Give research assistants and autonomous workflows fresh, source-backed context before they reason or act.
Bring live pages into an ingestion path as clean Markdown, links, metadata, and structured records.
Search, follow, and normalize public sources without building a new extraction pipeline for every site.
Inspect the stack, choose your providers, and keep deployment and data decisions close to your team.
// Your next request
Start in the Dashboard Playground, then take the same request into your own stack.