BeeCrawl is open source. Explore the release notes and self-hosting guide.Read the updates
Web context infrastructure

The Web Context InfrastructureYour AI Can Trust

Search, scrape, crawl, and extract clean context from the live web.

01 / Try it nowStart with a question or URL.
Playground ready
live webBeeCrawlagent context
Open sourceSelf-hostableAgent-ready outputFirecrawl-compatible routes
One API for the path from source to context
SearchScrapeCrawlExtract

Web context infrastructure without the black box.

From one page to an entire website, BeeCrawl gives your agents clean, structured web context through an open and familiar API.

01

Own the stack

Inspect the code, self-host the infrastructure, and keep control of your data, providers, and deployment.

Open source
02

Move without rebuilding

Point supported Firecrawl workflows at BeeCrawl and keep the integration your application already understands.

Compatible
03

Useful output by default

Turn web pages into clean Markdown, links, metadata, and structured JSON ready for agents, RAG, and automation.

Agent-ready
04

Fast path, browser fallback

A lean Rust core handles everyday extraction, while browser rendering steps in for dynamic websites.

Architecture

Every request moves one step closer to an answer.

Use one operation or compose the whole path. Each endpoint returns a useful building block for the system you are already building.

02/v2/scrape

Make a page useful

Scrape

Convert a webpage into clean Markdown, HTML, links, and metadata ready for agents and retrieval pipelines.
URL in. Agent-ready content out.Details
View request
curl -X POST https://api.beecrawl.dev/v2/scrape \
  -H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","formats":["markdown","links"]}'
03/v2/crawl

Traverse a whole site

Crawl

Follow a website with depth, page limits, and browser rendering under your control.
A starting URL in. A site map out.Details
View request
curl -X POST https://api.beecrawl.dev/v2/crawl \
  -H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","limit":100,"maxDepth":2}'
04/v2/extract

Shape the answer

Extract

Turn unstructured page content into predictable JSON using the exact fields your workflow needs.
A schema in. Structured JSON out.Details
View request
curl -X POST https://api.beecrawl.dev/v2/extract \
  -H "X-Web-Extract-Api-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","schema":{"title":"Page title"}}'

Compare the path, not the promise.

BeeCrawl's performance story should be earned in a public, repeatable run. The protocol is ready before the numbers are.

Benchmark protocol Ready to reproduce
01
Freeze the inputs

Use the same question set, target pages, model, and output contract.

02
Measure the path

Record end-to-end latency, retrieval quality, and extraction correctness.

03
Publish the run

Include runtime conditions, date, raw results, and the commands to reproduce it.

Read the full evaluation method
ComparisonCurrent state
BeeCrawlRun under the shared protocolAwaiting published run
FirecrawlRun under the shared protocolAwaiting matched run

Same dataset. Same model. Same runtime conditions. Any future result belongs beside its methodology.

One request. Useful output.

Bring the web into your application with the SDK or HTTP client you already use.

POST /scrape
request.py
from beecrawl_sdk import BeeCrawlClient

client = BeeCrawlClient(
    api_key="YOUR_API_KEY",
    base_url="https://api.beecrawl.dev",
)
page = client.scrape(
    "https://example.com",
    formats=["markdown", "links"],
    use_browser="auto",
)

print(page["markdown"])
markdown
# Example Domain

Example Domain is for use in illustrative
examples in documents and applications.

## Links

- https://www.iana.org/domains/example

---
source: https://example.com
status: 200
Compatible with your existing stackBrowse SDK documentation

Bring your existing Firecrawl workflow along.

BeeCrawl exposes overlapping v2-compatible routes for supported scrape, map, crawl, extract, and search workflows. Change the base URL, check the supported surface, and keep the integration your application already knows.

POST /v2/scrapePOST /v2/mapPOST /v2/crawlPOST /v2/extractPOST /v2/search
Read the migration notes

Live web context for the work that matters next.

Use the same open foundation for agents, retrieval, research, and infrastructure you control.

01

AI agents

Give research assistants and autonomous workflows fresh, source-backed context before they reason or act.

02

RAG pipelines

Bring live pages into an ingestion path as clean Markdown, links, metadata, and structured records.

03

Research automation

Search, follow, and normalize public sources without building a new extraction pipeline for every site.

04

Self-hosted infrastructure

Inspect the stack, choose your providers, and keep deployment and data decisions close to your team.

Put the open web to work.

Start in the Dashboard Playground, then take the same request into your own stack.

Open Dashboard