cli README

The project's README from github.com/brightdata/cli. As one markdown file: README.md.

Bright Data CLI

Bright Data CLI

Scrape, search, and extract structured web data — directly from your terminal.

npm version node requirement license

Overview

@brightdata/cli is the official npm package for the Bright Data CLI. It installs the brightdata command (with bdata as a shorthand alias) for access to the full Bright Data API surface:

Command What it does
brightdata scrape Scrape any URL — bypasses CAPTCHAs, JS rendering, anti-bot protections
brightdata search Google / Bing / Yandex search with structured JSON output
brightdata discover AI-powered web discovery - find and rank results by intent with optional full-page content
brightdata scraper create Build a Bright Data scraper from a natural-language description using AI
brightdata scraper run Run a Bright Data scraper on a URL and return the data
brightdata scraper heal Fix an existing scraper in place via AI self-healing (stops at an approval gate)
brightdata scraper approve Approve (or reject) a self-healing fix that is awaiting approval
brightdata pipelines Extract structured data from 40+ platforms (Amazon, LinkedIn, TikTok…)
brightdata browser Control a real browser via Bright Data's Scraping Browser — navigate, snapshot, click, type, and more
brightdata zones List and inspect your Bright Data proxy zones
brightdata budget View account balance and per-zone cost & bandwidth
brightdata skill Install Bright Data AI agent skills into your coding agent
brightdata add mcp Add the Bright Data MCP server to Claude Code, Cursor, or Codex
brightdata config Manage CLI configuration
brightdata init Interactive setup wizard

Table of Contents

Installation

Requires Node.js ≥ 20

macOS / Linux

curl -fsSL https://cli.brightdata.com/install.sh | sh

Windows

npm install -g @brightdata/cli

Or install manually on any platform

npm install -g @brightdata/cli

You can also run without installing:

npx --yes --package @brightdata/cli brightdata <command>

Quick Start

# 1. Run the interactive setup wizard
brightdata init

# 2. Scrape a page as markdown
brightdata scrape https://example.com

# 3. Search Google
brightdata search "web scraping best practices"

# 4. Extract a LinkedIn profile
brightdata pipelines linkedin_person_profile "https://linkedin.com/in/username"

# 5. Check your account balance
brightdata budget

# 6. Install the Bright Data MCP server into your coding agent
brightdata add mcp

Authentication

Get your API key from brightdata.com/cp/setting/users.

# Interactive — opens browser, saves key automatically
brightdata login

# Via GitHub CLI — no browser needed (requires gh)
brightdata login --github

# Non-interactive — pass key directly
brightdata login --api-key <your-api-key>

# Environment variable — no login required
export BRIGHTDATA_API_KEY=your-api-key

On first login the CLI checks for required zones (cli_unlocker, cli_browser) and creates them automatically if missing.

# Clear saved credentials
brightdata logout

brightdata add mcp uses the API key stored by brightdata login. It does not currently read BRIGHTDATA_API_KEY or the global --api-key flag, so log in first before using it.

Free Tier

Every new Bright Data account includes a recurring monthly free tier — no credit card or commitment required to start. See the Free Tier docs for full details.

5,000 free credits per month (≈ $7.50), renewing on the 1st of each month. Unused credits don't roll over.

These credits cover the products most CLI commands use:

Product CLI commands Credit cost
Unlocker API scrape 1 credit / request
SERP API search 1 credit / request
Web Scraper API pipelines 1 credit / request
Scraper Studio scraper create / run / heal 1 credit / page load (shared pool, not per record)

Not included in the monthly free credits: proxy products (Datacenter, ISP, Residential) and the Browser API (brightdata browser). These get a separate one-time $2 trial (valid 7 days), plus a $5 bonus when you add a payment method (valid 30 days).

Adding a credit card is a verification step only — you are not charged unless your free credits are exhausted and you have funds deposited. Accounts on custom pay-as-you-go or pre-commit subscription plans don't receive the recurring monthly free credits.

Check your remaining balance any time with brightdata budget.

Commands

init

Interactive setup wizard. The recommended way to get started.

brightdata init

Walks through: API key detection → zone selection → default output format → quick-start examples.

Flag Description
--skip-auth Skip the authentication step
-k, --api-key <key> Provide API key directly

scrape

Scrape any URL using Bright Data's Web Unlocker. Handles CAPTCHAs, JavaScript rendering, and anti-bot protections automatically.

brightdata scrape <url> [options]
Flag Description
-f, --format <fmt> markdown · html · screenshot · json (default: markdown)
--country <code> Geo-target by ISO country code (e.g. us, de, jp)
--zone <name> Web Unlocker zone name
--mobile Use a mobile user agent
--async Submit async job, return a snapshot ID
-o, --output <path> Write output to file
--json / --pretty JSON output (raw / indented)
-k, --api-key <key> Override API key

Examples

# Scrape as markdown (default)
brightdata scrape https://news.ycombinator.com

# Scrape as raw HTML
brightdata scrape https://example.com -f html

# US geo-targeting, save to file
brightdata scrape https://amazon.com -f json --country us -o product.json

# Pipe to a markdown viewer
brightdata scrape https://docs.github.com | glow -

# Async — returns a snapshot ID you can poll with `status`
brightdata scrape https://example.com --async

Search Google, Bing, or Yandex via Bright Data's SERP API. Google results include structured data (organic results, ads, people-also-ask, related searches).

brightdata search <query> [options]
Flag Description
--engine <name> google · bing · yandex (default: google)
--country <code> Localized results (e.g. us, de)
--language <code> Language code (e.g. en, fr)
--page <n> Page number, 0-indexed (default: 0)
--type <type> web · news · images · shopping (default: web)
--device <type> desktop · mobile
--zone <name> SERP zone name
-o, --output <path> Write output to file
--json / --pretty JSON output (raw / indented)
-k, --api-key <key> Override API key

Examples

# Formatted table output (default)
brightdata search "typescript best practices"

# German localized results
brightdata search "restaurants berlin" --country de --language de

# News search
brightdata search "AI regulation" --type news

# Page 2 of results
brightdata search "web scraping" --page 1

# Extract just the URLs
brightdata search "open source scraping" --json | jq -r '.organic[].link'

# Search Bing
brightdata search "bright data pricing" --engine bing

discover

AI-powered web discovery. Submit a query with optional intent, and Bright Data finds, ranks, and optionally extracts full-page content for each result.

brightdata discover <query> [options]
Flag Description
--intent <text> AI intent to evaluate and rank result relevance
--country <code> ISO country code (default: US)
--city <name> City for localized results (e.g. "New York")
--language <code> Language code (default: en)
--num-results <n> Number of results to return
--filter-keywords <words> Comma-separated keywords that must appear in results
--include-content Include full page content in each result
--no-remove-duplicates Keep duplicate results
--start-date <date> Only content updated from date (YYYY-MM-DD)
--end-date <date> Only content updated until date (YYYY-MM-DD)
--timeout <seconds> Polling timeout (default: 600)
-o, --output <path> Write output to file
--json / --pretty JSON output (raw / indented)
-k, --api-key <key> Override API key

Examples

# Basic discovery — table output
brightdata discover "AI trends"

# With AI intent for relevance ranking
brightdata discover "AI trends" \
  --intent "Prioritize institutional reports for VC research"

# Include full page content as markdown
brightdata discover "AI trends" --include-content --num-results 5

# Geo-targeted with date range
brightdata discover "best restaurants" --country US --city "New York" \
  --start-date 2025-01-01 --end-date 2025-12-31

# Filter results by keywords
brightdata discover "generative AI SaaS" --filter-keywords "revenue,SaaS"

# JSON output to file
brightdata discover "AI trends" --num-results 10 --pretty -o results.json

# Pipe-friendly — redirected stdout outputs JSON automatically
brightdata discover "AI trends" --include-content --num-results 3 > results.json

scraper create

Build a Bright Data scraper from a natural-language description using AI.

brightdata scraper create <url> <description> [options]
Flag Description
--name <name> Scraper template name (default: cli-scraper-<timestamp>)
--deliver-webhook <url> Webhook URL for the deliver stub (default: https://example.com/webhook)
--timeout <seconds> Polling timeout in seconds (default: 600)
--max-retries <n> Max retries on the AI-Flow concurrent-job cap 429 (default: 4). See below.
--no-retry Fail immediately on 429 instead of waiting. Same as --max-retries 0.
-o, --output <path> Write the JSON envelope to a file (see below)
--json / --pretty JSON output (raw / indented)
--legacy-output Write the pre-v0.3 bare AI-progress payload to -o instead of the envelope. Migration only.
--timing Show request timing
-k, --api-key <key> Override API key

Note: The scraper is created with a placeholder webhook delivery target (https://example.com/webhook). You can reconfigure the actual delivery endpoint in the Bright Data web UI after creation.

Output envelope (-o create.json)

Every termination path — success or failure — writes the same JSON envelope shape:

{
  "collector_id":    "c_mp7x8a9b2c0d1e2f",
  "name":            "my-product-scraper",
  "status":          "done",
  "completed_steps": ["prepare_intent_analyzer", "planner", "..."],
  "view_url":        "https://brightdata.com/cp/scrapers/c_mp7x8a9b2c0d1e2f",
  "created_at":      "2026-05-18T07:28:30Z"
}

On failure paths the envelope adds an error field and the status reflects the failure category (ai_trigger_failed, failed, poll_failed). The collector_id and view_url are still present so you can recover or inspect the half-built scraper.

This makes the documented chain in recipes.md work as written:

brightdata scraper create https://example.com/product/1 "..." \
    -o create.json
COLLECTOR_ID=$(jq -r '.collector_id' create.json)
brightdata scraper run "$COLLECTOR_ID" https://example.com/product/2

The file format follows the -o extension, so .json is written compact (ideal for jq). Use --pretty for indented JSON on stdout when you omit -o.

Use --legacy-output if you have an existing script that depended on the pre-v0.3 bare-progress shape; the flag is supported for one minor version while you migrate.

Concurrent-job cap & auto-backoff

The Bright Data AI Flow caps concurrent scraper create generations per account (currently 3). If you exceed it, the API returns 429 Cannot run more than N jobs in parallel. The CLI handles this automatically: it waits with exponential backoff + jitter and retries up to --max-retries times (default 4). During the wait the CLI prints status lines so you know it isn't hung:

Triggering AI generation...
Hit AI-Flow concurrent-job cap (429). Waiting 32s before retry 1/4...
Hit AI-Flow concurrent-job cap (429). Waiting 67s before retry 2/4...
Generating scraper...

If the cap is still hit after all retries, the CLI exits with a stderr note pointing at the half-built collector's dashboard URL so you can inspect or delete it manually (Bright Data does not yet expose programmatic collector deletion).

Use --no-retry if you want the old fail-fast behavior — typically for scripts that prefer to handle backoff themselves.

Examples

# Build a scraper for a product page
brightdata scraper create https://example.com/product/1 \
    "Extract title, price, and image URL from this product page"

# Name the scraper and save the envelope to a file
brightdata scraper create https://example.com/product/1 \
    "Extract title, price, and image URL from this product page" \
    --name my-product-scraper -o create.json

# Capture the collector_id for chaining
COLLECTOR_ID=$(jq -r '.collector_id' create.json)

# Fan out 10 parallel creates — the CLI serialises automatically via 429 backoff
for url in $(cat urls.txt); do
    brightdata scraper create "$url" "Extract title, price, ..." \
        --name "scraper-$(basename $url)" &
done; wait

# Disable the auto-backoff (fail fast on 429)
brightdata scraper create https://example.com/product/1 \
    "Extract title, price, and image URL from this product page" \
    --no-retry

# Use a custom webhook delivery URL
brightdata scraper create https://example.com/product/1 \
    "Extract title, price, and image URL from this product page" \
    --deliver-webhook https://my-app.com/ingest

scraper run

Run a scraper (built with scraper create or in the web UI) against one or more URLs and get the extracted data.

brightdata scraper run <collector_id> [url] [options]

Provide URLs in exactly one of three ways:

Flag Description
--urls <list> Comma-separated list of URLs (multi-URL batch path)
--input-file <path> File with URLs (txt one-per-line, or JSON array)
--sync Use the synchronous /dca/crawl endpoint (single-URL only, server-side cap of 25–50s)
--sync-timeout <seconds> Sync-mode server timeout, 25–50 (default: 50)
--timeout <seconds> Polling timeout (default: 600 single-URL, 3600 batch)
--name <name> Human-readable job name
--version <version> Scraper version (e.g. dev)
-o, --output <path> Write output to file
--json / --pretty JSON output (raw / indented)
--timing Show request timing
-k, --api-key <key> Override API key

Routing

If a single URL expands to more pages than the realtime job limit allows (paginated listings, infinite scroll), the CLI automatically falls back to the batch endpoint and prints a one-line notice. No flag required.

Examples

# Default: async + poll until results arrive
brightdata scraper run c_mp3tuab31lswoxvpws https://www.amazon.com/dp/B08N5WRWNW

# Save pretty-printed results to a file
brightdata scraper run c_mp3tuab31lswoxvpws https://www.amazon.com/dp/B08N5WRWNW \
    --pretty -o product.json

# Sync mode for fast pages
brightdata scraper run c_mp3tuab31lswoxvpws https://example.com/p/1 --sync

# Sync with a shorter server timeout and a job name
brightdata scraper run c_mp3tuab31lswoxvpws https://example.com/p/1 \
    --sync --sync-timeout 30 --name first-test

# Multi-URL batch — one API call, one snapshot, one merged result array
brightdata scraper run c_mp3tuab31lswoxvpws \
    --urls "https://example.com/p/1,https://example.com/p/2,https://example.com/p/3" \
    --pretty -o products.json

# Multi-URL from a file (one URL per line; # comments and blank lines skipped)
brightdata scraper run c_mp3tuab31lswoxvpws --input-file urls.txt -o products.json

# Multi-URL from a JSON array
echo '["https://example.com/p/1","https://example.com/p/2"]' > urls.json
brightdata scraper run c_mp3tuab31lswoxvpws --input-file urls.json

scraper heal

Fix an existing scraper in place when it ran but returned wrong, empty, or partial data. The collector_id stays the same — the scraper is improved, not replaced. This is the maintenance twin of scraper create: it triggers Bright Data's AI self-healing flow (POST /dca/collectors/{id}/refactor_template), then polls progress.

brightdata scraper heal <collector_id> "<prompt>" [options]

You are the detector. The CLI never decides on its own that a scraper is broken — you inspect the run output and decide. The <prompt> is required (max 1000 chars); name exactly what is wrong and what the correct output should be. Vague prompts produce vague heals.

Flag Description
--url <url> Verify target woven into the success next_step hint (not sent to the h

cli (brightdata/cli)

Searching Database…

6.2K ·GitHub
Cli
这是什么？这是 Bright Data 官方的命令行工具（CLI），可以直接在终端里通过 Bright Data 的 API 抓取、搜索网页数据，并将其提取为结构化数据。
这是独立的命令行工具，还是需要 Bright Data 的其他服务？它是一个 npm 包，会安装 `brightdata` 命令（也可以用简写 `bdata`），通过 Bright Data 的 API 产品（如 Web Unlocker、SERP API 和 Scraping Browser）工作。它是一个客户端，不是独立的爬虫。
我该如何安装和设置？可以通过 npm 安装（需要 Node.js ≥20），支持 macOS、Linux 和 Windows，也可以不安装直接运行。然后运行 `brightdata init`，它会启动交互式向导，引导你完成 API 密钥检测、区域选择和输出格式设置。
使用它需要 Bright Data 的 API 密钥吗？需要。你需要从 Bright Data 控制台获取 API 密钥并登录；首次登录时，命令行工具会检查所需的代理区域（`cliunlocker`、`clibrowser`），如果缺少则自动创建。
AI 爬虫命令有什么需要注意的地方吗？有。`scraper create` 每个账户的 AI 生成任务并发数上限为 3 个；超出时，命令行工具会逐步拉长等待时间后自动重试，默认最多重试 4 次。另外，`add mcp` 只有在你已登录的情况下才能使用，因为它不会从环境变量或命令行参数中读取 API 密钥。
在需要付费之前有免费额度吗？有。每个新账户每月可获得 5,000 个免费积分（约合 7.50 美元），可用于大多数命令行命令，如 scrape、search 和 pipelines，每次请求消耗 1 个积分。积分不会结转到下个月，代理和浏览器产品是单独计费的。
自愈爬虫功能是如何工作的？`scraper heal` 命令让你描述现有爬虫输出中出现的问题，然后触发 Bright Data 的 AI 自愈流程，在原地修复（采集器 ID 保持不变）。修复结果可能需要通过 `scraper approve'` 手动批准。
我能把它和 Claude Code、Cursor 这类编程智能体集成吗？可以。`brightdata skill` 会把 Bright Data 的 AI 智能体技能安装到你的编程智能体中，`brightdata add mcp` 则可以把 Bright Data MCP 服务器添加到 Claude Code、Cursor 或 Codex 中。
pipelines 命令可以从哪些数据源提取数据？`pipelines` 命令可以从 40 多个平台提取结构化数据，包括 Amazon、LinkedIn 和 TikTok。
What are similar projects?oc, bb-browser, reverse-api-engineer, Agent-Reach, cli-printing-press, captcha-api, agent-browser, testsprite-cli
Which topics is it in?scraping, search-engine, web-scraping, browser-automation, cli-tool, proxy, sdk, command-line-tool, mcp, npm, api, terminal
Creator
brightdata
Size
0.6 MB
Stars
6.2K
Updated
CommitOct 13Releasev0.4.0Oct 12
Age
7mo
Commits · 90 Days