Global

Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: and follow here for daily tips and tricks:

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫

Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

These skills help agents find outlier posts, mine comments, summarize transcripts, tear down competitors, analyze ad libraries, and turn public social data into useful business artifacts.

ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.).

Extract Google Maps business leads, emails, reviews, phone numbers, websites, ratings, coordinates, and more with a free open-source CLI, Web UI, REST API, and optional self-hosted platform.

Python - 100天从新手到大师

Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.

IPcook provides reliable residential proxies for online research, username discovery, and public data collection workflows. 2/GB.

Camoufox is an open source anti-detect browser built for webscraping & AI agents. 🦊

Scrapy, a fast high-level web crawling & scraping framework for Python.

FlareSolverr is a proxy server to bypass Cloudflare and DDoS-GUARD protection.

NOTE] > This is a demo version currently supporting Chinese only. A full-featured version with better customization and English content support will be released later.

This GitHub repo is a powerhouse collection of scraping APIs for developers that you can start using immediately to build everything from simple automations to full-scale applications.

Convert any URL to an LLM-friendly input with a simple prefix

js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs.

> Turn websites into APIs. Browse (or let an agent browse), and get a clean, typed client for the endpoints the site actually uses.

🐙 CDP Mode bypasses bot-detection with Chromium-based browsers. 🎭 Stealthy Playwright Mode extends CDP Mode's stealth to Playwright. for the main framework.

) into tracker-site-specific HTTP queries, parses the HTML or JSON response, and sends results back to the requesting software.

Lightning Fast and Elegant Scraping Framework for Gophers

一款在线的 微信公众号文章批量下载 工具，支持导出阅读量与评论数据，无需搭建任何环境，可通过 在线网站 使用，支持 docker 私有化部署和 Cloudflare 部署。 支持下载各种文件格式，其中 HTML 格式可100%还原文章排版与样式。

A browser extension for automating your browser by connecting blocks

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

The fast, flexible, and elegant library for parsing and manipulating HTML and XML.

   想自建爬虫？接入 Bright Data MCP Server，让 Claude、Cursor、Windsurf 等 AI 助手直接实时抓取网页——自动破解验证码、绕过地区限制。Scraper Studio 支持 AI 一键生成或 JS 代码定制，全托管基础设施运行，无需自购代理、无需搭服务器，分钟级上线。所有产品底层均由同一套顶级代理网络驱动。

You-Get is a tiny command-line utility to download media contents (videos, audios, images) from the Web, in case there is no other handy way to do it.

js. In general, the goal of the project is to emulate enough of a subset of a web browser to be useful for testing and scraping real-world web applications.

g. scraping the web) as simple and intuitive as possible.