Global

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫

ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.).

🚀 自托管的抖音、TikTok 数据采集与无水印视频下载 API，支持 AI agent 通过 MCP 调用与离线语音转文字，Docker 一键部署。| Self-hosted TikTok & Douyin scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text for downloaded videos.

Extract Google Maps business leads, emails, reviews, phone numbers, websites, ratings, coordinates, and more with a free open-source CLI, Web UI, REST API, and optional self-hosted platform.

job-ops: DevOps principles applied to job hunting. A self-hosted pipeline to track, analyze, and assist your application process

AI 时代的私人影库

Command-line program to download image galleries and collections from several image hosting sites

抖音 / TikTok 平台作品下载/数据采集工具

Scrapy, a fast high-level web crawling & scraping framework for Python.

Turn any website into a structured API. Extract, automate, search and monitor the web.

Download pictures (or videos) along with their captions and other metadata from Instagram.

Create agents that monitor and act on your behalf. Your agents are standing by!

参数说明： | 参数 | 说明 | 示例 | | :--- | :--- | :--- | | --xd | 指定学段 | 小学（六三学制）、初中（六三学制）、小学（五四学制）、初中（五·四学制）、高中、培智学校、聋校、盲校（盲文版）、盲校（低视力版） | | --xk | 指定学科 | 语文、数学、英语、物理、化学、历史、道德与法治 等 | | --nj | 指定年级/册次 | 一年级、二年级 ... /downloads | | --yes, -y | 跳过确认提示直接开始下载 | 开启免交互 |

A completely free (including for commercial use and secondary development) visual browser automation test/data collection/crawler software, which can be used to design and execute tasks in a code-free visual way.

This GitHub repo is a powerhouse collection of APIs you can start using immediately to build everything from simple automations to full-scale applications. One of the most valuable API lists on GitHub—period. 💪

This project has not provided a description yet.

👾 Fast and simple video download library and CLI tool written in Go

js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs.

MangoProxy is a Residential, ISP, Mobile, and Datacenter proxy service designed for professional tasks where stability, speed, and anonymity matter.

Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

A browser extension for automating your browser by connecting blocks

Node.js（app.js）每隔1s读取一次file.txt文件，并将读取结果通过socket.io推送给客户端（index.html）。

This GitHub repo is a powerhouse collection of scraping APIs for developers that you can start using immediately to build everything from simple automations to full-scale applications.

Lightning Fast and Elegant Scraping Framework for Gophers

Crawl a site to generate knowledge files to create your own custom GPT from a URL

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.