#39 · +488 · 1d ·GitHub
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Global
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust
🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: and follow here for daily tips and tricks:
#603 · +48 · 1d ·GitHub
ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.).
#1007 · +27 · 1d ·GitHub
Extract Google Maps business leads, emails, reviews, phone numbers, websites, ratings, coordinates, and more with a free open-source CLI, Web UI, REST API, and optional self-hosted platform.
Scrapy, a fast high-level web crawling & scraping framework for Python.
> Puppeteer is a JavaScript library which provides a high-level API to control > Chrome or Firefox over the > DevTools Protocol or WebDriver BiDi. > Puppeteer runs in the headless (no visible UI) by default
Turn any website into a structured API. Extract, automate, search and monitor the web.
Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.
Photon Wiki • How To Use • Compatibility • Photon Library • Contribution • Roadmap
100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.
Turns websites into a command line interface for AI agents. oc open fetches a page and hands back a compact, numbered view instead of raw HTML or a screenshot, so agents like Claude Code, Codex, and Antigravity can browse without burning tokens.
#8029 · +0 · 1d ·GitHub
TikTok X-Gnarly, X-Bogus, and X-Dynosaur signature generator for Web API requests. 316). Python implementation with ChaCha20 and RC4 encryption, dynamic round selection, and custom Base64 encoding.