Global

data-extraction
32 projects tagged “data-extraction” that gained the most stars yesterday
#91 · +459 · 1d ·GitHub

Applicant Profiles Directory - manage profiles, linked devices, scan configs, and WhatsApp connections

Size
21.1 MB
Stars
1.7K
Age
4mo
Commits · 90 Days
#133 · +329 · 1d ·GitHub

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

Creator
firecrawl
Size
181.9 MB
Stars
193.1K
Age
2y
Commits · 90 Days
#150 · +290 · 1d ·GitHub

A highly specialized reverse-engineering pipeline designed to structurally transplant original Vanilla character models into the EverQuest Online Adventures: Frontiers (PS2) game engine, resolving complex Vector Unit 1 (VU1) shader bugs, Emotion Engine (EE) memory overflows…

Size
487.3 MB
Stars
1.1K
Updated
CommitJul 03
Age
4mo
Commits · 90 Days
#193 · +219 · 1d ·GitHub

Python tool for converting files and office documents to Markdown.

Creator
microsoft
Size
5.7 MB
Stars
191.2K
Age
1y
Commits · 90 Days
#209 · +204 · 1d ·GitHub

🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: and follow here for daily tips and tricks:

Creator
D4Vinci
Size
15.5 MB
Stars
88.7K
Age
2y
Commits · 90 Days
#289 · +147 · 1d ·GitHub

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫 | 知乎问答文章|评论爬虫

Creator
NanmiCoder
Size
29.8 MB
Stars
67.3K
Updated
CommitOct 15
Age
3y
Commits · 90 Days
#467 · +83 · 1d ·GitHub

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. Join our Discord

Creator
unclecode
Size
148.1 MB
Stars
85.9K
Age
2y
Commits · 90 Days
#587 · +65 · 1d ·GitHub

ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.).

Size
16.9 MB
Stars
32K
Age
2y
Commits · 90 Days
#619 · +61 · 1d ·GitHub

Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.

Size
2.3 MB
Stars
6.2K
Updated
CommitAug 24
Age
8mo
Commits · 90 Days
#1102 · +30 · 1d ·GitHub

Turn any website into a structured API. Extract, automate, search and monitor the web.

Creator
getmaxun
Size
7.8 MB
Stars
17.9K
Age
2y
Commits · 90 Days
#1195 · +27 · 1d ·GitHub

Scrapy, a fast high-level web crawling & scraping framework for Python.

Creator
scrapy
Size
32.7 MB
Stars
64.9K
Age
16y
Commits · 90 Days
#1623 · +18 · 1d ·GitHub

Convert PDF to markdown + JSON quickly with high accuracy

Creator
datalab-to
Size
20.5 MB
Stars
40.6K
Age
2y
Commits · 90 Days
#1764 · +16 · 1d ·GitHub

Stagehand is the SDK for browser agents. Playwright was built for testing, Stagehand is built for agents. Use familiar APIs, self-healing actions, and network-level security across TypeScript, Python, and Go.

Size
199.8 MB
Stars
25.7K
Age
2y
Commits · 90 Days
#2229 · +12 · 1d ·GitHub

This GitHub repo is a powerhouse collection of scraping APIs for developers that you can start using immediately to build everything from simple automations to full-scale applications.

Creator
cporter202
Size
10.7 MB
Stars
4K
Updated
CommitOct 17
Age
10mo
Commits · 90 Days
#2318 · +11 · 1d ·GitHub

Extract and decrypt browser data, supporting multiple data types, runnable on various operating systems (macOS, Windows, Linux).

Creator
moonD4rk
Size
29.1 MB
Stars
14.6K
Age
6y
Commits · 90 Days
#3130 · +6 · 1d ·GitHub

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

Creator
google
Size
12.8 MB
Stars
39K
Age
1y
Commits · 90 Days
#3507 · +5 · 1d ·GitHub

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models.

Size
231.8 MB
Stars
15.6K
Age
4y
Commits · 90 Days
#3674 · +5 · 1d ·GitHub

1+ no longer caches the raw database encryption key in process memory — only a passphrase remains, from which the real key must be derived. This breaks every existing tool that relies on x' ' memory pattern scanning.

Creator
TANGandXUE
Size
<0.1 MB
Stars
661
Updated
CommitAug 04
Age
5mo
Commits · 90 Days
#3752 · +4 · 1d ·GitHub

Note 1: For Windows, please install torch and torchvision first by following the official instructions here On the pytorch website, be sure to select the right CUDA version you have. If you intend to run on CPU mode only, select CUDA = None.

Creator
JaidedAI
Size
157.8 MB
Stars
30.1K
Age
6y
Commits · 90 Days
#3821 · +4 · 1d ·GitHub

Our managed platform runs both Surya, and variants of our highest accuracy model, Chandra.

Creator
datalab-to
Size
68.1 MB
Stars
21.5K
Age
2y
Commits · 90 Days
#4471 · +3 · 1d ·GitHub

新一代爬虫平台,以图形化方式定义爬虫流程,不写代码即可完成爬虫。

Size
3.3 MB
Stars
11.4K
Age
6y
Commits · 90 Days
#5786 · +1 · 1d ·GitHub
archived

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

Size
4.5 MB
Stars
16.4K
Age
9y
Commits · 90 Days
#5853 · +1 · 1d ·GitHub

goquery brings a syntax and a set of features similar to jQuery to the Go language. It is based on Go's net/html package and the CSS Selector library cascadia.

Size
0.7 MB
Stars
15K
Age
14y
Commits · 90 Days
#6025 · +1 · 1d ·GitHub

The maintainFormat option tries to return the markdown in a consistent format by passing the output of a prior page in as additional context for the next page. This requires the requests to run synchronously, so it's a lot slower.

Creator
getomni-ai
Size
168 MB
Stars
12.2K
Age
2y
Commits · 90 Days
#6141 · +1 · 1d ·GitHub

Works best on machine-generated, rather than scanned, PDFs. six.

Creator
jsvine
Size
21.2 MB
Stars
10.8K
Age
11y
Commits · 90 Days
#6366 · +1 · 1d ·GitHub

Stack blocks visually to build complex browser workflows and execute them via API

Creator
figranium
Size
34.1 MB
Stars
771
Age
9mo
Commits · 90 Days
#6452 · +1 · 1d ·GitHub

本地优先的 Windows 微信个人记忆库:筛选、归档并导出可追溯的聊天上下文

Creator
ikevss
Size
8.6 MB
Stars
349
Updated
CommitAug 25
Age
1mo
Commits · 90 Days
#7001 · +0 · 1d ·GitHub
archived

pyspider [![Build Status]][Travis CI] [![Coverage Status]][Coverage] ========

Creator
binux
Size
4.1 MB
Stars
16.8K
Age
12y
Commits · 90 Days
#7076 · +0 · 1d ·GitHub

GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line retrieval, dot notation paths, iteration, and parsing json lines.

Creator
tidwall
Size
0.5 MB
Stars
15.6K
Updated
CommitOct 05
Age
10y
Commits · 90 Days
#7997 · +0 · 1d ·GitHub

Our managed platform runs improved extraction with higher accuracy than the open weights, plus per-field verification, citations, and confidence scores.

Creator
datalab-to
Size
1.4 MB
Stars
1.2K
Age
4mo
Commits · 90 Days