Global

This is a repo with links to everything you'd ever want to learn about data engineering

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.

In this course, we'll go from experimentation (design + development) to production (deployment + iteration). We'll do this iteratively by motivating the components that will enable us to build a reliable production system.

The open-source tool for building high-quality datasets and computer vision models

> ydata-profiling is now fg-data-profiling. This package has been renamed to fg-data-profiling. Please follow the Migration Guide as soon as possible — the old package will no longer receive updates or bug fixes.

Its powerful technical tools start with Expectations: expressive and extensible unit tests for your data. Expectations foster collaboration by giving teams a common language to express data quality tests in an intuitive way.

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Open source SDK for scalable multimodal data pipelines in robotics and physical AI

End-to-end streaming data platform on AWS: Kafka → Iceberg lake → Kimball model → Athena, orchestrated with Airflow 3. Terraform, Glue PySpark, dbt.

数据分析与 Excel 全流程 skill：体检脏表、清洗、对齐需求、分析、对账、交付。让 AI 算出来的数字经得起追问。跨 agent 通用，依赖仅 openpyxl。

4 core scan, test, diff, and direct OOXML repair workflows work locally without Microsoft Excel, LibreOffice, an AI key, or a cloud service. Those normal workflows do not calculate formulas, execute VBA, open embedded objects, or fetch external links. xlsm files remain read-only.