data-engineer-handbook README

The project's README from github.com/DataExpert-io/data-engineer-handbook. As one markdown file: README.md.

The Data Engineering Handbook

DataExpert-io%2Fdata-engineer-handbook | Trendshift

This repo has all the resources you need to become an amazing data engineer!

Join the free Databricks AI boot camp on August 3rd here and signup for Databricks Free Edition here

Getting started

If you are new to data engineering, start by following this 2024 breaking into data engineering roadmap

If you are here for the 4-week free beginner boot camp you can check out:

If you are here for the 6-week free intermediate boot camp you can check out

For more applied learning:

Resources

Great list of over 25 books

Top 3 must read books are:

Great list of over 10 communities to join:

Top must-join communities for DE:

Top must-join communities for ML:

Companies:

Data Engineering blogs of companies:

Data Engineering Whitepapers:

Social Media Accounts

Here's the mostly comprehensive list of data engineering creators: (You have to have at least 5k followers somewhere to be added!)

YouTube

Name YouTube Channel Follower Count
ByteByteGo ByteByteGo 1,000,000+
Data with Baraa Data with Baraa 195,000+
Zach Wilson Data with Zach 150,000+
Shashank Mishra E-learning Bridge 100,000+
Seattle Data Guy Seattle Data Guy 100,000+
TrendyTech TrendyTech 100,000+
Darshil Parmar Darshil Parmar 100,000+
Andreas Kretz Andreas Kretz 100,000+
The Ravit Show The Ravit Show 100,000+
Guy in a Cube Guy in a Cube 100,000+
Adam Marczak Adam Marczak 100,000+
nullQueries nullQueries 100,000+
TECHTFQ by Thoufiq TECHTFQ by Thoufiq 100,000+
SQLBI SQLBI 100,000+
Alex Freberg Alex The Analyst 100,000+
Ankur Ranjan Big Data Show 100,000+
Prashanth Kumar Pandey ScholarNest 77,000+
ITVersity ITVersity 67,000+
Soumil Shah Soumil Shah 50,000
Ansh Lamba Ansh Lamba 18,000+
Azure Lib Azure Lib 10,000+
Advancing Analytics Advancing Analytics 10,000+
Kahan Data Solutions Kahan Data Solutions 10,000+
Ankit Bansal Ankit Bansal 10,000+
Mr. K Talks Tech Mr. K Talks Tech 10,000+
Samuel Focht Python Basics 10,000+
Mehdi Ouazza Mehdio DataTV 3,000+
Alex Merced Alex Merced Data N/A
John Kutay John Kutay N/A
Emil Kaminski Databricks For Professionals 5,000+

LinkedIn

Name LinkedIn Profile Follower Count
Zach Wilson Zach Wilson 400,000+
Chip Huyen Chip Huyen 250,000+
Shashank Mishra Shashank Mishra 100,000+
Seattle Data Guy Ben Rogojan 100,000+
TrendyTech Sumit Mittal 100,000+
Darshil Parmar Darshil Parmar 100,000+
Andreas Kretz Andreas Kretz 100,000+
ByteByteGo (Alex Xu) Alex Xu 100,000+
Azure Lib (Deepak Goyal) Deepak Goyal 100,000+
Alex Freberg Alex Freberg 100,000+
SQLBI (Marco Russo) Marco Russo 50,000+
Ankit Bansal Ankit Bansal 50,000+
Marc Lamberti Marc Lamberti 50,000+
Ankur Ranjan Ankur Ranjan 48,000+
ITVersity (Durga Gadiraju) Durga Gadiraju 48,000+
Prashanth Kumar Pandey Prashanth Kumar Pandey 37,000+
Alex Merced Alex Merced 30,000+
Ijaz Ali Ijaz Ali 24,000+
Mehdi Ouazza Mehdi Ouazza 20,000+
Ananth Packkildurai Ananth Packkildurai 18,000+
Ansh Lamba Ansh Lamba 13,000+
Manojkumar Vadivel Manojkumar Vadivel 12,000+
Advancing Analytics Simon Whiteley 10,000+
Li Yin Li Yin 10,000+
Jaco van Gelder Jaco van Gelder 10,000+
Joseph Machado Joseph Machado 10,000+
Eric Roby Eric Roby 10,000+
Simon Späti Simon Späti 10,000+
Constantin Lungu Constantin Lungu 10,000+
Lakshmi Sontenam Lakshmi Sontenam 9,500+
Dani Pálma Daniel Pálma 9,000+
Soumil Shah Soumil Shah 8,000+
Arnaud Milleker Arnaud Milleker 7,000+
Dimitri Visnadi Dimitri Visnadi 7,000+
Lenny Lenny A 6,000+
Dipankar Mazumdar Dipankar Mazumdar 5,000+
Daniel Ciocirlan Daniel Ciocirlan 5,000+
Hugo Lu Hugo Lu 5,000+
Tobias Macey Tobias Macey 5,000+
Marcos Ortiz Marcos Ortiz 5,000+
Julien Hurault Julien Hurault 5,000+
John Kutay John Kutay 5,000+
Hassaan Akbar Hassaan Akbar 5,000+
Subhankar Subhankar 5,000+
Nitin Nitin N/A
Hassaan Hassaan 5000+
Javier de la Torre [Ja

data-engineer-handbook (DataExpert-io/data-engineer-handbook)

Searching Database…

这是什么?这是一份精选的数据工程学习资源手册,包括学习路线图、训练营材料、书籍、社区、公司工具清单、博客、白皮书,以及数据工程内容创作者目录。
这是一门课程、一份阅读清单,还是别的什么?它主要是一份精选的阅读清单,同时也是其他资源(训练营、书籍、博客、白皮书、工具)的索引,并不是一门自带课程内容的完整课程。
初学者应该从哪里开始?可以从 2024 年的 “breaking into data engineering” 学习路线图开始,也可以直接进入免费的 4 周初学者训练营或 6 周中级训练营的入门介绍和软件安装指南。
跟着学需要购买付费工具或注册账号吗?不需要任何付费工具。手册链接了免费训练营(包括免费的 Databricks AI 训练营)以及免费的社区 Slack/Discord 服务器。
这份清单是经过筛选的,还是什么都会收录?社交媒体部分有一定的收录门槛,创作者需要在某个平台至少拥有 5,000 名粉丝才能被列入。除此之外,它是对许多工具和公司的广泛汇总,文中没有说明严格的审核标准。
这份手册主要面向谁?面向有志成为或正在从事数据工程工作的人,包括按路线图和训练营学习的初学者,以及准备面试、或想通过书籍、订阅通讯和社区学习的人。
包含哪些动手实践内容?仓库中有一个单独的 projects 板块提供动手示例,此外还有偏理论的书籍、白皮书和博客链接。
推荐书籍涵盖哪些主题?首选书目包括 'Fundamentals of Data Engineering'、'Designing Data-Intensive Applications' 和 'Designing Machine Learning Systems',既涵盖数据工程核心内容,也涵盖机器学习系统设计。
这份手册收录了哪些类型的工具?它列出了多个领域的公司和工具:编排(Airflow、Dagster、Mage)、数据湖/数据仓库(Snowflake、Databricks、Iceberg)、数据质量(dbt、Great Expectations)、数据集成(Fivetran、Airbyte),以及现代 OLAP 引擎(ClickHouse、DuckDB、StarRocks)。
Size
60.9 MB
Stars
44.5K
Updated
CommitAug 03
Age
2y
Commits · 90 Days