applied-ml README

The project's README from github.com/eugeneyan/applied-ml. As one markdown file: README.md.

applied-ml

Curated papers, articles, and blogs on data science & machine learning in production. ⚙️

contributions welcome Summaries HitCount

Figuring out how to implement your ML project? Learn how other organizations did it:

P.S., Want a summary of ML advancements? 👉ml-surveys

P.P.S, Looking for guides and interviews on applying ML? 👉applyingML

Table of Contents

  1. Data Quality
  2. Data Engineering
  3. Data Discovery
  4. Feature Stores
  5. Classification
  6. Regression
  7. Forecasting
  8. Recommendation
  9. Search & Ranking
  10. Embeddings
  11. Natural Language Processing
  12. Sequence Modelling
  13. Computer Vision
  14. Reinforcement Learning
  15. Anomaly Detection
  16. Graph
  17. Optimization
  18. Information Extraction
  19. Weak Supervision
  20. Generation
  21. Audio
  22. Privacy-Preserving Machine Learning
  23. Validation and A/B Testing
  24. Model Management
  25. Efficiency
  26. Ethics
  27. Infra
  28. MLOps Platforms
  29. Practices
  30. Team Structure
  31. Fails

Data Quality

  1. Reliable and Scalable Data Ingestion at Airbnb Airbnb 2016
  2. Monitoring Data Quality at Scale with Statistical Modeling Uber 2017
  3. Data Management Challenges in Production Machine Learning (Paper) Google 2017
  4. Automating Large-Scale Data Quality Verification (Paper)Amazon 2018
  5. Meet Hodor — Gojek’s Upstream Data Quality Tool Gojek 2019
  6. Data Validation for Machine Learning (Paper) Google 2019
  7. An Approach to Data Quality for Netflix Personalization Systems Netflix 2020
  8. Improving Accuracy By Certainty Estimation of Human Decisions, Labels, and Raters (Paper) Facebook 2020

Data Engineering

  1. Zipline: Airbnb’s Machine Learning Data Management Platform Airbnb 2018
  2. Sputnik: Airbnb’s Apache Spark Framework for Data Engineering Airbnb 2020
  3. Unbundling Data Science Workflows with Metaflow and AWS Step Functions Netflix 2020
  4. How DoorDash is Scaling its Data Platform to Delight Customers and Meet Growing Demand DoorDash 2020
  5. Revolutionizing Money Movements at Scale with Strong Data Consistency Uber 2020
  6. Zipline - A Declarative Feature Engineering Framework Airbnb 2020
  7. Automating Data Protection at Scale, Part 1 (Part 2) Airbnb 2021
  8. Real-time Data Infrastructure at Uber Uber 2021
  9. Introducing Fabricator: A Declarative Feature Engineering Framework DoorDash 2022
  10. Functions & DAGs: introducing Hamilton, a microframework for dataframe generation Stitch Fix 2021
  11. Optimizing Pinterest’s Data Ingestion Stack: Findings and Learnings Pinterest 2022
  12. Lessons Learned From Running Apache Airflow at Scale Shopify 2022
  13. Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training Meta 2022
  14. Data Mesh — A Data Movement and Processing Platform @ Netflix Netflix 2022
  15. Building Scalable Real Time Event Processing with Kafka and Flink DoorDash 2022

Data Discovery

  1. Apache Atlas: Data Goverance and Metadata Framework for Hadoop (Code) Apache
  2. Collect, Aggregate, and Visualize a Data Ecosystem's Metadata (Code) WeWork
  3. Discovery and Consumption of Analytics Data at Twitter Twitter 2016
  4. Democratizing Data at Airbnb Airbnb 2017
  5. Databook: Turning Big Data into Knowledge with Metadata at Uber Uber 2018
  6. Metacat: Making Big Data Discoverable and Meaningful at Netflix (Code) Netflix 2018
  7. Amundsen — Lyft’s Data Discovery & Metadata Engine Lyft 2019
  8. Open Sourcing Amundsen: A Data Discovery And Metadata Platform (Code) Lyft 2019
  9. DataHub: A Generalized Metadata Search & Discovery Tool (Code) LinkedIn 2019
  10. Amundsen: One Year Later Lyft 2020
  11. Using Amundsen to Support User Privacy via Metadata Collection at Square Square 2020
  12. Turning Metadata Into Insights with Databook Uber 2020
  13. DataHub: Popular Metadata Architectures Explained LinkedIn 2020
  14. How We Improved Data Discovery for Data Scientists at Spotify Spotify 2020
  15. How We’re Solving Data Discovery Challenges at Shopify Shopify 2020
  16. Nemo: Data discovery at Facebook Facebook 2020
  17. Exploring Data @ Netflix (Code) Netflix 2021

Feature Stores

  1. Distributed Time Travel for Feature Generation Netflix 2016
  2. Building the Activity Graph, Part 2 (Feature Storage Section) LinkedIn 2017
  3. Fact Store at Scale for Netflix Recommendations Netflix 2018
  4. Zipline: Airbnb’s Machine Learning Data Management Platform Airbnb 2018
  5. Feature Store: The missing data layer for Machine Learning pipelines? Hopsworks 2018
  6. Introducing Feast: An Open Source Feature Store for Machine Learning (Code) Gojek 2019
  7. Michelangelo Palette: A Feature Engineering Platform at Uber Uber 2019
  8. The Architecture That Powers Twitter's Feature Store Twitter 2019
  9. Accelerating Machine Learning with the Feature Store Service Condé Nast 2019
  10. Feast: Bridging ML Models and Data Gojek 2020
  11. Building a Scalable ML Feature Store with Redis, Binary Serialization, and Compression DoorDash 2020
  12. Rapid Experimentation Through Standardization: Typed AI features for LinkedIn’s Feed LinkedIn 2020
  13. Building a Feature Store Monzo Bank 2020
  14. Butterfree: A Spark-based Framework for Feature Store Building (Code) QuintoAndar 2020
  15. Building Riviera: A Declarative Real-Time Feature Engineering Framework DoorDash 2021
  16. Optimal Feature Discovery: Better, Leaner Machine Learning Models Through Information Theory Uber 2021
  17. ML Feature Serving Infrastructure at Lyft Lyft 2021
  18. Near real-time features for near real-time personalization LinkedIn 2022
  19. Building the Model Behind DoorDash’s Expansive Merchant Selection DoorDash 2022
  20. Open sourcing Feathr – LinkedIn’s feature store for productive machine learning LinkedIn 2022
  21. Evolution of ML Fact Store Netflix 2022
  22. Developing scalable feature engineering DAGs Metaflow + Hamilton via Outerbounds 2022
  23. Feature Store Design at Constructor Constructor.io 2023

Classification

  1. Prediction of Advertiser Churn for Google AdWords (Paper) Google 2010
  2. High-Precision Phrase-Based Document Classification on a Modern Scale (Paper) LinkedIn 2011
  3. Chimera: Large-scale Classification using Machine Learning, Rules, and Crowdsourcing (Paper) Walmart 2014
  4. Large-scale Item Categorization in e-Commerce Using Multiple Recurrent Neural Networks (Paper) NAVER 2016
  5. Learning to Diagnose with LSTM Recurrent Neural Networks (Paper) Google 2017
  6. Discovering and Classifying In-app Message Intent at Airbnb Airbnb 2019
  7. Teaching Machines to Triage Firefox Bugs Mozilla 2019
  8. Categorizing Products at Scale Shopify 2020
  9. How We Built the Good First Issues Feature GitHub 2020
  10. Testing Firefox More Efficiently with Machine Learning Mozilla 2020
  11. Using ML to Subtype Patients Receiving Digital Mental Health Interventions (Paper) Microsoft 2020
  12. Scalable Data Classification for Security and Privacy (Paper) Facebook 2020
  13. Uncovering Online Delivery Menu Best Practices with Machine Learning DoorDash 2020
  14. Using a Human-in-the-Loop to Overcome the Cold Start Problem in Menu Item Tagging DoorDash 2020
  15. Deep Learning: Product Categorization and Shelving Walmart 2021
  16. Large-scale Item Categorization for e-Commerce (Paper) DianPing, eBay 2012
  17. Semantic Label Representation with an Application on Multimodal Product Categorization Walmart 2022
  18. Building Airbnb Categories with ML and Human-in-the-Loop Airbnb 2022

Regression

  1. Using Machine Learning to Predict Value of Homes On Airbnb Airbnb 2017
  2. Using Machine Learning to Predict the Value of Ad Requests Twitter 2020
  3. Open-Sourcing Riskquant, a Library for Quantifying Risk (Code) Netflix 2020
  4. Solving for Unobserved Data in a Regression Model Using a Simple Data Adjustment DoorDash 2020

Forecasting

  1. Engineering Extreme Event Forecasting at Uber with RNN Uber 2017
  2. Forecasting at Uber: An Introduction Uber 2018
  3. Transforming Financial Forecasting with Data Science and Machine Learning at Uber Uber 2018
  4. Under the Hood of Gojek’s Automated Forecasting Tool Gojek 2019
  5. BusTr: Predicting Bus Travel Times from Real-Time Traffic (Paper, Video) Google 2020
  6. Retraining Machine Learning Models in the Wake of COVID-19 DoorDash 2020
  7. Automatic Forecasting using Prophet, Databricks, Delta Lake and MLflow (Paper, Code) Atlassian 2020
  8. Introducing Orbit, An Open Source Package for Time Series Inference and Forecasting (Paper, Video, Code) Uber 2021
  9. Managing Supply and Demand Balance Through Machine Learning DoorDash 2021
  10. Greykite: A flexible, intuitive, and fast forecasting library LinkedIn 2021
  11. The history of Amazon’s forecasting algorithm Amazon 2021
  12. DeepETA: How Uber Predicts Arrival Times Using Deep Learning Uber 2022
  13. [Forecasting Grubhub Order Volume At Scale](https://bytes.grubhu

applied-ml (eugeneyan/applied-ml)

Searching Database…

30.6K ·GitHub
Applied-ml
What is this?applied-ml is a curated collection of papers, articles, and tech blogs where companies like Airbnb, Uber, Netflix, Google, and DoorDash share how they run data science and machine learning in production.
Is it a tool I install, or just a list to read?A list to read, not a tool to install — it is one GitHub README whose entries link out to each company's paper, engineering blog post, or open-source code.
Is this aimed at engineers running models in production, or at researchers?Practitioners shipping ML: next to modeling sections there are production-focused ones like Validation and A/B Testing, Model Management, Efficiency, Infra, and Team Structure, written by industry teams.
Are the links mostly academic papers or practitioner blog posts?Mostly practitioner tech blogs from company engineering teams; academic papers are marked (Paper) and open-sourced systems (Code), so you can tell them apart at a glance.
What is the difference between applied-ml and the author's ml-surveys?ml-surveys collects survey papers summarizing ML advancements, applyingML offers guides and interviews on applying ML, while applied-ml focuses specifically on company write-ups about running ML in production.
Which companies show up most often?Airbnb, Uber, Netflix, Google, DoorDash, and LinkedIn recur throughout — from Airbnb's Zipline and Uber's Michelangelo Palette to DoorDash's Riviera and LinkedIn's Feathr.
What are similar projects?DeepLearning, ML-YouTube-Courses, Awesome-pytorch-list, Deep-Learning-Papers-Reading-Roadmap, awesome-deep-learning-papers, transferlearning, awesome-deep-learning, spaCy
Which topics is it in?production, anomaly-detection, data-quality, data-engineering, papers, natural-language-processing, embeddings, reinforcement-learning, search, data-science, reference, computer-vision
Creator
eugeneyan
Size
0.4 MB
Stars
30.6K
Updated
CommitMay 02
Age
6y
Commits · 90 Days