awesome-public-datasets README

Awesome Public Datasets

.. image:: https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg :alt: Awesome :target: https://github.com/sindresorhus/awesome

This is a list of topic-centric public data sources <https://github.com/awesomedata/awesome-public-datasets>_ in high quality. They are collected and tidied from blogs, answers, and user responses. Most of the data sets listed below are free, however, some are not. This project was incubated at OMNILab <https://github.com/OMNILab>, Shanghai Jiao Tong University during Xiaming Chen's Ph.D. studies. OMNILab is now part of the BaiYuLan Open AI community <https://github.com/Bai-Yu-Lan>. Other amazingly awesome lists can be found in sindresorhus's awesome <https://github.com/sindresorhus/awesome>_ list.

NOTICE: This repo is automatically generated by apd-core <https://github.com/awesomedata/apd-core/tree/master/core>. Please DO NOT modify this file directly. We have provided a new way to contribute to this repo <https://github.com/awesomedata/apd-core/blob/master/CONTRIBUTING.md>. Join <https://join.slack.com/t/awesomedataworld/shared_invite/zt-dllew5xy-PJYi~mWUdY3hupohbmVZsA>_ the slack community <https://awesomedataworld.slack.com>_ for an instant touch of HQ data updates.

.. |OK_ICON| image:: https://raw.githubusercontent.com/awesomedata/apd-core/master/deploy/ok-24.png .. |FIXME_ICON| image:: https://raw.githubusercontent.com/awesomedata/apd-core/master/deploy/fixme-24.png

|OK_ICON| I am well.
|FIXME_ICON| Please fix me.

.. contents:: Table of Contents

Agriculture

|OK_ICON| Chinese Tea Dataset - A curated, open dataset of 100+ Chinese teas covering type, origin, [...] <https://chinatea.house/dataset/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Chinese-Tea-Dataset.yml>_]
|FIXME_ICON| Furrow Farmland Price Index - Agricultural land prices in 37 countries, each from that [...] <https://doi.org/10.5281/zenodo.22212366>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Furrow-Farmland-Price-Index.yml>_]
|OK_ICON| The global dataset of historical yields for major crops 1981–2016 - The Global Dataset of [...] <https://doi.pangaea.de/10.1594/PANGAEA.909132>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Global-dataset-of-historical-yields-for-major-crops.yml>_]
|FIXME_ICON| Hyperspectral benchmark dataset on soil moisture - This dataset was measured in a five-day [...] <https://doi.org/10.5281/zenodo.1227837>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Hyperspectral-Benchmark-Dataset-On-Soil-Moisture.yml>_]
|OK_ICON| Lemons quality control dataset - Lemon dataset has been prepared to investigate the [...] <https://github.com/softwaremill/lemon-dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Lemon-Dataset.yml>_]
|FIXME_ICON| Open Food Facts - Crowdsourced food composition database covering 2.5M+ products from 170+ [...] <https://world.openfoodfacts.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/OpenFoodFacts.yml>_]
|OK_ICON| Optimized Soil Adjusted Vegetation Index - The IDB is a tool for working with remote sensing [...] <https://www.indexdatabase.de/db/i-single.php?id=63>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Optimized Soil Adjusted Vegetation Index>_]
|OK_ICON| Robot Lawn Mower Specs - Open dataset of specifications for 23 robot lawn mower models across [...] <https://github.com/yumaheymans/robot-lawn-mower-specs>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Robot-Lawn-Mower-Specs.yml>_]
|FIXME_ICON| TempoLife Food Nutrition Dataset - Per-100g nutrition values (kcal, protein, carbs, fat, [...] <https://zenodo.org/records/22208174>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/TempoLife-Food-Nutrition-Dataset.yml>_]
|OK_ICON| Turkish Olive Production Data - Olive and olive oil production in Turkiye at province, region [...] <https://github.com/yudumnet/zeytinnet>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Turkish-Olive-Production.yml>_]
|OK_ICON| USDA FoodData Central - The primary source for food composition data from the USDA, providing [...] <https://fdc.nal.usda.gov/download-datasets>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/U.S.-Department-of-Agricultures-Nutrient-Database.yml>_]
|FIXME_ICON| U.S. Department of Agriculture's PLANTS Database - The Complete PLANTS Checklist is nearly 7 [...] <https://plants.usda.gov/downloads>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/U.S.-Department-of-Agricultures-PLANTS-Database.yml>_]
|OK_ICON| US Farmers Market Statistics 2026 - State-by-state statistics for ~7,056 US farmers markets — [...] <https://harvestlymarkets.com/farmers-market-statistics/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/US-Farmers-Market-Statistics-2026.yml>_]
|OK_ICON| U.S. ZIP/ZCTA Growing Degree Days (base 50°F) - Daily-updated cumulative growing degree days [...] <https://whentoapplypreemergent.com/datasets/gdd/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/US-ZIP-ZCTA-Growing-Degree-Days.yml>_]
|OK_ICON| USDA NASS County Crop Yields - Free static JSON API serving USDA NASS county-level annual [...] <https://productofamerica.github.io/usda-county-yields/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/USDA-NASS-County-Crop-Yields.yml>_]
|OK_ICON| World Cuisine Ingredient Overlap - Ingredient overlap between 24 world cuisines, measured [...] <https://www.kaggle.com/datasets/alperacax/world-cuisine-ingredient-overlap-24-cuisines>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/World-Cuisine-Ingredient-Overlap.yml>_]
|OK_ICON| Yard Material Coverage Data - An open reference dataset for cubic-yard coverage by depth, [...] <https://demi-valerith.github.io/yard-material-coverage-data/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Agriculture/Yard-Material-Coverage-Data.yml>_]

Architecture

|FIXME_ICON| Swiss Apartment Models - This dataset contains detailed data on 42,207 apartments (242,257 [...] <https://zenodo.org/record/7070952#.Y0mACy0RqO0>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Architecture/appartment-models.yml>_]

Biology

|OK_ICON| 1000 Genomes - The 1000 Genomes Project ran between 2008 and 2015, creating the largest [...] <https://www.internationalgenome.org/data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/1000-Genomes.yml>_]
|OK_ICON| ANHIR - Automatic Non-rigid Histological Image Registration (ANHIR) consists of 2D [...] <https://anhir.grand-challenge.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/ANHIR.yml>_]
|OK_ICON| American Gut (Microbiome Project) - The American Gut project is the largest crowdsourced [...] <https://github.com/biocore/American-Gut>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/American-Gut-Microbiome-Project.yml>_]
|OK_ICON| BCNB - There are WSIs of 1058 patients, part of tumor regions are annotated in WSIs. Except [...] <https://bupt-ai-cz.github.io/BCNB/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/BCNB.yml>_]
|OK_ICON| Broad Bioimage Benchmark Collection (BBBC) - The Broad Bioimage Benchmark Collection (BBBC) [...] <https://www.broadinstitute.org/bbbc>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Broad-Bioimage-Benchmark-Collection-BBBC.yml>_]
|OK_ICON| Broad Cancer Cell Line Encyclopedia (CCLE) <http://www.broadinstitute.org/ccle/home>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Broad-Cancer-Cell-Line-Encyclopedia-CCLE.yml>_]
|OK_ICON| CIMA: Histological Microscopy Tissue Slices - CIMA dataset includes images of 2D histological [...] <https://www.kaggle.com/datasets/jirkaborovec/histology-cima-dataset>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/CIMA.yml>_]
|FIXME_ICON| Cell Image Library <https://www.cellimagelibrary.org/home>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Cell-Image-Library.yml>_]
|OK_ICON| CytoImageNet - A large-scale dataset of microscopy images. Contains 890,737 total grayscale [...] <https://www.kaggle.com/stanleyhua/cytoimagenet>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/CytoImageNet.yml>_]
|OK_ICON| DNA Chip Coverage - Which SNP markers are present on consumer DNA genotyping arrays. 70 [...] <https://github.com/AndreySoloviev/dna-chip-coverage>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/DNA-Chip-Coverage.yml>_]
|OK_ICON| EBI ArrayExpress - ArrayExpress Archive of Functional Genomics Data stores data from high- [...] <http://www.ebi.ac.uk/arrayexpress/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/EBI-ArrayExpress.yml>_]
|OK_ICON| EBI Protein Data Bank in Europe - The Electron Microscopy Data Bank (EMDB) is a public [...] <https://www.ebi.ac.uk/emdb/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/EBI-Protein-Data-Bank-in-Europe.yml>_]
|OK_ICON| ENCODE project - The Encyclopedia of DNA Elements (ENCODE) Consortium is an ongoing [...] <https://www.encodeproject.org>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/ENCODE-project.yml>_]
|OK_ICON| Electron Microscopy Pilot Image Archive (EMPIAR) - EMPIAR, the Electron Microscopy Public [...] <http://www.ebi.ac.uk/pdbe/emdb/empiar/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Electron-Microscopy-Pilot-Image-Archive-EMPIAR.yml>_]
|OK_ICON| Ensembl Genomes <https://ensemblgenomes.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Ensembl-Genomes.yml>_]
|OK_ICON| Gene Expression Omnibus (GEO) - GEO is a public functional genomics data repository [...] <http://www.ncbi.nlm.nih.gov/geo/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Gene-Expression-Omnibus-GEO.yml>_]
|OK_ICON| Gene Ontology (GO) - GO annotation files <http://geneontology.org/docs/download-go-annotations/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Gene-Ontology-GO.yml>_]
|OK_ICON| Global Biotic Interactions (GloBI) <https://github.com/jhpoelen/eol-globi-data/wiki#accessing-species-interaction-data>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Global-Biotic-Interactions-GloBI.yml>_]
|OK_ICON| Harvard Medical School (HMS) LINCS Project - The Harvard Medical School (HMS) LINCS Center is [...] <http://lincs.hms.harvard.edu>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Harvard-Medical-School-LINCS-Project.yml>_]
|OK_ICON| Human Genome Diversity Project - A group of scientists at Stanford University have [...] <hhttps://rosenberglab.stanford.edu/hgdpsnpDownload.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Human-Genome-Diversity-Project.yml>_]
|OK_ICON| ICOS PSP Benchmark - The ICOS PSP benchmarks repository contains an adjustable real-world [...] <http://ico2s.org/datasets/psp_benchmark.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/ICOS-PSP-Benchmark.yml>_]
|OK_ICON| International HapMap Project <http://hapmap.ncbi.nlm.nih.gov/downloads/index.html.en>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/International-HapMap-Project.yml>_]
|OK_ICON| Journal of Cell Biology DataViewer - All JCB data was moved to Biostudies <https://www.ebi.ac.uk/biostudies/JCB/studies>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Journal-of-Cell-Biology-DataViewer.yml>_]
|OK_ICON| KEGG - KEGG is a database resource for understanding high-level functions and utilities of [...] <http://www.genome.jp/kegg/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/KEGG.yml>_]
|OK_ICON| NCBI Proteins <http://www.ncbi.nlm.nih.gov/guide/proteins/#databases>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NCBI-Proteins.yml>_]
|OK_ICON| NCBI Taxonomy - The NCBI Taxonomy database is a curated set of names and classifications for [...] <http://www.ncbi.nlm.nih.gov/taxonomy>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NCBI-Taxonomy.yml>_]
|OK_ICON| NCI Genomic Data Commons - The GDC Data Portal is a robust data-driven platform that allows [...] <https://gdc.cancer.gov/access-data/gdc-data-portal>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NCI-Genomic-Data-Commons.yml>_]
|OK_ICON| NIH Microarray data <ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE6532/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/NIH-Microarray-data.yml>_]
|FIXME_ICON| OpenSNP genotypes data - openSNP allows customers of direct-to-customer genetic tests to [...] <https://opensnp.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/OpenSNP-genotypes-data.yml>_]
|OK_ICON| Palmer Penguins - The goal of palmerpenguins is to provide a great dataset for data [...] <https://allisonhorst.github.io/palmerpenguins/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Palmer-Penguins.yml>_]
|FIXME_ICON| Pathguid - Protein-Protein Interactions Catalog <http://www.pathguide.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Pathguid.yml>_]
|OK_ICON| Protein Data Bank - This resource is powered by the Protein Data Bank archive-information [...] <http://www.rcsb.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Protein-Data-Bank.yml>_]
|OK_ICON| Psychiatric Genomics Consortium - The purpose of the Psychiatric Genomics Consortium (PGC) is [...] <https://www.med.unc.edu/pgc/downloads>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Psychiatric-Genomics-Consortium.yml>_]
|OK_ICON| PubChem Project - PubChem is the world's largest collection of freely accessible chemical [...] <https://pubchem.ncbi.nlm.nih.gov/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/PubChem-Project.yml>_]
|OK_ICON| PubGene (now Coremine Medical) - COREMINE™ is a family of tools developed by the Norwegian [...] <https://www.coremine.com/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/PubGene-now-Coremine-Medical.yml>_]
|OK_ICON| Sanger Catalogue of Somatic Mutations in Cancer (COSMIC) - COSMIC, the Catalogue Of Somatic [...] <http://cancer.sanger.ac.uk/cosmic>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Sanger-Catalogue-of-Somatic-Mutations-in-Cancer-COSMIC.yml>_]
|FIXME_ICON| Sanger Genomics of Drug Sensitivity in Cancer Project (GDSC) <https://www.cancerrxgene.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Sanger-Genomics-of-Drug-Sensitivity-in-Cancer-Project-GDSC.yml>_]
|OK_ICON| Sequence Read Archive(SRA) - The Sequence Read Archive (SRA) stores raw sequence data from [...] <http://www.ncbi.nlm.nih.gov/Traces/sra/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Sequence-Read-ArchiveSRA.yml>_]
|OK_ICON| Serratus - Analysis of 7.1 million RNA/DNA sequencing datasets to discover the total [...] <https://github.com/ababaian/serratus/wiki/Access-Data-Release>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Serratus-Open-Virome.yml>_]
|FIXME_ICON| Stanford Microarray Data (Retired NOW) <http://smd.princeton.edu/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Stanford-Microarray-Data.yml>_]
|FIXME_ICON| Stowers Institute Original Data Repository <http://www.stowers.org/research/publications/odr>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Stowers-Institute-Original-Data-Repository.yml>_]
|OK_ICON| Systems Science of Biological Dynamics (SSBD) Database - Systems Science of Biological [...] <http://ssbd.qbic.riken.jp>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Systems-Science-of-Biological-Dynamics-SSBD-Database.yml>_]
|OK_ICON| The Cancer Genome Atlas (TCGA), available via Broad GDAC <https://gdac.broadinstitute.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/The-Cancer-Genome-Atlas-TCGA-available-via-Broad-GDAC.yml>_]
|OK_ICON| The Catalogue of Life - The Catalogue of Life is a quality-assured checklist of more than 1.8 [...] <https://www.catalogueoflife.org/data/download>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/The-Catalogue-of-Life.yml>_]
|OK_ICON| The Personal Genome Project - The Personal Genome Project, initiated in 2005, is a vision and [...] <http://www.personalgenomes.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/The-Personal-Genome-Project.yml>_]
|OK_ICON| UCSC Public Data <http://hgdownload.soe.ucsc.edu/downloads.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/UCSC-Public-Data.yml>_]
|OK_ICON| UniGene <https://ftp.ncbi.nlm.nih.gov/repository/UniGene/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/UniGene.yml>_]
|OK_ICON| Universal Protein Resource (UnitProt) - The Universal Protein Resource (UniProt) is a [...] <http://www.uniprot.org/downloads>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/Universal-Protein-Resource.yml>_]
|OK_ICON| iNaturalist - Community-maintained species observation database with 180M+ observations [...] <https://www.inaturalist.org/>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/iNaturalist.yml>_]
|OK_ICON| Rfam - The Rfam database is a collection of RNA families, each represented by multiple [...] <https://docs.rfam.org/en/latest/database.html>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Biology/rfam.yml>_]

Chemistry

|OK_ICON| Ionic Liquids Database - ILThermo <https://ilthermo.boulder.nist.gov>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Chemistry/ionicliquids.yml>_]
|OK_ICON| LibertyCES Industrial Chemical Equipment Data - Real, open industrial chemical equipment [...] <https://data.libertyces.com>_ [Meta <https://github.com/awesomedata/apd-core/tree/master/core//Chemistry/libertyces-industrial-chemical-equipment-data.yml>_]
|OK_ICON| `State of Peptides 2026 - Open reference dataset of 156 peptide and peptide-adjace

79.6K ·GitHub

Awesome-public-datasets

What is this?

A huge topic-centric list of high-quality open public datasets, organized by subject area like biology, chemistry, economics, and more.

Is this one big dataset or an index of many?

It is an index: a curated README linking out to hundreds of external datasets, each with a short description and source link, grouped by topic rather than hosted here.

How is the list organized, and where should I start?

Entries are grouped under topic headings such as Biology, Chemistry, and others, each entry listing the dataset name, a short description, and a link to the source; browse the topic matching your interest and click through to the original data provider.

Do I need any tools or accounts to use these datasets?

No, this repo itself is just a markdown/reST index; requirements depend entirely on each external dataset's own access terms, which vary by source.

Are all the linked datasets actually working and up to date?

No, some entries are marked as broken with a FIXME icon next to them, meaning the link or dataset status is unverified or outdated, alongside OK-marked entries that are confirmed good.

Who is this useful for?

Data scientists, researchers, and machine learning practitioners looking for free public datasets across many domains like genomics, chemistry, and species data.

Are entries vetted for quality?

Yes, entries are marked with OK or FIXME icons indicating whether the link/dataset has been verified as working, giving a quick signal of reliability before you click through.

Is the list still updated?

Yes, it includes recently added entries like a 2026 peptide dataset, suggesting ongoing maintenance and additions.

What kinds of topics are covered?

Topics span biology, chemistry, genomics, agriculture, and many other scientific and general-interest domains, each with its own section of linked datasets.

Who was Aaron Swartz and why is he referenced?

The project topic 'aaron-swartz' honors the open-data activist Aaron Swartz, reflecting the list's mission of promoting free access to public data.

What are similar projects?

google-research, 500-AI-Machine-learning-Deep-learning-Computer-vision-NLP-Projects-with-code, COVID-19, awesome-computer-vision, NLP-progress, transferlearning, 3D-Machine-Learning, AI-Papers-of-the-Week

Which topics is it in?

science, datasets, dataset, academic-research, data-science, awesome-list, list, curated-list, resources, research, machine-learning, education

Creator: awesomedata
Size: 2.3 MB
Stars: 79.6K
Updated: CommitOct 14 Releasev20230903Sep 03
Age: 11y
Commits · 90 Days