14.9K ·GitHub
What is this?
A collection of short PyTorch implementations covering core NLP and deep learning models, from word embeddings to Transformers and BERT, meant as a study resource.
A collection of short PyTorch implementations covering core NLP and deep learning models, from word embeddings to Transformers and BERT, meant as a study resource.Is this a library I import, or a set of standalone examples?It's a curriculum of standalone Jupyter notebooks/scripts, each implementing one model, not a reusable library you install and import.
It's a curriculum of standalone Jupyter notebooks/scripts, each implementing one model, not a reusable library you install and import.How do I run these examples?Open any notebook locally with Jupyter or run it directly in Google Colab using the linked Colab buttons for each model.
Open any notebook locally with Jupyter or run it directly in Google Colab using the linked Colab buttons for each model.What do I need installed to run the code?Python 3.5+ and PyTorch 1.0.0 or higher.
Python 3.5+ and PyTorch 1.0.0 or higher.Does it still support TensorFlow?No, the old TensorFlow v1 implementations have been moved to an archive folder; only the PyTorch versions are actively maintained for readability.
No, the old TensorFlow v1 implementations have been moved to an archive folder; only the PyTorch versions are actively maintained for readability.Who is this curriculum meant for?Deep learning researchers and students studying NLP who want to see minimal, readable reference implementations of classic and modern models.
Deep learning researchers and students studying NLP who want to see minimal, readable reference implementations of classic and modern models.How is the curriculum organized?It's structured in five progressive sections: basic embedding models (NNLM, Word2Vec, FastText), CNNs, RNNs, attention-based Seq2Seq models, and Transformer-based models (Transformer, BERT).
It's structured in five progressive sections: basic embedding models (NNLM, Word2Vec, FastText), CNNs, RNNs, attention-based Seq2Seq models, and Transformer-based models (Transformer, BERT).Where should a beginner start?Start with section 1, Basic Embedding Models (NNLM), and work through sections in order up to the Transformer and BERT implementations at the end.
Start with section 1, Basic Embedding Models (NNLM), and work through sections in order up to the Transformer and BERT implementations at the end.Does each model include the original research paper?Yes, every implemented model links to its source paper, such as 'Attention Is All You Need' for the Transformer and the original BERT paper.
Yes, every implemented model links to its source paper, such as 'Attention Is All You Need' for the Transformer and the original BERT paper.What does the BERT example actually do?It implements BERT's pretraining tasks: Next Sentence Prediction classification and Masked Token prediction.
It implements BERT's pretraining tasks: Next Sentence Prediction classification and Masked Token prediction.What are similar projects?DeepLearning, lectures, Chinese-BERT-wwm, llama3-from-scratch, deeplearning-models, spaCy, deep-learning-with-python-notebooks, bert
DeepLearning, lectures, Chinese-BERT-wwm, llama3-from-scratch, deeplearning-models, spaCy, deep-learning-with-python-notebooks, bertWhich topics is it in?word2vec, bert, attention, colab, neural-networks, tensorflow, paper, natural-language-processing, neural-network, jupyter notebook, transformer, nlp
word2vec, bert, attention, colab, neural-networks, tensorflow, paper, natural-language-processing, neural-network, jupyter notebook, transformer, nlp