What is this?
Fairseq is Facebook AI Research's sequence modeling toolkit for training custom models for translation, summarization, language modeling and other text generation tasks, built on PyTorch.
Fairseq is Facebook AI Research's sequence modeling toolkit for training custom models for translation, summarization, language modeling and other text generation tasks, built on PyTorch.Is this a ready-made app or a toolkit I build on?It is a Python research toolkit/library on top of PyTorch, used through command-line training/generation tools or by registering your own models, criterions, tasks and optimizers.
It is a Python research toolkit/library on top of PyTorch, used through command-line training/generation tools or by registering your own models, criterions, tasks and optimizers.How do I install it and get a model running?Clone the repo and install it locally with pip (editable install), then use the provided torch.hub interface or pre-trained models listed for translation and language modeling.
Clone the repo and install it locally with pip (editable install), then use the provided torch.hub interface or pre-trained models listed for translation and language modeling.What do I need before training a model?You need PyTorch 1.10.0+ and Python 3.8+; for training new models you'll also need an NVIDIA GPU with NCCL, and optionally NVIDIA's apex library or PyArrow for large datasets.
You need PyTorch 1.10.0+ and Python 3.8+; for training new models you'll also need an NVIDIA GPU with NCCL, and optionally NVIDIA's apex library or PyArrow for large datasets.Is this still actively maintained?No, the repository is archived, meaning it is read-only and no longer receives updates even though the code, papers and pretrained models remain available for use.
No, the repository is archived, meaning it is read-only and no longer receives updates even though the code, papers and pretrained models remain available for use.Do I need an NVIDIA GPU to use it?Yes for training new models (requires NVIDIA GPU + NCCL), though generation can run on CPU as well, and mixed precision training on NVIDIA tensor cores speeds things up further.
Yes for training new models (requires NVIDIA GPU + NCCL), though generation can run on CPU as well, and mixed precision training on NVIDIA tensor cores speeds things up further.What tasks can fairseq models handle?Translation, language modeling, summarization, speech recognition (via wav2vec), and other text/speech generation tasks, with reference implementations for each.
Translation, language modeling, summarization, speech recognition (via wav2vec), and other text/speech generation tasks, with reference implementations for each.Which well-known models were first released through fairseq?RoBERTa, BART, mBART, wav2vec/wav2vec 2.0, XLM-R, and Transformer-based WMT translation models all shipped as fairseq reference implementations.
RoBERTa, BART, mBART, wav2vec/wav2vec 2.0, XLM-R, and Transformer-based WMT translation models all shipped as fairseq reference implementations.Can I use pretrained models without training my own?Yes, fairseq provides pre-trained translation and language modeling checkpoints accessible through a convenient torch.hub interface.
Yes, fairseq provides pre-trained translation and language modeling checkpoints accessible through a convenient torch.hub interface.What search/decoding strategies does generation support?Beam search, Diverse Beam Search, unconstrained/top-k/top-p sampling, and lexically constrained decoding are all built in for fast CPU or GPU generation.
Beam search, Diverse Beam Search, unconstrained/top-k/top-p sampling, and lexically constrained decoding are all built in for fast CPU or GPU generation.What are similar projects?char-rnn, gpt-2, llama, nanoGPT, minGPT, train-llm-from-scratch, DeepLearningExamples, FireRedAudio
char-rnn, gpt-2, llama, nanoGPT, minGPT, train-llm-from-scratch, DeepLearningExamples, FireRedAudioWhich topics is it in?self-supervised-learning, machine-translation, text-generation, language-model, speech-recognition, large-language-models, transformer, nlp, artificial-intelligence, nvidia, deprecated, cuda
self-supervised-learning, machine-translation, text-generation, language-model, speech-recognition, large-language-models, transformer, nlp, artificial-intelligence, nvidia, deprecated, cuda