What is this?
A collection of machine learning algorithms — Gaussian mixture models, HMMs, neural networks, tree ensembles, reinforcement learning agents and more — implemented from scratch in pure NumPy.
A collection of machine learning algorithms — Gaussian mixture models, HMMs, neural networks, tree ensembles, reinforcement learning agents and more — implemented from scratch in pure NumPy.What kinds of models are inside?Gaussian mixture and hidden Markov models, LDA topic models, a full neural-network stack (CNNs, LSTMs, attention, ResNet/WaveNet blocks, VAEs, WGAN-GP, word2vec), tree ensembles, linear and Bayesian models, n-grams, bandits, RL agents, KNN, Gaussian processes, matrix factorization — plus preprocessing like DFT, MFCC, BPE and TF-IDF.
Gaussian mixture and hidden Markov models, LDA topic models, a full neural-network stack (CNNs, LSTMs, attention, ResNet/WaveNet blocks, VAEs, WGAN-GP, word2vec), tree ensembles, linear and Bayesian models, n-grams, bandits, RL agents, KNN, Gaussian processes, matrix factorization — plus preprocessing like DFT, MFCC, BPE and TF-IDF.How do I get started with it?Clone the repo, create a virtualenv and start hacking on the source — or, if you just want to use it, install it as a package with pip3 install -u numpyml.
Clone the repo, create a virtualenv and start hacking on the source — or, if you just want to use it, install it as a package with pip3 install -u numpyml.Do I need anything besides Python and NumPy?No — every model uses only the Python standard library and NumPy (SciPy only in special cases). The RL agents additionally train on OpenAI gym environments, installed via the numpyml[rl] extra.
No — every model uses only the Python standard library and NumPy (SciPy only in special cases). The RL agents additionally train on OpenAI gym environments, installed via the numpyml[rl] extra.Is it fast enough to train real models on?It is deliberately 'inefficient but somewhat legible' — built for readability and experimentation, not for the throughput you would expect from a production training stack.
It is deliberately 'inefficient but somewhat legible' — built for readability and experimentation, not for the throughput you would expect from a production training stack.Who is this for — people learning ML or people building products?It is aimed at ML prototyping and experimentation: students and practitioners who want to read, tweak and learn from the implementations rather than treat them as a black box.
It is aimed at ML prototyping and experimentation: students and practitioners who want to read, tweak and learn from the implementations rather than treat them as a black box.Does it cover modern deep learning pieces like Transformer attention?Yes — the neural network module includes dot-product attention, Transformer-style multi-headed scaled dot product attention, WaveNet-style residual blocks with dilated causal convolutions, ResNet-style residual blocks, and bidirectional LSTMs.
Yes — the neural network module includes dot-product attention, Transformer-style multi-headed scaled dot product attention, WaveNet-style residual blocks with dilated causal convolutions, ResNet-style residual blocks, and bidirectional LSTMs.Which reinforcement learning agents does it ship with?Cross-entropy method, first-visit and importance-sampling Monte Carlo, expected SARSA, TD-0 Q-learning, and Dyna-Q / Dyna-Q+ with prioritized sweeping — all trained on OpenAI gym environments.
Cross-entropy method, first-visit and importance-sampling Monte Carlo, expected SARSA, TD-0 Q-learning, and Dyna-Q / Dyna-Q+ with prioritized sweeping — all trained on OpenAI gym environments.What preprocessing and utility helpers are included?DFT, DCT, MFCC and Huffman coding, text tokenization, byte pair encoding, TF-IDF, feature hashing and one-hot encoding, plus utilities like im2col/col2im, distance metrics, a ball tree and a priority queue.
DFT, DCT, MFCC and Huffman coding, text tokenization, byte pair encoding, TF-IDF, feature hashing and one-hot encoding, plus utilities like im2col/col2im, distance metrics, a ball tree and a priority queue.What are similar projects?DeepLearning, maths-cs-ai-compendium, ailearning, 100-Days-Of-ML-Code, MLAlgorithms, lectures, d2l-en, annotated_deep_learning_paper_implementations
DeepLearning, maths-cs-ai-compendium, ailearning, 100-Days-Of-ML-Code, MLAlgorithms, lectures, d2l-en, annotated_deep_learning_paper_implementationsWhich topics is it in?gradient-boosting, word2vec, lstm, machine-learning-algorithms, linear-algebra, attention, from-scratch, generative-adversarial-network, cnn, math, attention-mechanism, neural-networks
gradient-boosting, word2vec, lstm, machine-learning-algorithms, linear-algebra, attention, from-scratch, generative-adversarial-network, cnn, math, attention-mechanism, neural-networks