22.1K ·GitHub
What is this?
ONNX Runtime is a cross-platform, high performance ML accelerator for inference and training. It runs models from PyTorch, TensorFlow/Keras and classical ML libraries like scikit-learn, LightGBM and XGBoost.
ONNX Runtime is a cross-platform, high performance ML accelerator for inference and training. It runs models from PyTorch, TensorFlow/Keras and classical ML libraries like scikit-learn, LightGBM and XGBoost.Is this a library I integrate into my app, or a tool I run on its own?It's a runtime you integrate into your own app or training script, not a standalone tool with a UI. You bring the model you trained, and ONNX Runtime executes it optimized on your hardware.
It's a runtime you integrate into your own app or training script, not a standalone tool with a UI. You bring the model you trained, and ONNX Runtime executes it optimized on your hardware.How do I get started with it?The docs and tutorials live at onnxruntime.ai/docs, with YouTube video tutorials as well. Companion repos microsoft/onnxruntime-inference-examples and microsoft/onnxruntime-training-examples contain working samples.
The docs and tutorials live at onnxruntime.ai/docs, with YouTube video tutorials as well. Companion repos microsoft/onnxruntime-inference-examples and microsoft/onnxruntime-training-examples contain working samples.Which frameworks' models can it run?Deep learning models from PyTorch and TensorFlow/Keras, plus classical ML from scikit-learn, LightGBM, XGBoost and similar libraries.
Deep learning models from PyTorch and TensorFlow/Keras, plus classical ML from scikit-learn, LightGBM, XGBoost and similar libraries.Does it send any telemetry to Microsoft?It may collect usage data and send it to Microsoft to help improve products and services; Microsoft's privacy statement covers the details.
It may collect usage data and send it to Microsoft to help improve products and services; Microsoft's privacy statement covers the details.How is it licensed — can I use it in my own products?It's MIT-licensed, so it's free to use, including in commercial products.
It's MIT-licensed, so it's free to use, including in commercial products.Do I need a beefy GPU for it to be worth using?No specific hardware is required — it works across different hardware, drivers and operating systems, using hardware accelerators where available along with graph optimizations and transforms.
No specific hardware is required — it works across different hardware, drivers and operating systems, using hardware accelerators where available along with graph optimizations and transforms.Can it lower my inference costs?That's a stated goal: the README says inference can enable faster customer experiences and lower costs, achieved through graph optimizations and leveraging hardware accelerators where they exist.
That's a stated goal: the README says inference can enable faster customer experiences and lower costs, achieved through graph optimizations and leveraging hardware accelerators where they exist.Is there a plugin for Qualcomm's QNN accelerator?Yes. The QNN Plugin EP is maintained in a separate companion repository, onnxruntime/onnxruntime-qnn, alongside the inference and training example repos.
Yes. The QNN Plugin EP is maintained in a separate companion repository, onnxruntime/onnxruntime-qnn, alongside the inference and training example repos.What are similar projects?deeplearning4j, CNTK, TensorRT-LLM, openvino, FlashMLA, spaCy, jeeves, DeepGEMM
deeplearning4j, CNTK, TensorRT-LLM, openvino, FlashMLA, spaCy, jeeves, DeepGEMMWhich topics is it in?ai-framework, keras, scikit-learn, inference-engine, neural-networks, onnx, tensorflow, training, high-performance, transformer, inference, nvidia
ai-framework, keras, scikit-learn, inference-engine, neural-networks, onnx, tensorflow, training, high-performance, transformer, inference, nvidia