What is this?
Qxotic is an open-source AI stack built specifically for the JVM, giving Java applications local AI inference — chat, vision, audio, embeddings, reranking and text-to-speech — without needing Python or ONNX.
Qxotic is an open-source AI stack built specifically for the JVM, giving Java applications local AI inference — chat, vision, audio, embeddings, reranking and text-to-speech — without needing Python or ONNX.Is this one library, or a bundle of several?It is a stack of six modules — jinfer (inference engine), toknroll (tokenizers), jam (quantized matmul), jota (tensor engine), and gguf/safetensors readers/writers — that together cover tokenization through model loading and inference.
It is a stack of six modules — jinfer (inference engine), toknroll (tokenizers), jam (quantized matmul), jota (tensor engine), and gguf/safetensors readers/writers — that together cover tokenization through model loading and inference.How do I build and use it?You build it with Maven, which pulls in the modules you need; native jam kernels additionally require cmake and a C compiler, and `make help` lists the available build targets.
You build it with Maven, which pulls in the modules you need; native jam kernels additionally require cmake and a C compiler, and `make help` lists the available build targets.Do I need Python or ONNX to run it?No — everything runs end-to-end on the JVM with zero external services, so Python and ONNX are never part of the pipeline.
No — everything runs end-to-end on the JVM with zero external services, so Python and ONNX are never part of the pipeline.Is GPU acceleration required to run it?No, native GPU acceleration is optional — the tensor engine works on CPU by default and can be sped up on CUDA, HIP, Metal or OpenCL hardware when available.
No, native GPU acceleration is optional — the tensor engine works on CPU by default and can be sped up on CUDA, HIP, Metal or OpenCL hardware when available.What is GraalVM Native Image support used for?It lets Qxotic apps compile to self-contained native binaries with a small footprint and millisecond startup, instead of running on a warmed-up full JVM.
It lets Qxotic apps compile to self-contained native binaries with a small footprint and millisecond startup, instead of running on a warmed-up full JVM.Which model formats can it read?It ships pure-Java, zero-dependency readers/writers for GGUF (llama.cpp's format) and Safetensors (HuggingFace's format).
It ships pure-Java, zero-dependency readers/writers for GGUF (llama.cpp's format) and Safetensors (HuggingFace's format).Which GPU backends are supported?The jota tensor engine targets CUDA, HIP, Metal and OpenCL, alongside native C kernels for several CPU instruction sets via the jam module.
The jota tensor engine targets CUDA, HIP, Metal and OpenCL, alongside native C kernels for several CPU instruction sets via the jam module.Why build this instead of calling llama.cpp from Java?Qxotic is designed from first principles for the JVM so AI runs natively end-to-end, without sidecar servers, IPC or JNI glue — tokenization, model loading and inference all stay in Java/native JVM code.
Qxotic is designed from first principles for the JVM so AI runs natively end-to-end, without sidecar servers, IPC or JNI glue — tokenization, model loading and inference all stay in Java/native JVM code.What are similar projects?unsloth, rizzo-flow, TornadoVM, Speech, FireRedAudio, airllm, speech-swift, faster-whisper
unsloth, rizzo-flow, TornadoVM, Speech, FireRedAudio, airllm, speech-swift, faster-whisperWhich topics is it in?rerank, hip, opencl, text-generation, tokenization, asr, llama, metal, speech-recognition, native, llm-inference, tts
rerank, hip, opencl, text-generation, tokenization, asr, llama, metal, speech-recognition, native, llm-inference, tts