# cactus (cactus-compute/cactus)

> Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction that ships as a single 14MB binary and runs a full session in about 28MB of RAM. This repo is its Python package for inference, LoRA fine-tuning and export.

- GitHub: https://github.com/cactus-compute/cactus
- GitHub.Locker page: https://github.locker/cactus-compute/cactus
- Stars: 14.5K
- Language: Python
- License: Apache-2.0
- Last push: 2026-11-05

## Questions and answers

Written by GitHub.Locker from the README and the repository; check the README below for details.

**Is it a Python library or a model that runs on devices?**

Both stages: this repo is a pip-installable Python package, and what it produces is a self-contained .cact model for on-device engines. The project targets phones, wearables, smart homes, robots, cars and microcontrollers.

**How do I get it running?**

pip install cactus-needle, describe your tools, and call them from Python — the inference engine is fetched once from Hugging Face and cached, with nothing else to build. Offline setup for air-gapped devices is covered in doc/apis.md.

**How long can a conversation get before it runs out of memory?**

It keeps a 256-token sliding window with the tools pinned as KV sinks, so RAM stays near 28MB however long the session runs — but the window itself is small. For large tool catalogues a built-in retrieval head renders only the top five tools per turn.

**Does it send any of my data anywhere?**

Inference does no network at all. Cactus collects strictly anonymous usage telemetry — function name, package version and OS, never your prompts, outputs or data — and you can opt out with NEEDLE_TELEMETRY=0 or DO_NOT_TRACK=1.

**How does it compare to other small models?**

On the README's benchmarks it trades wins with FunctionGemma 270M, LFM2.5 230M and Apple FM while being 5x to 70x smaller and running at 2 bits against their f16.

**Can I try it in the browser before installing anything?**

Yes — the Playground lets you pick a preset, edit the tools or prompt and run, with follow-up queries continuing the same conversation. Its Finetune button runs the fine-tuning pipeline and hands back a downloadable .cact file.

**What architecture is under the hood?**

It's a Simple Attention Network: a Hadamard MLP replaces the FFN, plus GQA attention, engram key-value memory and multi-lane hyper-connections. The design and ablations are in the paper at arXiv:2607.18363.

**Can it extract structured data from plain text, not just call tools?**

Yes — declare the shape as a Pydantic model and call extract(); you get a typed object back. Raw JSON schemas and driving the loop manually with complete() are covered in doc/apis.md.


---

## README (checked against GitHub 2026-11-07)

Source: https://github.com/cactus-compute/cactus/blob/main/README.md

![Cactus Compute - Sub-50mb Super Intelligence For Consumer Devices](assets/banner.svg)

Run text, speech and vision models locally with one Python package. Cactus provides tool calling, structured extraction, transcription, image understanding and model fine-tuning.

## Install

```sh
pip install -U cactus-compute
```

Python 3.9 or later. The package downloads and caches the engine and requested model weights on first use. Inference runs locally and does not require JAX.

## Agents

[Needle 3](https://cactuscompute.com/needle) calls tools, extracts structured data and embeds text.

```python
import cactus

@cactus.tool
def get_weather(city: str):
    "Get the current weather for a city."
    return {"city": city, "temp_c": 27, "sky": "clear"}

with cactus.Cactus(tools=[get_weather]) as agent:
    print(agent.run("What's the weather in Lagos?")["results"])
```

`run()` executes your functions and returns their results. Use `complete()` when you want to inspect the proposed calls before executing them.

## Voice

[Whistle](https://cactuscompute.com/blog/whistle) transcribes speech in seven languages, with word timestamps and speech embeddings.

```python
import cactus

print(cactus.transcribe("clip.wav")["text"])
```

Use a 16 kHz WAV file. Install `cactus-compute[mic]` for other sample rates or microphone capture.

## Vision

[Mirage](https://huggingface.co/Cactus-Compute/mirage) describes images and extracts structured data.

```python
import cactus

print(cactus.Mirage().describe("photo.png"))
```

## Documentation

- [Python API and CLI](https://cactuscompute.com/blog/needle-python-docs)
- [Runtime binaries and supported platforms](https://huggingface.co/Cactus-Compute/cactus)
- [Fine-tune Needle](https://cactuscompute.com/blog/finetuning-needle) or [Whistle](https://cactuscompute.com/blog/finetuning-whistle) with `cactus-compute[train]`
- [Reference for coding assistants](llms.txt)

Migrating from `cactus-needle`? Use `import cactus` and the `cactus` command. Needle 2 remains on the [legacy branch](https://github.com/cactus-compute/cactus/tree/legacy) with `cactus-compute<4`. The earlier general inference SDK is now [cactus-engine](https://pypi.org/project/cactus-engine/).

Anonymous Python usage counts can be disabled with `CACTUS_TELEMETRY=0`. See [LICENSE](LICENSE) for the Apache 2.0 license.