hypoarena README

The project's README from github.com/OpSafari/hypoarena. As one markdown file: README.md.

hypoarena

hypoarena is a fully offline scientific hypothesis-discovery workbench. It decomposes the mechanism layer of generate–debate–evolve "AI co-scientist" pipelines into individually testable components: a citation-bearing hypothesis/evidence graph, a synthetic literature factory, span-level grounding verification, pluggable agent adapters, Bradley–Terry / Elo tournaments, paraphrase deduplication (TF-IDF, MinHash LSH), hypothesis evolution operators, and Bayesian evidence accumulation.

At runtime the package depends only on NumPy. PyTorch (CPU-only) is an optional extra used solely to demonstrate a small trainable ranker. Every default test and example runs without network access, without calling real models, and without downloading weights.

Installation

python -m venv .venv
.venv/bin/pip install -e ".[dev]"          # development env (hatchling, pytest, ruff, mypy)
.venv/bin/pip install -e ".[dev,torch]" \
    --extra-index-url https://download.pytorch.org/whl/cpu   # when the optional ranker is needed

Quick start

.venv/bin/hypoarena demo --chains 2 --chain-length 2 --out /tmp/hya

This runs the complete pipeline offline over a synthetic corpus, prints whether each planted causal chain was recovered, and writes report.md / report.html. It is a mechanism demonstration and makes no claim about real scientific discovery capability. A typical run reports every planted link (e.g. planted links: 6 recovered: 6 rate: 1.0 on the demo corpus) and ends with an explicit note that it is a synthetic demonstration only.

Common commands

Command Purpose
make build Build a wheel with the local venv (--no-isolation)
make test Fast test suite (excludes the slow mark)
make test-all Full suite (includes slow / model tests)
make format / make format-check ruff formatting and its check mode
make lint ruff check + format --check
make typecheck mypy static analysis
make demo End-to-end offline demonstration

CLI

The hypoarena command-line tool exposes one subcommand per pipeline stage, in pipeline order: corpus, generate, verify, dedup, debate, rank, evolve, accumulate, report, and demo. All of them run offline over synthetic corpora. One-shot demonstration:

$ hypoarena demo --chains 2 --chain-length 2 --out /tmp/hya

Subcommand options, shared arguments, and exit codes are documented in docs/cli.md.

What each stage does

Artifact stability

All JSONL/JSON artifacts are written through a single canonical serializer (sorted keys, fixed separators, floats quantized at their production points), so run outputs — and the golden digests pinned over them — are byte-identical across platforms and interpreter versions. Checkpointed runs replay completed stages from the ledger and produce artifacts byte-identical to straight runs.

Examples and documentation

examples/ contains three fully offline, runnable examples — planted-corpus generation with grounding verification, a full generate→debate→rank tournament showing Elo recovery of a planted skill order, and an evolution + dedup cycle with measured novelty statistics. Each ships with its own README and real captured output; see docs/examples.md for the overview.

Architecture, schemas, per-module notes, and the honesty conventions live under docs/; start the module index at docs/api.md.

License

MIT — see LICENSE. Release history is in CHANGELOG.md.

hypoarena (OpSafari/hypoarena)

Searching Database…

555 ·GitHub
Hypoarena
What is this?hypoarena is a fully offline workbench that decomposes the generate-debate-evolve 'AI co-scientist' pipeline into individually testable mechanisms: a citation-bearing hypothesis-evidence graph, a synthetic literature factory, grounding verification, pluggable agent adapters, Elo tournaments, paraphrase dedup, and Bayesian evidence accumulation.
Is this a Python library or a command-line tool?Both — it is a NumPy-based Python package you can import, and it ships a `hypoarena` CLI with one subcommand per pipeline stage (corpus, generate, verify, dedup, debate, rank, evolve, accumulate, report, demo).
How do I try it out?Run `make demo` for a one-shot end-to-end offline demonstration, or run the individual CLI stages (corpus -> generate/debate/evolve -> verify -> dedup -> rank -> accumulate -> report) yourself; this produces report.md and report.html.
What do I need to install to run it?At runtime it only needs NumPy. PyTorch (CPU-only) is an optional extra used just to demo a small trainable ranker — no GPU or network access is required.
Does hypoarena actually discover real scientific hypotheses?No — every default test and example runs on synthetic corpora without calling real models or the network; the project explicitly states it is a mechanism demonstration, not a claim about real scientific discovery capability.
How does the Elo/tournament ranking actually work?Hypotheses are judged round-robin on rubric scores (novelty, testability, grounding, consistency) using Bradley-Terry/Elo updates with draw handling and K-factor decay; property tests confirm ratings recover a planted skill order within tolerance.
How does the paraphrase deduplication detect near-duplicate hypotheses?It combines normalized-text hashing, character/word n-gram Jaccard similarity, TF-IDF cosine similarity, and MinHash LSH, with documented false-positive/false-negative behavior on planted paraphrase clusters.
Are results reproducible across different machines?Yes — all JSON/JSONL artifacts go through a single canonical serializer (sorted keys, fixed separators, quantized floats), so outputs and their golden digests are byte-identical across platforms and Python interpreter versions.
What kinds of agents can I plug into the debate loop?The generate-debate-evolve loop uses a pluggable propose/critique/revise agent protocol, supporting scripted, replay, or loopback-mock agents for fully offline runs.
What are similar projects?livenerf, sktime, context-language-models, KISS-Knowledge-Infrastructure-for-Scientific-Simulation, strands-decider, scikit-learn, jevals, codex-claude-academic-skills
Which topics is it in?deduplication, reproducible-research, graph, statistics, numpy, simulation, academic, algorithms, evaluation, nlp, data-science, offline
Creator
OpSafari
Size
1.5 MB
Stars
555
Updated
CommitOct 01Releasev0.1.4Oct 09
Age
20d
Commits · 90 Days