1.7K ·GitHub
What is this?
ModelTrace is a locally-run active model attribution tool: it sends three fixed challenge prompts to a target model, extracts an output fingerprint, and matches it against a known-model library to guess the model family and specific version.
ModelTrace is a locally-run active model attribution tool: it sends three fixed challenge prompts to a target model, extracts an output fingerprint, and matches it against a known-model library to guess the model family and specific version.Is this an app I install, or just a page I open?It's a static, backend-free web page (static/index.html, also deployed to GitHub Pages) plus an optional Codex plugin called ModelTrace Guard that runs background monitoring inside Codex sessions.
It's a static, backend-free web page (static/index.html, also deployed to GitHub Pages) plus an optional Codex plugin called ModelTrace Guard that runs background monitoring inside Codex sessions.How do I run a test?Open the GitHub Pages site or static/index.html locally. For manual testing, copy the three challenge prompts to the target model and paste back its outputs; for automatic testing, enter the Base URL, API Key and model name and it handles the requests itself.
Open the GitHub Pages site or static/index.html locally. For manual testing, copy the three challenge prompts to the target model and paste back its outputs; for automatic testing, enter the Base URL, API Key and model name and it handles the requests itself.Do I need an API key to use it?Only for the automatic testing mode, which supports OpenAI Chat Completions and Anthropic Messages compatible endpoints. Manual testing needs no key at all — you just paste model outputs yourself.
Only for the automatic testing mode, which supports OpenAI Chat Completions and Anthropic Messages compatible endpoints. Manual testing needs no key at all — you just paste model outputs yourself.Does it always correctly identify the model?No — results are only probabilities within the current fingerprint library, not proof. A model that isn't in the library will still be matched to whichever known model is closest, so an unfamiliar model can produce a confident-looking but wrong result. System prompts also skew results heavily, so testing inside Claude Code specifically is known to be unreliable.
No — results are only probabilities within the current fingerprint library, not proof. A model that isn't in the library will still be matched to whichever known model is closest, so an unfamiliar model can produce a confident-looking but wrong result. System prompts also skew results heavily, so testing inside Claude Code specifically is known to be unreliable.Does it run only in a browser, or do I need to install anything?It runs entirely in the browser with no backend required for the manual/API modes; only the Codex Guard plugin needs to be installed inside Codex CLI.
It runs entirely in the browser with no backend required for the manual/API modes; only the Codex Guard plugin needs to be installed inside Codex CLI.What does the Codex plugin ModelTrace Guard actually do?It monitors a running Codex task in the background by periodically forking the frozen task snapshot to run normal and re-verification probes without blocking the main task, and silently logs consistent results while alerting and pausing the agent on repeated inconsistencies.
It monitors a running Codex task in the background by periodically forking the frozen task snapshot to run normal and re-verification probes without blocking the main task, and silently logs consistent results while alerting and pausing the agent on repeated inconsistencies.What's currently in the fingerprint library?Two model families covering 17 models total: GPT data collected from an official Codex subscription, and Claude data collected via OAIPro.
Two model families covering 17 models total: GPT data collected from an official Codex subscription, and Claude data collected via OAIPro.How does the attribution algorithm score a model?All models share one global feature space; the tool projects out an estimated interference direction, scores the three responses against every model center, averages the scores, and converts them to probabilities with a global softmax plus a calibration temperature.
All models share one global feature space; the tool projects out an estimated interference direction, scores the three responses against every model center, averages the scores, and converts them to probabilities with a global softmax plus a calibration temperature.What inspired this project?The author credits hanlinwenyuan/hlwy-ai-checker, an earlier project that used LLM random-number-generation bias to check third-party API channel consistency.
The author credits hanlinwenyuan/hlwy-ai-checker, an earlier project that used LLM random-number-generation bias to check third-party API channel consistency.What are similar projects?lm-detector, Ai-jailbreak, wardrobe, strix, promptfoo, repomix, ai-guide, ruflo
lm-detector, Ai-jailbreak, wardrobe, strix, promptfoo, repomix, ai-guide, rufloWhich topics is it in?fingerprinting, detection, analysis, llm-security, github-pages, ai-security, statistics, ai-tools, gpt, reverse-engineering, artificial-intelligence, testing
fingerprinting, detection, analysis, llm-security, github-pages, ai-security, statistics, ai-tools, gpt, reverse-engineering, artificial-intelligence, testing