Agent-S README

The project's README from github.com/simular-ai/Agent-S. As one markdown file: README.md.

Logo Agent S: Open Computer Use

An open-source computer use agent that can operate a real GUI like a human or work in the way beyond human capacity.

🏆 Sai tops OSWorld 2.0, beating GPT and Opus at lower cost (73%)  ·  🏆 Agent S3 is the first to surpass human performance on OSWorld (72.60%)

  simular-ai%2FAgent-S | Trendshift

Windows macOS Linux Discord    PyPI Downloads

Deutsch | Español | français | 日本語 | 한국어 | Português | Русский | 中文

Agent S is an open-source computer use agent framework from Simular. Give it a task in plain English. It looks at the screen and completes the task by clicking, typing, and scrolling in ordinary desktop and web apps. It runs on macOS, Windows, and Linux.

🥳 Updates

Older updates
  • 2026/08/28: Sai tops OSWorld 2.0, beating GPT and Opus with lower costs (73%) — our production computer use agent is now state of the art on the hardest desktop benchmark!
  • 2026/07/30: The Agent S3 paper "Scaling Agents for Computer Use" is accepted to TMLR 2026!
  • 2025/12/15: Agent S3 is the first to surpass human-level performance on OSWorld with an impressive score of 72.60%!
  • 2025/10/02: Released Agent S3 and its technical paper, setting a new SOTA of 69.9% on OSWorld (approaching 72% human performance), with strong generalizability on WindowsAgentArena and AndroidWorld! It is also simpler, faster, and more flexible.
  • 2025/08/01: Agent S2.5 is released (gui-agents v0.2.5): simpler, better, and faster! New SOTA on OSWorld-Verified!
  • 2025/07/07: The Agent S2 paper is accepted to COLM 2025! See you in Montreal!
  • 2025/04/27: The Agent S paper won the Best Paper Award 🏆 at ICLR 2025 Agentic AI for Science Workshop!
  • 2025/04/01: Released the Agent S2 paper with new SOTA results on OSWorld, WindowsAgentArena, and AndroidWorld!
  • 2025/03/12: Released Agent S2 along with v0.2.0 of gui-agents, the new state-of-the-art for computer use agents (CUA), outperforming OpenAI's CUA/Operator and Anthropic's Claude 3.7 Sonnet Computer-Use!
  • 2025/01/22: The Agent S paper is accepted to ICLR 2025!
  • 2025/01/21: Released v0.1.2 of gui-agents library, with support for Linux and Windows!
  • 2024/12/05: Released v0.1.0 of gui-agents library, allowing you to use Agent-S for Mac, OSWorld, and WindowsAgentArena with ease!
  • 2024/10/10: Released the Agent S paper and codebase!

🚀 Quick Start

📚 Agent S Versions

Version What it is Docs Paper
Agent S4 Evolves the tool interfaces a computer use agent works through, from its own task experience. Open-source state of the art on OSWorld V2 (21.3% with Qwen3.8-27B). README arXiv 2026
Agent S3 The agent in gui-agents today. First to surpass human performance on OSWorld (72.60%). README TMLR 2026
Agent S2.5 A simpler, faster Agent S2. README —
Agent S2 A compositional generalist-specialist framework. README COLM 2025
Agent S The first version: uses computers like a human, through an agent-computer interface. README ICLR 2025

💬 Citations

If you find this codebase useful, please cite:

@misc{Agent-S4,
      title={Evolving Agent Interfaces}, 
      author={Saaket Agashe and Yuzhe Yang and Gonzalo Gonzalez-Pumariega and Bhuvan Chandra Koduru and Chih-Lun Lee and Jiachen Yang and Ang Li and Xin Eric Wang},
      year={2026},
      eprint={2610.28601},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2610.28601}, 
}

@misc{Agent-S3,
      title={The Unreasonable Effectiveness of Scaling Agents for Computer Use}, 
      author={Gonzalo Gonzalez-Pumariega and Vincent Tu and Chih-Lun Lee and Jiachen Yang and Ang Li and Xin Eric Wang},
      year={2025},
      eprint={2510.02250},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2510.02250}, 
}

@misc{Agent-S2,
      title={Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents}, 
      author={Saaket Agashe and Kyle Wong and Vincent Tu and Jiachen Yang and Ang Li and Xin Eric Wang},
      year={2025},
      eprint={2504.00906},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2504.00906}, 
}

@inproceedings{Agent-S,
    title={{Agent S: An Open Agentic Framework that Uses Computers Like a Human}},
    author={Saaket Agashe and Jiuzhou Han and Shuyu Gan and Jiachen Yang and Ang Li and Xin Eric Wang},
    booktitle={International Conference on Learning Representations (ICLR)},
    year={2025},
    url={https://arxiv.org/abs/2410.08164}
}

Star History

Star History Chart

Agent-S (simular-ai/Agent-S)

grounding
Searching Database…

What is this?Agent S is an open-source agent framework that controls a real desktop GUI - clicking, typing, and scrolling - to complete natural-language tasks the way a human would, without per-app API integrations.
Is this a ready-made app or something I wire into my own code?It's a Python framework/library (the gui-agents package) with both a CLI and an SDK (the AgentS3 class) - you run it from the terminal or import it into your own scripts. Sai is a separate, hosted commercial product built on the same research if you'd rather not run anything yourself.
How do I install and run it?Install the gui-agents package with pip, or clone the repo and install it locally if you want to modify the code; on macOS also run brew install tesseract for OCR support. Then launch the CLI with flags like --provider, --model, --groundprovider, --groundurl and --groundmodel, or drive the AgentS3 class directly from the SDK.
Which models do I need to supply?A main reasoning model (OpenAI, Anthropic, Gemini, Azure OpenAI, Open Router, or vLLM) plus a separate grounding model that turns screen understanding into click coordinates - the README recommends UI-TARS-1.5-7B hosted on Hugging Face Inference Endpoints.
What are the big warnings before I let it loose on my machine?The agent runs real Python code to control your computer, and its optional local coding environment executes arbitrary Python/Bash with your own user permissions - the README says to only use it in trusted environments. It's also designed for single-monitor setups only.
Is this meant for personal automation or is it a research project?It's primarily a research framework for building and benchmarking computer-use agents on tests like OSWorld; for a polished, managed agent the authors point users to their commercial product Sai instead.
Does it work the same on Mac, Windows, and Linux?Yes, Agent S3 officially supports Linux, Mac, and Windows.
What is OSWorld, and how good is Agent S3 on it?OSWorld is a benchmark of long, realistic desktop tasks; Agent S3 alone scores 66% and climbs to 72.6% with Behavior Best-of-N, surpassing the roughly 72% human baseline.
What is Sai, and how does it relate to this repo?Sai is Simular's separate hosted, production computer-use agent built on Agent S's ideas; it reached 73% on the newer OSWorld 2.0 benchmark, ahead of GPT-5.6 Sol, and targets people who want a managed service instead of running the research framework themselves.
Why does it need two different models?The main model does high-level planning from screenshots and instructions, while a separate grounding model (like UI-TARS-1.5-7B) converts that plan into precise pixel coordinates for clicks and keystrokes.
Can it run code instead of just clicking around?Yes - an optional local coding environment lets the agent call a callcodeagent action to execute Python or Bash directly for tasks like file manipulation or data processing, though it runs with the same permissions as your user account.
Creator
simular-ai
Size
60 MB
Stars
13K
Age
2y
Commits · 90 Days