Topics / #gui-agent

GUI agent projects on GitHub

Agents that operate graphical interfaces on behalf of users.

As of , the most-starred GUI agent projects on GitHub are UI-TARS-desktop (39.3K stars), cua (30K stars) and OmniParser (25.5K stars). GitHub.Locker tracks 6 GUI agent repositories with 123.2K stars combined. The fastest-growing on October 16 was cua, with +95 stars in a day.

6Projects
123.2KStars in total
+201Stars in a day
Browse in the catalog →github.com/topics/gui-agent ↗

Hot right now

Most stars gained in a day · Oct 16

+95

cua

trycua/cua · ★ 30K

An open-source stack for computer-use AI agents: desktop automation drivers, isolated cloud desktops, local macOS VMs, small specialist decision models, and benchmarks for training, evaluation, and data generation.

+66

nanoMuse

nano-muse/nanoMuse · ★ 1.4K

nanoMuse is an open-source personal AI agent that runs across every device you own. It does things rather than just answering questions, keeps working while the app is closed, remembers you, and stops to ask before doing anything irreversible.

+18

UI-TARS-desktop

bytedance/UI-TARS-desktop · ★ 39.3K

TARS is an open-source multimodal AI agent stack from ByteDance that ships two projects: Agent TARS, a CLI/Web UI agent for terminal, browser and tool-use tasks, and UI-TARS Desktop, a native app that lets a vision-language model see your screen and control mouse/keyboard.

+13

midscene

web-infra-dev/midscene · ★ 15.3K

Midscene is a vision-driven GUI Agent combined with an E2E testing kit: it looks at screenshots to act on and verify UI across web, mobile, and desktop apps through one set of Agent APIs.

The all-time top

6 projects · by stars

  1. #1 · bytedance/UI-TARS-desktop

    UI-TARS-desktop

    TARS is an open-source multimodal AI agent stack from ByteDance that ships two projects: Agent TARS, a CLI/Web UI agent for terminal, browser and tool-use tasks, and UI-TARS Desktop, a native app that lets a vision-language model see your screen and control mouse/keyboard.

    39.3K
  2. #2 · trycua/cua

    cua

    An open-source stack for computer-use AI agents: desktop automation drivers, isolated cloud desktops, local macOS VMs, small specialist decision models, and benchmarks for training, evaluation, and data generation.

    30K
  3. #3 · microsoft/OmniParser

    OmniParser

    OmniParser parses GUI screenshots into structured, labeled elements (icons, text, interactable regions) so a vision model like GPT-4V can accurately decide what to click or type on a screen.

    25.5K
  1. 4

    midscene

    15.3Kweb-infra-dev/midscene · TypeScript

    Midscene is a vision-driven GUI Agent combined with an E2E testing kit: it looks at screenshots to act on and verify UI across web, mobile, and desktop apps through one set of Agent APIs.

  2. 5

    UI-TARS

    11.6Kbytedance/UI-TARS · Python

    UI-TARS is an open-source multimodal agent model built on a vision-language model that can operate GUIs directly — clicking, typing, scrolling, dragging — across desktop apps, mobile screens, browsers, and even video games like Minecraft, reasoning about its actions before taking them.

  3. 6

    nanoMuse

    1.4Knano-muse/nanoMuse · Swift

    nanoMuse is an open-source personal AI agent that runs across every device you own. It does things rather than just answering questions, keeps working while the app is closed, remembers you, and stops to ask before doing anything irreversible.

Often tagged together