robotgo README

The project's README from github.com/go-vgo/robotgo. As one markdown file: README.md.

Robotgo

Build Status CircleCI Status Go Report Card GoDoc GitHub release Join the Discord chat at https://discord.gg/npPb3NzE4A

English | 简体中文 | 繁體中文 | 日本語 | 한국어 | Français | Deutsch | Español | Русский | Português

Golang Desktop Automation, auto test and AI Computer Use.
Control the mouse, keyboard, read the screen, process, Window Handle, image and bitmap and global event listener.

RobotGo supports Mac, Windows, and Linux; and robotgo supports arm64 and x86-amd64.

I build Codg now, Easy code and work AI agent system: auto, asynchronous, concurrency, efficiently and High accuracy

Codg Demo

RobotGo-Pro get the JavaScript, Python, Lua and others version, tech supports, new features and newest robotgo version ("no open-source version now").

Contents

Docs

Binding:

ADB, packaging android adb API.

Requirements:

Now, Please make sure Golang, GCC is installed correctly before installing RobotGo.

ALL:

Golang

GCC

For MacOS:

brew install go

Xcode Command Line Tools;
And Privacy setting, add Screen Recording and Accessibility under:
System Settings > Privacy & Security > Accessibility, Screen & System Audio Recording.

xcode-select --install

For Windows:

winget install Golang.go

llvm-mingw

winget install MartinStorsjo.LLVM-MinGW.UCRT

or Mingw-w64

winget install BrechtSanders.WinLibs.POSIX.UCRT

Or Download the Mingw-w64 and the others gcc, then set system environment variables like C:\mingw64\bin to the env Path. Set environment variables to run GCC from command line.

Or The others GCC (Except the Mingw-w64, you should compile the "libpng" with yourself when use the bitmap.)

For everything else:

GCC

X11 with the XTest extension (the Xtst library)

"Clipboard": xsel xclip

"Bitmap": libpng (Just used by the "bitmap".)

"Event-Gohook": xcb, xkb, libxkbcommon (Just used by the "hook".)
Ubuntu:
# sudo apt install golang
sudo snap install go  --classic

# gcc
sudo apt install gcc libc6-dev

# x11
sudo apt install libx11-dev xorg-dev libxtst-dev

# Clipboard
sudo apt install xsel xclip

# Bitmap
sudo apt install libpng++-dev

# GoHook
sudo apt install xcb libxcb-xkb-dev x11-xkb-utils libx11-xcb-dev libxkbcommon-x11-dev libxkbcommon-dev
Fedora:
# x11
sudo dnf install libXtst-devel

# Clipboard
sudo dnf install xsel xclip

# Bitmap
sudo dnf install libpng-devel

# GoHook
sudo dnf install libxkbcommon-devel libxkbcommon-x11-devel xkbcomp-devel
xorg-x11-xkb-utils-devel (< Fedora 34)

Wayland

The Wayland backend is a pure-Go (Cgo-free) implementation, so no system C libraries are required. It needs a wlroots-based compositor (Sway, Hyprland, Wayfire, ...) that supports the following protocols:

zwlr_virtual_pointer_v1            (mouse control)
zwp_virtual_keyboard_v1            (keyboard control)
zwlr_screencopy_v1                 (screen capture)
zwlr_foreign_toplevel_management_v1 (window management)

GNOME and KDE do not support these protocols natively.

libei (GNOME / KDE)

The libei backend is also a pure-Go (Cgo-free) implementation. It drives input through the freedesktop xdg-desktop-portal RemoteDesktop interface, so it works on GNOME and KDE (unlike the wlroots Wayland backend). It requires:

xdg-desktop-portal               (the portal D-Bus service)
xdg-desktop-portal-gnome / -kde  (your desktop's portal backend)

Note: the libei backend handles mouse and keyboard input only. Screen capture and window management report ErrNotSupported. Absolute Move uses a ScreenCast monitor stream linked to the session (the first-run consent dialog covers both); set libei.LinkScreenCast = false to opt out and keep relative motion only. The portal never exposes the physical cursor, so Location() returns the last position injected by robotgo, not movement made with the real mouse.

Installation:

With Go module support (Go 1.11+), just import:

import "github.com/go-vgo/robotgo"

Otherwise, to install the robotgo package, run the command:

go get github.com/go-vgo/robotgo

png.h: No such file or directory? Please see issues/47.

Update:

go get -u github.com/go-vgo/robotgo

Note go1.10.x C file compilation cache problem, golang #24355. go mod vendor problem, golang #26366.

Cgo-free Builds:

RobotGo ships pure-Go (Cgo-free) backends for Windows, macOS, X11, Wayland and libei (Linux), it's experimental. They expose the same robotgo API, so your code needs no changes — only a build tag. These backends cross-compile with CGO_ENABLED=0 (no GCC, MinGW, Xcode, or X11 headers required).

Backend Build tag Go package
Windows (Cgo-free) win github.com/go-vgo/robotgo/win
macOS (Quartz via purego) mac github.com/go-vgo/robotgo/darwin
X11 (Linux, pure-Go X protocol) x11 github.com/go-vgo/robotgo/x11
Wayland (Linux, wlroots) wayland github.com/go-vgo/robotgo/wayland
libei (Linux, GNOME/KDE portal) libei github.com/go-vgo/robotgo/libei
Pure-Go default (all platforms) purego selects mac/win/wayland above 
# Pure-Go default backend per platform, one tag for all targets:
# macOS -> mac, Windows -> win, Linux -> wayland (combine with x11/libei to override)
go build -tags purego .
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags "purego,x11" .

# Windows, no Cgo / no MinGW required
CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -tags win .

# macOS, Quartz/CoreGraphics loaded at runtime via purego (no Xcode required)
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -tags mac .

# X11, pure-Go X protocol (XTEST) — no X11 headers required
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags x11 .

# Wayland, wlroots-based compositor (Sway, Hyprland, Wayfire, ...)
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags wayland .

# libei, GNOME/KDE via xdg-desktop-portal RemoteDesktop
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags libei .

Note: the examples build the module root (.) rather than ./..., because examples/ and some OS-specific subpackages use APIs that are only available with the default Cgo backend.

Under the win tag the default Cgo/Win32 backend is excluded and calls are forwarded to the pure-Go win package; under the mac tag the default Cgo/Quartz backend is excluded and calls are forwarded to the pure-Go darwin package (window management reports ErrNotSupported); under the x11 tag the Cgo/X11 backend is excluded and calls are forwarded to the pure-Go x11 package; under the wayland tag the Cgo/X11 backend is excluded and calls are forwarded to the pure-Go wayland package; under the libei tag both the Cgo/X11 and wlroots Wayland backends are excluded and calls are forwarded to the pure-Go libei package.

The purego tag is a cross-platform shortcut: it excludes the Cgo backend everywhere and picks the pure-Go default for the target OS — mac on macOS, win on Windows and wayland on Linux. On Linux you can combine it with x11 or libei (e.g. -tags "purego,libei") to pick a different pure-Go backend.

Examples:

Mouse

package main

import (
  "fmt"
  "github.com/go-vgo/robotgo"
)

func main() {
  robotgo.MouseSleep = 300

  robotgo.Move(100, 100)
  fmt.Println(robotgo.Location())
  robotgo.Move(100, -200) // multi screen supported
  robotgo.MoveSmooth(120, -150)
  fmt.Println(robotgo.Location())

  robotgo.ScrollDir(10, "up")
  robotgo.ScrollDir(20, "right")

  robotgo.Scroll(0, -10)
  robotgo.Scroll(100, 0)

  robotgo.MilliSleep(100)
  robotgo.ScrollSmooth(-10, 6)
  // robotgo.ScrollRelative(10, -100)

  robotgo.Move(10, 20)
  robotgo.MoveRelative(0, -10)
  robotgo.DragSmooth(10, 10)

  robotgo.Click("wheelRight")
  robotgo.Click("left", true)
  robotgo.MoveSmooth(100, 200, 1.0, 10.0)

  robotgo.Toggle("left")
  robotgo.Toggle("left", "up")
}

Keyboard

package main

import (
  "fmt"

  "github.com/go-vgo/robotgo"
)

func main() {
  robotgo.Type("Hello World")
  robotgo.Type("だんしゃり", 0, 1)
  // robotgo.Type("テストする")

  robotgo.Type("Hi, Seattle space needle, Golden gate bridge, One world trade center.")
  robotgo.Type("Hi galaxy, hi stars, hi MT.Rainier, hi sea. こんにちは世界.")
  robotgo.Sleep(1)

  // ustr := uint32(robotgo.CharCodeAt("Test", 0))
  // robotgo.UnicodeType(ustr)

  robotgo.KeySleep = 100
  robotgo.KeyTap("enter")
  // robotgo.Type("en")
  robotgo.KeyTap("i", "alt", "cmd")

  arr := []string{"alt", "cmd"}
  robotgo.KeyTap("i", arr)

  robotgo.MilliSleep(100)
  robotgo.KeyToggle("a")
  robotgo.KeyToggle("a", "up")

  robotgo.WriteAll("Test")
  text, err := robotgo.ReadAll()
  if err == nil {
    fmt.Println(text)
  }
}

Screen

package main

import (
  "fmt"
  "strconv"

  "github.com/go-vgo/robotgo"
  "github.com/vcaesar/imgo"
)

func main() {
  x, y := robotgo.Location()
  fmt.Println("pos: ", x, y)

  color := robotgo.GetPixelColor(100, 200)
  fmt.Println("color---- ", color)

  sx, sy := robotgo.GetScreenSize()
  fmt.Println("get screen size: ", sx, sy)

  bit := robotgo.CaptureScreen(10, 10, 30, 30)
  defer robotgo.FreeBitmap(bit)

  img := robotgo.ToImage(bit)
  imgo.Save("test.png", img)

  num := robotgo.DisplaysNum()
  for i := 0; i < num; i++ {
    robotgo.DisplayID = i
    img1, _ := robotgo.CaptureImg()
    path1 := "save_" + strconv.Itoa(i)
    robotgo.Save(img1, path1+".png")
    robotgo.SaveJpeg(img1, path1+".jpeg", 50)

    img2, _ := robotgo.CaptureImg(10, 10, 20, 20)
    robotgo.Save(img2, "test_"+strconv.Itoa(i)+".png")

    x, y, w, h := robotgo.GetDisplayBounds(i)
    img3, err := robotgo.CaptureImg(x, y, w, h)
    fmt.Println("Capture error: ", err)
    robotgo.Save(img3, path1+"_1.png")
  }
}

Bitmap

package main

import (
  "fmt"

  "github.com/go-vgo/robotgo"
  "github.com/vcaesar/bitmap"
)

func main() {
  bit := robotgo.CaptureScreen(10, 20, 30, 40)
  // use `defer robotgo.FreeBitmap(bit)` to free the bitmap
  defer robotgo.FreeBitmap(bit)

  fmt.Println("bitmap...", bit)
  img := robotgo.ToImage(bit)
  // robotgo.SavePng(img, "test_1.png")
  robotgo.Save(img, "test_1.png")

  bit2 := robotgo.ToCBitmap(robotgo.ImgToBitmap(img))
  fx, fy := bitmap.Find(bit2)
  fmt.Println("FindBitmap------ ", fx, fy)
  robotgo.Move(fx, fy)

  arr := bitmap.FindAll(bit2)
  fmt.Println("Find all bitmap: ", arr)

  fx, fy = bitmap.Find(bit)
  fmt.Println("FindBitmap------ ", fx, fy)

  bitmap.Save(bit, "test.png")
}

OpenCV

package main

import (
  "fmt"
  "math/rand"

  "github.com/go-vgo/robotgo"
  "github.com/vcaesar/gcv"
  "github.com/vcaesar/bitmap"
)

func main() {
  opencv()
}

func opencv() {
  name := "test.png"
  name1 := "test_001.png"
  robotgo.SaveCapture(name1, 10, 10, 30, 30)
  robotgo.SaveCapture(name)

  fmt.Print("gcv find image: ")
  fmt.Println(gcv.FindImgFile(name1, name))
  fmt.Println(gcv.FindAllImgFile(name1, name))

  bit := bitmap.Open(name1)
  defer robotgo.FreeBitmap(bit)
  fmt.Print("find bitmap: ")
  fmt.Println(bitmap.Find(bit))

  // bit0 := robotgo.CaptureScreen()
  // img := robotgo.ToImage(bit0)
  // bit1 := robotgo.CaptureScreen(10, 10, 30, 30)
  // img1 := robotgo.ToImage(bit1)
  // defer robotgo.FreeBitmapArr(bit0, bit1)
  img, _ := robotgo.CaptureImg()
  img1, _ := robotgo.CaptureImg(10, 10, 30, 30)

  fmt.Print("gcv find image: ")
  fmt.Println(gcv.FindImg(img1, img))
  fmt.Println()

  res := gcv.FindAllImg(img1, img)
  fmt.Println(res[0].TopLeft.Y, res[0].Rects.TopLeft.X, res)
  x, y := res[0].TopLeft.X, res[0].TopLeft.Y
  robotgo.Move(x, y-rand.Intn(5))
  robotgo.MilliSleep(100)
  robotgo.Click()

  res = gcv.FindAll(img1, img) // use find template and sift
  fmt.Println("find all: ", res)
  res1 := gcv.Find(img1, img)
  fmt.Println("find: ", res1)

  img2, _, _ := robotgo.DecodeImg("test_001.png")
  x, y = gcv.FindX(img2, img)
  fmt.Println(x, y)
}

Event

package main

import (
  "fmt"

  // "github.com/go-vgo/robotgo"
  hook "github.com/robotn/gohook"
)

func main() {
  add()
  low()
  event()
}

func add() {
  fmt.Println("--- Please press ctrl + shift + q to stop hook ---")
  hook.Register(hook.KeyDown, []string{"q", "ctrl", "shift"}, func(e hook.Event) {
    fmt.Println("ctrl-shift-q")
    hook.End()
  })

  fmt.Println("--- Please press w---")
  hook.Register(hook.KeyDown, []string{"w"}, func(e hook.Event) {
    fmt.Println("w")
  })

  s := hook.Start()
  <-hook.Process(s)
}

func low() {
	evChan := hook.Start()
	defer hook.End()

	for ev := range evChan {
		fmt.Println("hook: ", ev)
	}
}

func event() {
  ok := hook.AddEvents("q", "ctrl", "shift")
  if ok {
    fmt.Println("add events...")
  }

  keve := hook.AddEvent("k")
  if keve {
    fmt.Println("you press... ", "k")
  }

  mleft := hook.AddEvent("mleft")
  if mleft {
    fmt.Println("you press... ", "mouse left button")
  }
}

Window

package main

import (
  "fmt"

  "github.com/go-vgo/robotgo"
)

func main() {
  fpid, err := robotgo.FindIds("Google")
  if err == nil {
    fmt.Println("pids... ", fpid)

    if len(fpid) > 0 {
      robotgo.Type("Hi galaxy!", fpid[0])
      robotgo.KeyTap("a", fpid[0], "cmd")

      robotgo.KeyToggle("a", fpid[0])
      robotgo.KeyToggle("a", fpid[0], "up")

      robotgo.ActivePid(fpid[0])

      robotgo.Kill(fpid[0])
    }
  }

  robotgo.ActiveName("chrome")

  isExist, err := robotgo.PidExists(100)
  if err == nil && isExist {
    fmt.Println("pid exists is", isExist)

    robotgo.Kill(100)
  }

  abool := robotgo.Alert("test", "robotgo")
  if abool {
 	  fmt.Println("ok@@@ ", "ok")
  }

  title := robotgo.GetTitle()
  fmt.Println("title@@@ ", title)
}

Authors

Plans

Contributors

License

Robotgo is primarily distributed under the terms of "the Apache License (Version 2.0)", with portions covered by various BSD-like licenses.

See LICENSE-APACHE, LICENSE.

robotgo (go-vgo/robotgo)

Searching Database…

10.9K ·GitHub
Robotgo
What is this?A Go library for native cross-platform desktop automation: it drives the mouse and keyboard, reads the screen, manages windows and processes, and listens to global events — aimed at RPA, auto-testing and AI computer use.
Is it a library I import into my own code, or a tool I run?A library, not an app: you import github.com/go-vgo/robotgo into your Go program and call its API — clicks, keystrokes, screen reads, window handles. There is no standalone GUI.
How do I add it to a project?With Go modules (Go 1.11+) just import the package in your code; otherwise run the package install command for robotgo. Make sure the GCC toolchain is set up first or the build will fail.
What do I need installed before it will build?Golang plus a working GCC: Xcode Command Line Tools on macOS, llvm-mingw or Mingw-w64 on Windows with its bin folder on Path. Any other GCC requires compiling libpng yourself for bitmap support.
Any permission or desktop-environment gotchas?macOS requires Screen Recording and Accessibility permissions. On Linux the wlroots Wayland backend needs a wlroots compositor (Sway, Hyprland...) — GNOME and KDE do not support those protocols natively; their libei backend handles input only, so screen capture and window management report ErrNotSupported.
Which systems does it run on?macOS, Windows and Linux, on both arm64 and x86-amd64. Experimental pure-Go backends add coverage for Windows, macOS, X11, Wayland and libei (GNOME/KDE via xdg-desktop-portal).
Is all of it open source, and can I use it freely?The library is under the Apache License 2.0, with portions covered by BSD-like licenses. The RobotGo-Pro versions (JavaScript, Python, Lua, newest builds, tech support) are not open source.
What can it actually do out of the box?Mouse and keyboard control, screen reading, bitmap and image processing, OpenCV-based image work, window-handle management, a global event listener, and an ADB binding for driving Android devices from your code — the repo includes examples for each area.
Do I really need GCC and Cgo to build it?Not necessarily: experimental pure-Go backends ship behind build tags (win, mac, x11, wayland, libei, purego). They expose the same API and cross-compile with CGO disabled — no GCC, MinGW, Xcode or X11 headers required.
Is there a paid or Pro version?Yes. RobotGo-Pro bundles JavaScript, Python, Lua and other language versions, tech support, new features and the newest releases, but it is not open source. The core Go library stays free under Apache-2.0.
Why does it market itself for AI agents?Computer-use agents need exactly this layer — moving the mouse, typing and reading the screen. The author is building Codg, an AI agent system ('auto, asynchronous, concurrency, efficiently and High accuracy'), on top of this stack.
What are similar projects?pyautogui, robotjs, Codex-Minecraft-Gameplay, midscene, self-operating-computer, typesafe-computer-use, MaaEnd, agent-desktop
Which topics is it in?hook, input, rpa, events, golang-library, mouse, opencv, window-management, desktop-automation, robot, gui-automation, x86-64
Creator
go-vgo
Size
6.9 MB
Stars
10.9K
Updated
CommitOct 07Releasev1.3.6Oct 16
Age
10y
Commits · 90 Days