robotgo README

The project's README from github.com/go-vgo/robotgo. As one markdown file: README.md.

Robotgo

Build Status CircleCI Status Go Report Card GoDoc GitHub release Join the Discord chat at https://discord.gg/npPb3NzE4A

English | 简体中文 | 繁體中文 | 日本語 | 한국어 | Français | Deutsch | Español | Русский | Português

Golang Desktop Automation, auto test and AI Computer Use.
Control the mouse, keyboard, read the screen, process, Window Handle, image and bitmap and global event listener.

RobotGo supports Mac, Windows, and Linux; and robotgo supports arm64 and x86-amd64.

I build Codg now, Easy code and work AI agent system: auto, asynchronous, concurrency, efficiently and High accuracy

Codg Demo

RobotGo-Pro get the JavaScript, Python, Lua and others version, tech supports, new features and newest robotgo version ("no open-source version now").

Contents

Docs

Binding:

ADB, packaging android adb API.

Requirements:

Now, Please make sure Golang, GCC is installed correctly before installing RobotGo.

ALL:

Golang

GCC

For MacOS:

brew install go

Xcode Command Line Tools;
And Privacy setting, add Screen Recording and Accessibility under:
System Settings > Privacy & Security > Accessibility, Screen & System Audio Recording.

xcode-select --install

For Windows:

winget install Golang.go

llvm-mingw

winget install MartinStorsjo.LLVM-MinGW.UCRT

or Mingw-w64

winget install BrechtSanders.WinLibs.POSIX.UCRT

Or Download the Mingw-w64 and the others gcc, then set system environment variables like C:\mingw64\bin to the env Path. Set environment variables to run GCC from command line.

Or The others GCC (Except the Mingw-w64, you should compile the "libpng" with yourself when use the bitmap.)

For everything else:

GCC

X11 with the XTest extension (the Xtst library)

"Clipboard": xsel xclip

"Bitmap": libpng (Just used by the "bitmap".)

"Event-Gohook": xcb, xkb, libxkbcommon (Just used by the "hook".)
Ubuntu:
# sudo apt install golang
sudo snap install go  --classic

# gcc
sudo apt install gcc libc6-dev

# x11
sudo apt install libx11-dev xorg-dev libxtst-dev

# Clipboard
sudo apt install xsel xclip

# Bitmap
sudo apt install libpng++-dev

# GoHook
sudo apt install xcb libxcb-xkb-dev x11-xkb-utils libx11-xcb-dev libxkbcommon-x11-dev libxkbcommon-dev
Fedora:
# x11
sudo dnf install libXtst-devel

# Clipboard
sudo dnf install xsel xclip

# Bitmap
sudo dnf install libpng-devel

# GoHook
sudo dnf install libxkbcommon-devel libxkbcommon-x11-devel xkbcomp-devel
xorg-x11-xkb-utils-devel (< Fedora 34)

Wayland

The Wayland backend is a pure-Go (Cgo-free) implementation, so no system C libraries are required. It needs a wlroots-based compositor (Sway, Hyprland, Wayfire, ...) that supports the following protocols:

zwlr_virtual_pointer_v1            (mouse control)
zwp_virtual_keyboard_v1            (keyboard control)
zwlr_screencopy_v1                 (screen capture)
zwlr_foreign_toplevel_management_v1 (window management)

GNOME and KDE do not support these protocols natively.

libei (GNOME / KDE)

The libei backend is also a pure-Go (Cgo-free) implementation. It drives input through the freedesktop xdg-desktop-portal RemoteDesktop interface, so it works on GNOME and KDE (unlike the wlroots Wayland backend). It requires:

xdg-desktop-portal               (the portal D-Bus service)
xdg-desktop-portal-gnome / -kde  (your desktop's portal backend)

Note: the libei backend handles mouse and keyboard input only. Screen capture and window management report ErrNotSupported. Absolute Move uses a ScreenCast monitor stream linked to the session (the first-run consent dialog covers both); set libei.LinkScreenCast = false to opt out and keep relative motion only. The portal never exposes the physical cursor, so Location() returns the last position injected by robotgo, not movement made with the real mouse.

Installation:

With Go module support (Go 1.11+), just import:

import "github.com/go-vgo/robotgo"

Otherwise, to install the robotgo package, run the command:

go get github.com/go-vgo/robotgo

png.h: No such file or directory? Please see issues/47.

Update:

go get -u github.com/go-vgo/robotgo

Note go1.10.x C file compilation cache problem, golang #24355. go mod vendor problem, golang #26366.

Cgo-free Builds:

RobotGo ships pure-Go (Cgo-free) backends for Windows, macOS, X11, Wayland and libei (Linux), it's experimental. They expose the same robotgo API, so your code needs no changes — only a build tag. These backends cross-compile with CGO_ENABLED=0 (no GCC, MinGW, Xcode, or X11 headers required).

Backend Build tag Go package
Windows (Cgo-free) win github.com/go-vgo/robotgo/win
macOS (Quartz via purego) mac github.com/go-vgo/robotgo/darwin
X11 (Linux, pure-Go X protocol) x11 github.com/go-vgo/robotgo/x11
Wayland (Linux, wlroots) wayland github.com/go-vgo/robotgo/wayland
libei (Linux, GNOME/KDE portal) libei github.com/go-vgo/robotgo/libei
Pure-Go default (all platforms) purego selects mac/win/wayland above 
# Pure-Go default backend per platform, one tag for all targets:
# macOS -> mac, Windows -> win, Linux -> wayland (combine with x11/libei to override)
go build -tags purego .
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags "purego,x11" .

# Windows, no Cgo / no MinGW required
CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -tags win .

# macOS, Quartz/CoreGraphics loaded at runtime via purego (no Xcode required)
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -tags mac .

# X11, pure-Go X protocol (XTEST) — no X11 headers required
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags x11 .

# Wayland, wlroots-based compositor (Sway, Hyprland, Wayfire, ...)
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags wayland .

# libei, GNOME/KDE via xdg-desktop-portal RemoteDesktop
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags libei .

Note: the examples build the module root (.) rather than ./..., because examples/ and some OS-specific subpackages use APIs that are only available with the default Cgo backend.

Under the win tag the default Cgo/Win32 backend is excluded and calls are forwarded to the pure-Go win package; under the mac tag the default Cgo/Quartz backend is excluded and calls are forwarded to the pure-Go darwin package (window management reports ErrNotSupported); under the x11 tag the Cgo/X11 backend is excluded and calls are forwarded to the pure-Go x11 package; under the wayland tag the Cgo/X11 backend is excluded and calls are forwarded to the pure-Go wayland package; under the libei tag both the Cgo/X11 and wlroots Wayland backends are excluded and calls are forwarded to the pure-Go libei package.

The purego tag is a cross-platform shortcut: it excludes the Cgo backend everywhere and picks the pure-Go default for the target OS — mac on macOS, win on Windows and wayland on Linux. On Linux you can combine it with x11 or libei (e.g. -tags "purego,libei") to pick a different pure-Go backend.

Examples:

Mouse

package main

import (
  "fmt"
  "github.com/go-vgo/robotgo"
)

func main() {
  robotgo.MouseSleep = 300

  robotgo.Move(100, 100)
  fmt.Println(robotgo.Location())
  robotgo.Move(100, -200) // multi screen supported
  robotgo.MoveSmooth(120, -150)
  fmt.Println(robotgo.Location())

  robotgo.ScrollDir(10, "up")
  robotgo.ScrollDir(20, "right")

  robotgo.Scroll(0, -10)
  robotgo.Scroll(100, 0)

  robotgo.MilliSleep(100)
  robotgo.ScrollSmooth(-10, 6)
  // robotgo.ScrollRelative(10, -100)

  robotgo.Move(10, 20)
  robotgo.MoveRelative(0, -10)
  robotgo.DragSmooth(10, 10)

  robotgo.Click("wheelRight")
  robotgo.Click("left", true)
  robotgo.MoveSmooth(100, 200, 1.0, 10.0)

  robotgo.Toggle("left")
  robotgo.Toggle("left", "up")
}

Keyboard

package main

import (
  "fmt"

  "github.com/go-vgo/robotgo"
)

func main() {
  robotgo.Type("Hello World")
  robotgo.Type("だんしゃり", 0, 1)
  // robotgo.Type("テストする")

  robotgo.Type("Hi, Seattle space needle, Golden gate bridge, One world trade center.")
  robotgo.Type("Hi galaxy, hi stars, hi MT.Rainier, hi sea. こんにちは世界.")
  robotgo.Sleep(1)

  // ustr := uint32(robotgo.CharCodeAt("Test", 0))
  // robotgo.UnicodeType(ustr)

  robotgo.KeySleep = 100
  robotgo.KeyTap("enter")
  // robotgo.Type("en")
  robotgo.KeyTap("i", "alt", "cmd")

  arr := []string{"alt", "cmd"}
  robotgo.KeyTap("i", arr)

  robotgo.MilliSleep(100)
  robotgo.KeyToggle("a")
  robotgo.KeyToggle("a", "up")

  robotgo.WriteAll("Test")
  text, err := robotgo.ReadAll()
  if err == nil {
    fmt.Println(text)
  }
}

Screen

package main

import (
  "fmt"
  "strconv"

  "github.com/go-vgo/robotgo"
  "github.com/vcaesar/imgo"
)

func main() {
  x, y := robotgo.Location()
  fmt.Println("pos: ", x, y)

  color := robotgo.GetPixelColor(100, 200)
  fmt.Println("color---- ", color)

  sx, sy := robotgo.GetScreenSize()
  fmt.Println("get screen size: ", sx, sy)

  bit := robotgo.CaptureScreen(10, 10, 30, 30)
  defer robotgo.FreeBitmap(bit)

  img := robotgo.ToImage(bit)
  imgo.Save("test.png", img)

  num := robotgo.DisplaysNum()
  for i := 0; i < num; i++ {
    robotgo.DisplayID = i
    img1, _ := robotgo.CaptureImg()
    path1 := "save_" + strconv.Itoa(i)
    robotgo.Save(img1, path1+".png")
    robotgo.SaveJpeg(img1, path1+".jpeg", 50)

    img2, _ := robotgo.CaptureImg(10, 10, 20, 20)
    robotgo.Save(img2, "test_"+strconv.Itoa(i)+".png")

    x, y, w, h := robotgo.GetDisplayBounds(i)
    img3, err := robotgo.CaptureImg(x, y, w, h)
    fmt.Println("Capture error: ", err)
    robotgo.Save(img3, path1+"_1.png")
  }
}

Bitmap

package main

import (
  "fmt"

  "github.com/go-vgo/robotgo"
  "github.com/vcaesar/bitmap"
)

func main() {
  bit := robotgo.CaptureScreen(10, 20, 30, 40)
  // use `defer robotgo.FreeBitmap(bit)` to free the bitmap
  defer robotgo.FreeBitmap(bit)

  fmt.Println("bitmap...", bit)
  img := robotgo.ToImage(bit)
  // robotgo.SavePng(img, "test_1.png")
  robotgo.Save(img, "test_1.png")

  bit2 := robotgo.ToCBitmap(robotgo.ImgToBitmap(img))
  fx, fy := bitmap.Find(bit2)
  fmt.Println("FindBitmap------ ", fx, fy)
  robotgo.Move(fx, fy)

  arr := bitmap.FindAll(bit2)
  fmt.Println("Find all bitmap: ", arr)

  fx, fy = bitmap.Find(bit)
  fmt.Println("FindBitmap------ ", fx, fy)

  bitmap.Save(bit, "test.png")
}

OpenCV

package main

import (
  "fmt"
  "math/rand"

  "github.com/go-vgo/robotgo"
  "github.com/vcaesar/gcv"
  "github.com/vcaesar/bitmap"
)

func main() {
  opencv()
}

func opencv() {
  name := "test.png"
  name1 := "test_001.png"
  robotgo.SaveCapture(name1, 10, 10, 30, 30)
  robotgo.SaveCapture(name)

  fmt.Print("gcv find image: ")
  fmt.Println(gcv.FindImgFile(name1, name))
  fmt.Println(gcv.FindAllImgFile(name1, name))

  bit := bitmap.Open(name1)
  defer robotgo.FreeBitmap(bit)
  fmt.Print("find bitmap: ")
  fmt.Println(bitmap.Find(bit))

  // bit0 := robotgo.CaptureScreen()
  // img := robotgo.ToImage(bit0)
  // bit1 := robotgo.CaptureScreen(10, 10, 30, 30)
  // img1 := robotgo.ToImage(bit1)
  // defer robotgo.FreeBitmapArr(bit0, bit1)
  img, _ := robotgo.CaptureImg()
  img1, _ := robotgo.CaptureImg(10, 10, 30, 30)

  fmt.Print("gcv find image: ")
  fmt.Println(gcv.FindImg(img1, img))
  fmt.Println()

  res := gcv.FindAllImg(img1, img)
  fmt.Println(res[0].TopLeft.Y, res[0].Rects.TopLeft.X, res)
  x, y := res[0].TopLeft.X, res[0].TopLeft.Y
  robotgo.Move(x, y-rand.Intn(5))
  robotgo.MilliSleep(100)
  robotgo.Click()

  res = gcv.FindAll(img1, img) // use find template and sift
  fmt.Println("find all: ", res)
  res1 := gcv.Find(img1, img)
  fmt.Println("find: ", res1)

  img2, _, _ := robotgo.DecodeImg("test_001.png")
  x, y = gcv.FindX(img2, img)
  fmt.Println(x, y)
}

Event

package main

import (
  "fmt"

  // "github.com/go-vgo/robotgo"
  hook "github.com/robotn/gohook"
)

func main() {
  add()
  low()
  event()
}

func add() {
  fmt.Println("--- Please press ctrl + shift + q to stop hook ---")
  hook.Register(hook.KeyDown, []string{"q", "ctrl", "shift"}, func(e hook.Event) {
    fmt.Println("ctrl-shift-q")
    hook.End()
  })

  fmt.Println("--- Please press w---")
  hook.Register(hook.KeyDown, []string{"w"}, func(e hook.Event) {
    fmt.Println("w")
  })

  s := hook.Start()
  <-hook.Process(s)
}

func low() {
	evChan := hook.Start()
	defer hook.End()

	for ev := range evChan {
		fmt.Println("hook: ", ev)
	}
}

func event() {
  ok := hook.AddEvents("q", "ctrl", "shift")
  if ok {
    fmt.Println("add events...")
  }

  keve := hook.AddEvent("k")
  if keve {
    fmt.Println("you press... ", "k")
  }

  mleft := hook.AddEvent("mleft")
  if mleft {
    fmt.Println("you press... ", "mouse left button")
  }
}

Window

package main

import (
  "fmt"

  "github.com/go-vgo/robotgo"
)

func main() {
  fpid, err := robotgo.FindIds("Google")
  if err == nil {
    fmt.Println("pids... ", fpid)

    if len(fpid) > 0 {
      robotgo.Type("Hi galaxy!", fpid[0])
      robotgo.KeyTap("a", fpid[0], "cmd")

      robotgo.KeyToggle("a", fpid[0])
      robotgo.KeyToggle("a", fpid[0], "up")

      robotgo.ActivePid(fpid[0])

      robotgo.Kill(fpid[0])
    }
  }

  robotgo.ActiveName("chrome")

  isExist, err := robotgo.PidExists(100)
  if err == nil && isExist {
    fmt.Println("pid exists is", isExist)

    robotgo.Kill(100)
  }

  abool := robotgo.Alert("test", "robotgo")
  if abool {
 	  fmt.Println("ok@@@ ", "ok")
  }

  title := robotgo.GetTitle()
  fmt.Println("title@@@ ", title)
}

Authors

Plans

Contributors

License

Robotgo is primarily distributed under the terms of "the Apache License (Version 2.0)", with portions covered by various BSD-like licenses.

See LICENSE-APACHE, LICENSE.

robotgo (go-vgo/robotgo)

Searching Database…

10.9K ·GitHub
Robotgo
这是做什么的?一个用于本地跨平台桌面自动化的 Go 库:它能控制鼠标和键盘、读取屏幕、管理窗口和进程,还能监听全局事件——主要面向 RPA、自动化测试和 AI 操作电脑的场景。
它是一个我要导入到自己代码里的库,还是一个能直接运行的工具?它是一个库,不是应用:你要把 github.com/go-vgo/robotgo 导入到自己的 Go 程序里,然后调用它的 API——点击、按键、读屏、窗口句柄等。它没有独立的图形界面。
怎么把它加到项目里?如果用的是 Go modules(Go 1.11+),直接在代码里导入这个包就行;否则运行 robotgo 的包安装命令。注意一定要先装好 GCC 工具链,不然编译会失败。
编译之前需要装好哪些东西?需要 Golang 加上一个能用的 GCC:macOS 上装 Xcode Command Line Tools,Windows 上装 llvm-mingw 或 Mingw-w64,并把它的 bin 文件夹加到 Path 里。如果用其他 GCC,就得自己编译 libpng 才能获得位图支持。
有什么权限或桌面环境方面的坑吗?macOS 上需要授予屏幕录制和辅助功能权限。在 Linux 上,wlroots 的 Wayland 后端需要搭配 wlroots 系合成器(Sway、Hyprland 等)——GNOME 和 KDE 原生不支持这些协议;它们的 libei 后端只能处理输入,截屏和窗口管理会报 ErrNotSupported。
它支持哪些系统?macOS、Windows 和 Linux,arm64 和 x86-amd64 架构都支持。实验性的纯 Go 后端还扩展了 Windows、macOS、X11、Wayland 和 libei 的支持(GNOME/KDE 通过 xdg-desktop-portal)。
它全部开源吗?我能自由使用吗?这个库采用 Apache License 2.0,部分代码使用类 BSD 许可证。RobotGo-Pro 版本(JavaScript、Python、Lua、最新构建、技术支持)则不开源。
开箱即用它能做些什么?鼠标和键盘控制、读取屏幕、位图和图像处理、基于 OpenCV 的图像操作、窗口句柄管理、全局事件监听,还有一个 ADB 绑定,可以在代码里操控 Android 设备——仓库里每个方向都附带了示例。
编译它真的必须要有 GCC 和 Cgo 吗?不一定:实验性的纯 Go 后端通过构建标签(win、mac、x11、wayland、libei、purego)提供。它们暴露相同的 API,并且可以在禁用 CGO 的情况下交叉编译——不需要 GCC、MinGW、Xcode 或 X11 头文件。
有付费版或 Pro 版吗?有。RobotGo-Pro 打包了 JavaScript、Python、Lua 等语言版本,还包含技术支持、新功能和最新发布版,但它不开源。核心的 Go 库仍然以 Apache-2.0 免费提供。
为什么它主打 AI 智能体场景?操作电脑的智能体正需要这一层能力——移动鼠标、输入文字、读取屏幕。作者正在这套技术栈之上开发 Codg,一个 AI 智能体系统('auto, asynchronous, concurrency, efficiently and High accuracy')。
What are similar projects?pyautogui, robotjs, Codex-Minecraft-Gameplay, midscene, self-operating-computer, typesafe-computer-use, MaaEnd, agent-desktop
Which topics is it in?hook, input, rpa, events, golang-library, mouse, opencv, window-management, desktop-automation, robot, gui-automation, x86-64
Creator
go-vgo
Size
6.9 MB
Stars
10.9K
Updated
CommitOct 07Releasev1.3.6Oct 16
Age
10y
Commits · 90 Days