robotjs README

The project's README from github.com/octalmage/robotjs. As one markdown file: README.md.

Node.js Desktop Automation. Control the mouse, keyboard, and read the screen.

RobotJS supports Mac, Windows, and Linux.

This is a work in progress so the exported functions could change at any time before the first stable release (1.0.0). Ideas?

Check out some of the cool things people are making with RobotJS! Have your own rad RobotJS project? Feel free to add it!

Contents

Installation

Install RobotJS using npm:

npm install robotjs

You can get npm here if you don't have it installed.

Published packages include Node-API prebuilds for Linux, macOS, and Windows on x64 and arm64. Other targets fall back to a source build.

If you need to build RobotJS, see the building section. Instructions for Electron.

Examples

Mouse

// Move the mouse across the screen as a sine wave.
const robot = require("robotjs");

// Speed up the mouse.
robot.setMouseDelay(2);

const twoPI = Math.PI * 2;
const screenSize = robot.getScreenSize();
const height = (screenSize.height / 2) - 10;
const width = screenSize.width;

for (let x = 0; x < width; x++) {
	const y = height * Math.sin((twoPI * x) / width) + height;
	robot.moveMouse(x, y);
}
Keyboard
// Type "Hello World" then press enter.
const robot = require("robotjs");

// Type "Hello World".
robot.typeString("Hello World");

// Press enter.
robot.keyTap("enter");
Screen
// Get pixel color under the mouse.
const robot = require("robotjs");

// Get mouse position.
const mouse = robot.getMousePos();

// Get pixel color in hex format.
const hex = robot.getPixelColor(mouse.x, mouse.y);
console.log(`#${hex} at x:${mouse.x} y:${mouse.y}`);

const robot = require("robotjs");

const screen = robot.screen.capture();
const target = robot.image.load("./target.bmp");
const match = screen.findImage(target, { tolerance: 0.1 });

if (match) {
	screen.click(match, target);
}

Captured and loaded images also provide findImages, countImage, findColor, findColors, countColor, colorAt, and save. Image searches return the target's top-left capture coordinates. click converts those coordinates to screen coordinates and clicks the target's center. Read the Wiki for more information!

API

The RobotJS API is hosted at https://robotjs.dev/docs/syntax.

macOS permissions

getAccessibilityPermission() and getScreenCapturePermission() report the current grants. requestAccessibilityPermission() and requestScreenCapturePermission() trigger the corresponding macOS system prompts. macOS still requires the user to approve each request. The Accessibility prompt is asynchronous, so requestAccessibilityPermission() returns the current grant. Check getAccessibilityPermission() again after the user responds.

Building

Please ensure you have the required dependencies before installing:

BMP image loading and saving is always available. PNG is enabled in all published prebuilds: Windows uses Windows Imaging Component, while macOS and Linux include statically linked libpng. PNG remains optional for macOS and Linux source builds. To enable it, install libpng and pkg-config, then force a source build with ROBOTJS_ENABLE_PNG=1. Check robot.image.supportsPNG at runtime.

Install node-gyp using npm:

npm install -g node-gyp

Then build:

node-gyp rebuild

See the node-gyp readme for more details.

Packaging prebuilds

The Prebuilds workflow builds the release binaries and uploads a complete npm-package artifact. It does not publish to npm. Download the artifact, then publish it manually:

npm publish ./robotjs-<version>.tgz

Plans

Progress

Module Status Notes
Mouse 100% All planned features implemented.
Keyboard 100% All planned features implemented.
Screen 100% Screen capture, image search, and pixel search.
Bitmap 100% BMP I/O and optional PNG support.

FAQ

Does RobotJS support global hotkeys?

Not currently, and I don't know if it ever will. I personally use Electron/NW.js for global hotkeys, and this works well. Later on I might add hotkey support or create a separate module. See #55 for details.

Can I take a screenshot with RobotJS?

Yes. robot.screen.capture() captures the main display. Pass x, y, width, height to capture a specific rectangle.

Why is <insert key> missing from the keyboard functions?

We've been implementing keys as we need them. Feel free to create an issue or submit a pull request!

How about multi-monitor support?

Use robot.getDisplays() to inspect displays and pass a rectangle from one display to robot.screen.capture(x, y, width, height). A capture rectangle cannot span multiple displays. Linux reports the current X11 screen.

For any other questions please submit an issue.

Story

I'm a huge fan of AutoHotkey, and I've used it for a very long time. AutoHotkey is great for automation and it can do a bunch of things that are very difficult in other languages. For example, it's imagesearch and pixel related functions are hard to reproduce on Mac, especially in scripting languages. These functions are great for automating apps that can't be automated like Netflix. This has never been a big deal since I've always used Windows at work, but for the past few years I've been using Mac exclusively.

I like AutoHotkey, but I like Node.js more. By developing RobotJS I get an AutoHotkey replacement on Mac (finally!), and I get to use my favorite language.

TLDR: There's nothing like AutoHotkey on Mac, so I'm making it.

License

MIT

Based on autopy. Maintained by Jason Stallings.

robotjs (octalmage/robotjs)

Searching Database…

12.8K ·GitHub
Robotjs
What is this?RobotJS is a Node.js library for desktop automation: it controls the mouse and keyboard, and reads pixels from the screen.
Is this a standalone app or a library I add to a Node project?It is a native Node.js library (a Node-API addon written in C) that you require into your own project, not a standalone app.
How do I add it to a project?Install it with npm; prebuilt Node-API binaries are included for Linux, macOS, and Windows on x64 and arm64, so no build step is needed on those targets.
Do I need a compiler or other tools before installing?Only if you fall outside the prebuilt targets: Windows needs a Visual Studio C++ toolchain, macOS needs Xcode Command Line Tools, and Linux needs Python 3, make, a C/C++ compiler, and libxtst-dev.
Is there anything that doesn't work out of the box, like global hotkeys or screenshots across monitors?Global hotkeys are not supported (the author uses Electron/NW.js for that), and a single screen capture rectangle cannot span multiple monitors. On macOS you also have to grant Accessibility and Screen Capture permissions through system prompts.
Which operating systems does it run on?RobotJS supports Mac, Windows, and Linux.
What can it actually do with images and colors on screen?It can search for images and colors in a captured screenshot via functions like findImages, countImage, findColor, findColors, countColor, and colorAt, and it can load/save bitmap captures to BMP (and PNG, if enabled).
Does it support PNG screenshots?PNG is enabled in all published prebuilds (Windows Imaging Component or statically linked libpng), and is optional for source builds on macOS/Linux by installing libpng and pkg-config and building with ROBOTJSENABLEPNG=1.
Can I take a screenshot of just part of the screen?Yes, robot.screen.capture() captures the main display by default, but you can pass x, y, width, height to capture a specific rectangle, including on a specific monitor via robot.getDisplays().
Why did the author build RobotJS instead of just using AutoHotkey?The author liked AutoHotkey but moved to using a Mac full-time, where no equivalent tool exists, so RobotJS was built to bring AutoHotkey-style automation to Node.js across platforms, especially macOS.
What are similar projects?robotgo, agent-desktop, typesafe-computer-use, Codex-Minecraft-Gameplay, pyautogui, midscene, MaaEnd, self-operating-computer
Which topics is it in?input, desktop-automation, gui-automation, screen-capture, cli-tool, image-processing, c, gui, npm, automation, nodejs, library
Creator
octalmage
Size
1.2 MB
Stars
12.8K
Updated
CommitAug 07Releasev0.9.1Aug 07
Age
12y
Commits · 90 Days