TacClaw-Infinity README

The project's README from github.com/daimon-robotics/TacClaw-Infinity. As one markdown file: README.md.

TacClaw-Infinity

A Large-Scale Vision-Tactile-Language-Action Dataset
for Dexterous Manipulation

Project website arXiv:2610.20320 Hugging Face dataset ModelScope dataset

Highlights · News · Dataset · Code · Data Plan · To-do List · More Works · Citation

TacClaw-Infinity overview: collection interface, multimodal observations, and cross-embodiment transfer

TacClaw-Infinity is a large-scale Vision-Tactile-Language-Action (VTLA) dataset for contact-rich manipulation. Demonstrations are collected with TacClaw, a tactile-enabled handheld interface that records synchronized visual observations, fingertip tactile signals, and end-effector trajectories without requiring a target robot during collection.

✨ Highlights

📊 Large Scale

Devices Tasks Hours Episodes Frames
200+ 1,000+ 10,000+ 1 million+ 1 billion+

🌍 High Diversity

Worldwide collection contributed by 20+ organizations across 50+ locations, spanning homes, laboratories, hotels, industrial workplaces, retail stores, outdoor environments, and more.

🖐️ Omni Modality

High-resolution Tactile + Vision + Language + Action

Four synchronized modalities connect contact, perception, instruction, and motion for contact-rich manipulation.

📢 News

🗂️ Dataset

A subset of TacClaw-Infinity is now publicly available on Hugging Face and ModelScope under Daimon-Infinity.

Platform Dataset portal
🤗 Hugging Face Daimon-Infinity
🧩 ModelScope Daimon-Infinity

🤖 Robot Replay and Evaluation

Robot guides, setup, and evaluation commands

Tools for replaying LeRobot v3 trajectories, evaluating inverse-kinematics reachability and manipulability, and visualizing robot workspaces are organized by robot in robot_executability_checks/:

Robot Directory and guide Main workflow
Tianji tianji/ Aik7 trajectory replay and dual-arm kinematic evaluation
DM-Sparky dm-sparky/ Fixed-torso evaluation and 18-DoF base, knee, waist, and dual-arm solving
RealMan RM75-B realman/ Dual-arm assembly, episode evaluation, and replay video rendering

Each directory contains its own model assets, configuration, dependencies, scripts, and tests. Run commands from the selected robot directory and keep its Python environment separate. The original compatibility workflows are retained; use the robot-specific commands below.

Quick start

Requirements: Python with venv, CMake, a C++17 compiler, and Eigen3 headers (libeigen3-dev on Debian/Ubuntu). Video rendering also requires FFmpeg.

git clone https://github.com/daimon-robotics/TacClaw-Infinity.git
cd TacClaw-Infinity/robot_executability_checks/tianji  # or dm-sparky / realman
./setup.sh
source .venv/bin/activate
export LEROBOT_DATASET=/absolute/path/to/your_lerobot_v3_dataset

Before evaluating a dataset, check the selected robot guide's coordinate-frame, TCP, and scene-registration requirements. LeRobot v3 describes the data format; it does not guarantee the required calibration frame.

Tianji — from robot_executability_checks/tianji/:

./evaluate.sh "$LEROBOT_DATASET" outputs/tianji_arm_quality --episodes all
python scripts/replay_lerobot.py --dataset "$LEROBOT_DATASET" --episode 0 --dry-run

DM-Sparky — from robot_executability_checks/dm-sparky/:

./evaluate.sh "$LEROBOT_DATASET" outputs/dm_sparky_quality --robot mobile-humanoid
# Allow base, knee, and waist motion in the kinematic solver:
python scripts/evaluate_humanoid_mobility.py \
  --dataset "$LEROBOT_DATASET" --episodes all \
  --output outputs/dm_sparky_mobility

RealMan RM75-B — from robot_executability_checks/realman/:

python scripts/evaluate_realman_episode.py \
  --dataset "$LEROBOT_DATASET" \
  --urdf models/ros_packages/dual_rm75_b_description/urdf/dual_rm75_b.urdf \
  --scene-config config/realman_dm_tr01_scene.json \
  --episode 0 --output outputs/realman_ep0_quality

These tools provide simulation and kinematic analysis. They do not measure policy task success or certify real-robot safety. Policy training and task-level evaluation code remain forthcoming. See the import notes for source versions and directory isolation.

📈 Data Plan

The current collection comprises 10,000+ hours and 1 million+ episodes. Future collection milestones will be listed below once their quantities and dates are confirmed.

Collection milestone Cumulative hours Cumulative episodes Target date Status
Current collection 10,000+ 1 million+ — Current scale
Next collection milestone To be announced To be announced To be announced Planned
Longer-term expansion To be announced To be announced To be announced Planned

Collection totals and publicly downloadable data may differ; dataset availability is listed on the two portals above.

✅ To-do List

Robot replay and kinematic evaluation tools are available in the Code section. Additional data-processing and policy-learning utilities will be announced here.

🔎 More Works

Explore more projects from Daimon Robotics.

📄 Paper

Read the paper on arXiv or open the PDF. Visit the project website for the video, figures, and results.

📝 Citation

If you use TacClaw-Infinity in your research, please cite our arXiv paper:

@misc{zhu2026tacclawinfinity,
  title  = {TacClaw-Infinity: A Large-Scale Vision-Tactile-Language-Action Dataset for Dexterous Manipulation},
  author = {Zhu, Fucai and Zhang, Xidong and Zhang, Yichi and Wang, Yipan and Wang, Fubo and
            Wu, Zibin and Li, Kaiqi and Jiao, Jianhao and Lu, Shihan and Hu, Di and Xu, Mengdi and
            Zhu, Siyu and Long, Xiaoxiao and Yao, Yao and Cao, Xun and Xu, Zhuo and
            Hashimoto, Koichi and Chen, Qifeng and Li, Zhe and Yang, Jianfei and Lu, Haofei and
            Kragic, Danica and Yan, Shuicheng and Duan, Jianghua and Wang, Michael Yu and
            Yuan, Weihao},
  year   = {2026},
  eprint = {2610.20320},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url    = {https://arxiv.org/abs/2610.20320}
}

The project website and its video and figure collection are maintained separately in tacclaw-infinity-page.

TacClaw-Infinity (daimon-robotics/TacClaw-Infinity)

Searching Database…