Real-World RL with GimArm
=========================
.. figure:: https://raw.githubusercontent.com/RLinf/misc/main/pic/gim-arm.png
   :align: center
   :width: 80%

   GimArm 6-DoF arm used for SocketCAN control, motor calibration, and peg-insertion training.

Run RLinf on a GimArm 6-DoF arm over SocketCAN. You will install the CAN control SDK, calibrate motors, set a target end-effector pose, collect demonstrations, and launch real-world training.

Overview
--------

Train a real-world peg-insertion policy on GimArm with CAN-based control.

.. grid:: 2 4 4 4
   :gutter: 2

   .. grid-item-card:: Models
      :text-align: center

      CNN policy

   .. grid-item-card:: Algorithms
      :text-align: center

      SAC · RLPD

   .. grid-item-card:: Tasks
      :text-align: center

      Peg insertion

   .. grid-item-card:: Hardware
      :text-align: center

      GimArm · CAN FD · gripper

| **You'll do:** install gim_arm_control → initialize CAN → calibrate zero → set target pose → train.
| **Prerequisites:** :doc:`Installation </rst_source/start/installation>` · GimArm hardware · CAN adapter · ``gim_arm_control`` SDK.

Tasks
~~~~~

.. list-table::
   :header-rows: 1
   :widths: 24 24 24

   * - Task
     - Config / entry point
     - Description
   * - Hardware check
     - ``test_gim_arm_env.py``
     - Verify controller launch, state reads, motion, reset, and gripper commands.
   * - Data collection
     - ``gim_arm_teleop``
     - Collect VR teleoperation demonstrations.
   * - Training
     - GimArm real-world config
     - Train on the peg-insertion task with target-pose reward.

Observation and Action
~~~~~~~~~~~~~~~~~~~~~~

.. list-table::
   :header-rows: 1
   :widths: 24 24

   * - Field
     - Description
   * - Observation
     - Camera frames plus GimArm joint/TCP state.
   * - Action
     - 6-DoF arm control with gripper command, depending on config.
   * - Reward
     - Sparse or dense FK distance to ``target_ee_pose``.
   * - Prompt
     - Peg-insertion task text in the env config.

Hardware Setup
----------------

.. warning::

  Ensure the controller node and the training node are in the same local network.
  The GimArm robot is connected to the controller node via CAN bus, not Ethernet.

.. note::

   Unlike the Franka setup, GimArm does **not** require a real-time kernel or ROS.
   Communication uses the Linux SocketCAN interface directly.

Installation
------------

The controller node and the training/rollout node(s) should be set up with different software dependencies.

Robot Controller Node
~~~~~~~~~~~~~~~~~~~~~~

1. Installation
^^^^^^^^^^^^^^^^^^^^^^^^^^^

a. Clone RLinf Repository
__________________________

.. code:: bash

   # For mainland China users, you can use the following for better download speed:
   # git clone https://ghfast.top/github.com/RLinf/RLinf.git
   git clone https://github.com/RLinf/RLinf.git
   cd RLinf

b. Install RLinf Dependencies
________________________________

.. code:: bash

   # For mainland China users, you can add the `--use-mirror` flag for better download speed.

   bash requirements/install.sh embodied --env gim_arm
   source .venv/bin/activate

c. Install gim_arm_control SDK
________________________________

The ``gim_arm_control`` package provides the low-level CAN communication driver and Python bindings
for controlling the GimArm robot. It also ships the helper shell scripts used in the next steps
(``sh/init_can.sh``, ``sh/set_zero.sh``), so install it before proceeding.

.. code:: bash

   # Clone the SDK alongside RLinf (the example below assumes ~/gim_arm_control).
   cd ~
   git clone https://github.com/RLinf/gim_arm_control.git
   cd ~/gim_arm_control/python
   pip install -e .

This builds the C++ core via CMake and installs Python bindings using nanobind.

**Build requirements**: ``scikit-build-core>=0.5``, ``nanobind>=2.0``, a C++17 compiler (GCC >= 7 or Clang >= 5).

**Runtime dependencies**: ``numpy``, ``pinocchio`` (imported as ``pin``).

.. note::

   ``pinocchio`` is required for forward kinematics and Jacobian computation used by the controller.
   It is automatically installed as a dependency of the SDK.
   For systems requiring NumPy 1.x compatibility, install with:

   .. code:: bash

      pip install -e ".[pin270]"

2. CAN Interface Initialization
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The CAN bus must be initialized before using the robot.
The ``gim_arm_control`` SDK (installed in the previous step) provides a convenience script,
or you can run the commands manually.

Using the script from the ``gim_arm_control`` repository:

.. code:: bash

   bash sh/init_can.sh can0

Or manually:

.. code:: bash

   sudo ip link set can0 type can bitrate 1000000 dbitrate 5000000 fd on
   sudo ip link set can0 txqueuelen 1000
   sudo ip link set can0 up

This sets a 1 Mbps standard bitrate and 5 Mbps CAN FD data bitrate.

.. warning::

  The CAN interface must be re-initialized after every system reboot.
  You can verify the interface is up with:

  .. code:: bash

     ip link show can0

3. Motor Zero Calibration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Before first use (or after replacing a motor), you must calibrate the motor zero positions.
This sets the current physical position as the zero reference for Damiao motors.

From the ``gim_arm_control`` repository:

.. code:: bash

   # Zero a single motor (CAN ID in hex)
   bash sh/set_zero.sh can0 001

   # Zero all motors (001-008)
   bash sh/set_zero.sh can0 --all

.. warning::

  Calibration should only be done with the arm in its mechanical home position.
  Incorrect calibration can cause the arm to move unexpectedly.
  Requires ``can-utils`` (install with ``sudo apt install can-utils``).

Training / Rollout Nodes
~~~~~~~~~~~~~~~~~~~~~~~~

Clone the RLinf repository (same as above), then install dependencies.

Install Dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^

**Option 1: Docker Image**

Use Docker image for the experiment.

.. code:: bash

   docker run -it --rm --gpus all \
      --shm-size 20g \
      --network host \
      --name rlinf \
      -v .:/workspace/RLinf \
      rlinf/rlinf:agentic-rlinf0.3-maniskill_libero
      # For mainland China users, you can use the following for better download speed:
      # docker.1ms.run/rlinf/rlinf:agentic-rlinf0.3-maniskill_libero

**Option 2: Custom Environment**

Install dependencies directly in your environment by running the following command:

.. code:: bash

   # For mainland China users, you can add the `--use-mirror` flag for better download speed.

   bash requirements/install.sh embodied --env gim_arm
   source .venv/bin/activate

   # To install model-specific dependencies (e.g. OpenVLA), add the --model flag:
   # bash requirements/install.sh embodied --model openvla --env maniskill_libero


Run It
------

Prerequisites
~~~~~~~~~~~~~~~

**Get the Target Pose for the Task**

To acquire the target end-effector pose for the peg-insertion task, you can use the hardware test script.

First, initialize the CAN interface (see above), then run:

.. code-block:: bash

   python toolkits/realworld_check/test_gim_arm_env.py --can can0 --variant gim_arm_xl

The script launches the controller, performs forward kinematics on the current joint positions,
and prints the TCP position and quaternion.
Manually move the arm to the desired target pose, then record the printed values.
Convert the quaternion to Euler XYZ angles for use in the ``target_ee_pose`` configuration field.

Data Collection
~~~~~~~~~~~~~~~~~

Refer to the VR teleoperation code in `gim_arm_teleop <https://github.com/RLinf/gim_arm_teleop>`_ , for data collection with the GimArm robot.
We recommend the following deployment steps:

1. Prepare the runtime environment:

   - Use Ubuntu 22.04 (x86) on the machine that controls the robot when possible;
   - Use a PICO 4 Ultra headset and keep it on the same LAN as the control PC;
   - If you need headset camera passthrough (VST), enable the corresponding device permissions.

2. Install and start the PC service (XRoboToolkit-PC-Service):

   - Download the PC service package: `XRoboToolkit_PC_Service_1.0.0_ubuntu_22.04_amd64.deb <https://github.com/XR-Robotics/XRoboToolkit-PC-Service/releases/download/v1.1.1/XRoboToolkit-PC-Service-1.1.1.deb>`_

   .. code-block:: bash

      sudo dpkg -i XRoboToolkit_PC_Service_1.0.0_ubuntu_22.04_amd64.deb
      bash /opt/apps/roboticsservice/runService.sh

3. Install and launch the XRoboToolkit app on the headset:

   - Install the APK `XRoboToolkit-PICO-1.1.1.apk <https://github.com/XR-Robotics/XRoboToolkit-Unity-Client/releases/download/v1.1.1/XRoboToolkit-PICO-1.1.1.apk>`_
   - Connect to the control PC's IP address;
   - In the app, enable ``head``\ /\ ``hand``\ /\ ``controller`` (as required by your task).

4. Install and run the Python teleoperation stack:

   Follow the `gim_arm_teleop <https://github.com/RLinf/gim_arm_teleop>`_
   , then on the control PC launch the GimArm hardware teleoperation script:

   .. code-block:: bash

      cd gim_arm_teleop
      python scripts/hardware/teleop_gim_arm_hardware.py

   Synchronously log VR control signals and robot arm state into your dataset.

Configuration File
~~~~~~~~~~~~~~~~~~~~~~

Before starting the experiment, you need to create or modify a configuration YAML file.
The key section is the cluster hardware configuration, which specifies the GimArm robot:

.. code-block:: yaml

  cluster:
    num_nodes: 2
    component_placement:
      actor:
        node_group: "4090"
        placement: 0
      env:
        node_group: gim_arm
        placement: 0
      rollout:
        node_group: "4090"
        placement: 0
    node_groups:
      - label: "4090"
        node_ranks: 0
      - label: gim_arm
        node_ranks: 1
        hardware:
          type: GimArm
          configs:
            - can_interface: can0
              arm_variant: gim_arm_xl
              camera_serials: ["YOUR_CAMERA_SERIAL"]  # Use [] if no cameras are available
              camera_type: realsense
              enable_gripper: true
              gripper_type: parallel
              node_rank: 1

Set the ``target_ee_pose`` in the environment override configuration:

.. code-block:: yaml

  env:
    train:
      override_cfg:
        target_ee_pose: [0.5, 0.0, 0.1, -3.14, 0.0, 0.0]
        reset_joint_qpos: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
        safe_retract_qpos: [0.0, -1.5, 1.5, 0.0, 0.0, 0.0]
        is_dummy: false

    eval:
      override_cfg:
        target_ee_pose: [0.5, 0.0, 0.1, -3.14, 0.0, 0.0]

Key configuration fields:

- ``target_ee_pose``: Target end-effector pose ``[x, y, z, rx, ry, rz]`` (meters / Euler XYZ radians).
- ``reset_joint_qpos``: Joint configuration for the start of each episode.
- ``safe_retract_qpos``: Joint configuration for safe retraction during peg-insertion reset.
- ``is_dummy``: Set to ``true`` for testing without hardware.

.. note::

   Camera support is optional. If ``camera_serials`` is set to an empty list
   ``[]`` or omitted, the environment will run without camera observations.
   The ``frames`` key in the observation space will be an empty dictionary.
   This is useful for state-only policies or when cameras are not yet set up.

Testing the Setup (Optional)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We provide several test scripts to verify that the setup is correct before starting the experiment. This step is optional but recommended.

1. Verify the CAN interface is up:

.. code-block:: bash

   ip link show can0

2. Test the robot controller:

.. code-block:: bash

   python toolkits/realworld_check/test_gim_arm_env.py --can can0 --variant gim_arm_xl

This script tests: controller launch, ``is_robot_up()``, ``get_state()`` output shapes, ``move_joints()``, ``reset_joint()``, and gripper open/close.

.. note::

   Camera setup has not been fully tested yet. To run the peg-insertion
   experiment, cameras should be available and configured via
   ``camera_serials`` in the hardware config.
