Robot Kinematics | RL Environment
An article by Scienceinsight
Article AI157
Target audiences: Robotics, Research, Aerospace, Gaming
FetchReach-v4 by Farama Foundation
Installation notes
Source: Farama Foundation
Framework for pre-built training environments: Farama Gymnasium (former OpenAI Gym library). Gymnasium is a maintained fork of OpenAI’s Gym library.
Version: FetchReach-v4. Our fetch-reach environment can run without the Farama's gymnasium-robotics package because it is being loaded via the Farama Gymnasium (import gymnasium as gym) library.
Alternative Packages: Other libraries such as stable-baselines3 might install necessary Mujoco dependencies and define the environment even if gymnasium-robotics is not explicitly installed.
Additional notes
The Farama Foundation has taken over maintenance of OpenAI Gym, releasing gymnasium-robotics. The official OpenAI FetchReach environment is now designed specifically to run on top of the MuJoCo physics engine. We will discuss using the MuJoCo physics engine in a later section.
Backward Compatibility: While gymnasium is the new standard, many older training scripts still run with the original gym library. If you are running FetchReach-v3 or newer, you typically need gymnasium-robotics and gymnasium. If you are running FetchReach-v0, you are using the older legacy OpenAI Gym.
Legacy OpenAI Gym: If you have gym (version < 0.26) installed, it comes with gym.envs.robotics built-in, which contains FetchReach-v0 or v1.
Robotic arm picks up target point (RL)
Continuous control of a robot arm based on existing robot hardware, specifically the grasping or picking up of an object. An RL-agent learns to interact precisely with an object in a simulated environment. Training a Robotic Arm to hit a target point, e.g. a gripper arm that hits or moves a target object.
This task offers sparse binary rewards and follows a multi-goal Reinforcement Learning (RL) framework (Farama Foundation). Farama Foundation uses following ML-Framework: PyTorch (Deep-Learning-Framework).
Configuration
Algorithm: SAC
Wrapper: Monitor
n_eval_episodes = 20 (Indicates how many episodes per evaluation)
eval_freq = 10.000 (Specifies how many steps are evaluated)
Untrained agent
Trained agent
Total timesteps: 500,000
⌀ Reward over 20 episodes = -1.80 ± 0.00
Reward = -1.45 ± 0.67
Transferable to Gazebo
Extensible to a real-world arm
Other possible analyses
Scientific data analysis
AI modeling of experimental data
Visualization and predictions
Reproducibility
Other target audiences: Chemistry, Physics, Pharmaceuticals
Business modules for service providers
Starter prototype: Training an RL agent in a simulation (e.g., Unity, Gazebo, IsaacGym), simple policy (strategy)
Custom behavior: Implementation of specific reward structures, multi-agent systems, optimization
Performance optimization: Improvement of training time, model architecture, hyperparameter tuning
Transfer to real hardware (robotics only): Support for the transition from the simulation agent to the physical platform (from a simulation agent to a physical platform)