Sim-to-Real Execution with Torobo
Controlled pick-and-place execution using inverse kinematics and ROS 2 integration in NVIDIA Isaac Sim.
Pick-and-place execution using Torobo in a controlled simulation setup.
Motivation
This project focuses on bridging the gap between perception research and actual robotic execution. While previous work explored synthetic data and detection performance, this step concentrates on validating the execution pipeline under controlled conditions.
The goal is to establish a reliable baseline for robotic manipulation before introducing perception-driven variability. By fixing object positions and simplifying the environment, the system behavior can be evaluated independently of perception uncertainty.
Project description
The project demonstrates a pick-and-place pipeline using a Torobo humanoid-like robot (Tokyo Robotics) in NVIDIA Isaac Sim, integrated with ROS 2. Motion generation is handled via inverse kinematics, allowing the robot to move between predefined task-space positions.
The setup is intentionally simplified: object position is known, grasp points are predefined, and the environment is static. This enables focused evaluation of motion execution, trajectory consistency, and system integration without additional sources of uncertainty.
The work builds on previous experiments with synthetic data for fruit detection and serves as a transition toward perception-driven manipulation.
System overview
The system combines simulation, control, and motion planning components into a unified execution pipeline. ROS 2 is used as the communication layer, while NVIDIA Isaac Sim provides the simulation environment and robot model.
- Integration of Torobo robot model in NVIDIA Isaac Sim
- Motion generation using Lula inverse kinematics solver
- ROS 2-based communication between components
- Predefined task-space targets for pick and place
- Deterministic execution in a controlled environment
Key challenges addressed
Even in a controlled setup, achieving reliable robotic execution requires careful handling of kinematics, coordinate transformations, and system integration. The challenge lies in ensuring consistent and repeatable motion while maintaining compatibility between simulation and control layers.
- Stable inverse kinematics solutions for reachable target poses
- Alignment of coordinate frames between simulation and control
- Ensuring repeatable grasp and placement execution
- Integration of ROS 2 with simulation components
Outcome
The project establishes a reliable baseline for robotic manipulation in simulation, demonstrating consistent pick-and-place execution using inverse kinematics. By isolating execution from perception, the system behavior can be evaluated in a controlled and reproducible manner.
This baseline provides a foundation for future work, where perception-driven object detection and dynamic scene understanding will be integrated into the execution pipeline.
What is intentionally not shown
Detailed implementation of motion planning, controller tuning, and system-specific integration logic are intentionally omitted.