Reconciling Reality Through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation

Marcel Torne1, Anthony Simeonov1, Zechu Li1,3, April Chan1, Tao Chen1, Abhishek Gupta2*, Pulkit Agrawal1*
1 Massachusetts Institute of Technology 2 University of Washington 3 TU Darmstadt
* Equal advising
Robotics: Science and Systems 2024
description Paper code Code link GUI link Assets

RialTo robustifies policies originally trained from real-world data in the simulation leveraging 3D reconstruction techniques to transfer the real-world scene to the simulator.

Abstract

Imitation learning methods need significant human supervision to learn policies robust to changes in object poses, physical disturbances, and visual distractors. Reinforcement learning, on the other hand, can explore the environment autonomously to learn robust behaviors but may require impractical amounts of unsafe real-world data collection. To learn performant, robust policies without the burden of unsafe real-world data collection or extensive human supervision, we propose RialTo, a new system for robustifying real-world imitation learning policies via reinforcement learning in "digital twin" simulation environments constructed on the fly from small amounts of real-world data. To enable this real-to-sim-to-real pipeline, RialTo proposes an easy-to-use interface for quickly scanning and constructing digital twins of real-world environments. We also introduce a novel "inverse distillation" procedure for bringing real-world demonstrations into simulated environments for efficient fine-tuning, with minimal human intervention and engineering required. We evaluate RialTo across a variety of robotic manipulation problems in the real world, such as robustly stacking dishes on a rack, placing books on a shelf and four other tasks. RialTo increases (over 67%) in policy robustness without requiring extensive human data collection.

RialTo learns policies robust policies to object poses, visual distractors and physical disturbances

RialTo scales to in-the-wild environments

RialTo leverages digital twins of real-world scenes to fine-tune the policies

Robustness Results

Task
robustness to

RialTo

Imitation Learning

RialTo Overview

1. Scan your target environment

2. Use RialTo's GUI to construct your environment

3. Collect demonstrations in the real world and transfer them to the simulation

4. RL fine-tuning in simulation

5. Teacher-student distillation with real-world demos co-training

BibTeX

@article{torne2024rialto,
        author    = {Torne, Marcel 
                    and Simeonov, Anthony 
                    and Li, Zechu 
                    and Chan, April 
                    and Chen, Tao 
                    and Gupta, Abhishek 
                    and Agrawal, Pulkit},
        title     = {Reconciling Reality Through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation},
        journal   = {Arxiv},
        year      = {2024},
      }