Project Page for the IROS 2022 submission

Marc Toussaint, Jason Harris, Jung-Su Ha, Danny Driess, Wolfgang Hönig

Learning & Intelligent Systems Lab, Science of Intelligence Excellence Cluster, TU Berlin

Paper: http://arxiv.org/abs/2203.05390

Brief: With this research we build on our previous methods for physical reasoning, but bring them towards highly reactive robotic manipulation, robust to an experimenter interfering.

Abstract: Task and Motion Planning has made great progress in solving hard sequential manipulation problems. However, a gap between such planning formulations and control methods for reactive execution remains. In this paper we propose a model predictive control approach dedicated to robustly execute a single sequence of constraints, which corresponds to a discrete decision sequence of a TAMP plan. We decompose the overall control problem into three sub- problems (solving for sequential waypoints, their timing, and a short receding horizon path) that each is a non-linear program solved online in each MPC cycle. The resulting control strategy can account for long-term interdependencies of constraints and reactively plan for a timing-optimal transition through all constraints. We additionally propose phase backtracking when running constraints are missed, leading to a fluent re-initiation behavior that is robust to perturbations and interferences by an experimenter.

Source Code: The code specific to these experiments is available here: https://github.com/MarcToussaint/22-SecMPC. The https://github.com/MarcToussaint/botop repository contains the actual code base, including the general SecMPC implementation (in src/BotOp/SecMPC). Please install botop in a folder next to the 22-SecMPC repository.

Funding: The research has been supported by the German Research Foundation (DFG) under Germany’s Excellence Strategy EXC 2002/1–390523135 “Science of Intelligence”.

Experiments

  • Push-with-Stick with the experimeter interfering: youtube
  • Pick-and-Place with the experimeter interfering: youtube
  • Paper accompanying video (short parts of the above): youtube