A survey of optimization-based task and motion planning: From classical to learning approaches

Z Zhao, S Cheng, Y Ding, Z Zhou… - IEEE/ASME …, 2024‏ - ieeexplore.ieee.org
Task and motion planning (TAMP) integrates high-level task planning and low-level motion
planning to equip robots with the autonomy to effectively reason over long-horizon, dynamic …

Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation

W Huang, C Wang, Y Li, R Zhang, L Fei-Fei - arxiv preprint arxiv …, 2024‏ - arxiv.org
Representing robotic manipulation tasks as constraints that associate the robot and the
environment is a promising way to encode desired robot behaviors. However, it remains …

isdf: Real-time neural signed distance fields for robot perception

J Ortiz, A Clegg, J Dong, E Sucar, D Novotny… - arxiv preprint arxiv …, 2022‏ - arxiv.org
We present iSDF, a continual learning system for real-time signed distance field (SDF)
reconstruction. Given a stream of posed depth images from a moving camera, it trains a …

NeuralFeels with neural fields: Visuotactile perception for in-hand manipulation

S Suresh, H Qi, T Wu, T Fan, L Pineda, M Lambeta… - Science Robotics, 2024‏ - science.org
To achieve human-level dexterity, robots must infer spatial awareness from multimodal
sensing to reason over contact interactions. During in-hand manipulation of novel objects …

Learning multi-object dynamics with compositional neural radiance fields

D Driess, Z Huang, Y Li, R Tedrake… - Conference on robot …, 2023‏ - proceedings.mlr.press
We present a method to learn compositional multi-object dynamics models from image
observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and …

Reinforcement learning with neural radiance fields

D Driess, I Schubert, P Florence, Y Li… - Advances in Neural …, 2022‏ - proceedings.neurips.cc
It is a long-standing problem to find effective representations for training reinforcement
learning (RL) agents. This paper demonstrates that learning state representations with …

Imitating task and motion planning with visuomotor transformers

M Dalal, A Mandlekar, C Garrett, A Handa… - arxiv preprint arxiv …, 2023‏ - arxiv.org
Imitation learning is a powerful tool for training robot manipulation policies, allowing them to
learn from expert demonstrations without manual programming or trial-and-error. However …

PRIF: primary ray-based implicit function

BY Feng, Y Zhang, D Tang, R Du… - European Conference on …, 2022‏ - Springer
We introduce a new implicit shape representation called Primary Ray-based Implicit
Function (PRIF). In contrast to most existing approaches based on the signed distance …

Neural grasp distance fields for robot manipulation

T Weng, D Held, F Meier… - 2023 IEEE International …, 2023‏ - ieeexplore.ieee.org
We formulate grasp learning as a neural field and present Neural Grasp Distance Fields
(NGDF). Here, the input is a 6D pose of a robot end effector and output is a distance to a …

DFields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Robotic Manipulation

Y Wang, M Zhang, Z Li… - ICRA 2024 Workshop …, 2023‏ - openreview.net
Scene representation has been a crucial design choice in robotic manipulation systems. An
ideal representation should be 3D, dynamic, and semantic to meet the demands of diverse …