Robocook: Long-horizon elasto-plastic object manipulation with diverse tools

H Shi, H Xu, S Clarke, Y Li, J Wu - arxiv preprint arxiv:2306.14447, 2023 - arxiv.org
Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use:
bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded …

DFields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Robotic Manipulation

Y Wang, M Zhang, Z Li… - ICRA 2024 Workshop …, 2023 - openreview.net
Scene representation has been a crucial design choice in robotic manipulation systems. An
ideal representation should be 3D, dynamic, and semantic to meet the demands of diverse …

See, hear, and feel: Smart sensory fusion for robotic manipulation

H Li, Y Zhang, J Zhu, S Wang, MA Lee, H Xu… - arxiv preprint arxiv …, 2022 - arxiv.org
Humans use all of their senses to accomplish different tasks in everyday activities. In
contrast, existing work on robotic manipulation mostly relies on one, or occasionally two …

Model-based control with sparse neural dynamics

Z Liu, G Zhou, J He, T Marcucci… - Advances in Neural …, 2024 - proceedings.neurips.cc
Learning predictive models from observations using deep neural networks (DNNs) is a
promising new approach to many real-world planning and control problems. However …

Planning with spatial-temporal abstraction from point clouds for deformable object manipulation

X Lin, C Qi, Y Zhang, Z Huang, K Fragkiadaki… - arxiv preprint arxiv …, 2022 - arxiv.org
Effective planning of long-horizon deformable object manipulation requires suitable
abstractions at both the spatial and temporal levels. Previous methods typically either focus …

Understanding World or Predicting Future? A Comprehensive Survey of World Models

J Ding, Y Zhang, Y Shang, Y Zhang, Z Zong… - arxiv preprint arxiv …, 2024 - arxiv.org
The concept of world models has garnered significant attention due to advancements in
multimodal large language models such as GPT-4 and video generation models such as …

Doughnet: A visual predictive model for topological manipulation of deformable objects

D Bauer, Z Xu, S Song - European Conference on Computer Vision, 2024 - Springer
Manipulation of elastoplastic objects like dough often involves topological changes such as
splitting and merging. The ability to accurately predict these topological changes that a …

DiffVL: scaling up soft body manipulation using vision-language driven differentiable physics

Z Huang, F Chen, Y Pu, C Lin… - Advances in Neural …, 2023 - proceedings.neurips.cc
Combining gradient-based trajectory optimization with differentiable physics simulation is an
efficient technique for solving soft-body manipulation problems. Using a well-crafted …

DFields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangement

Y Wang, M Zhang, Z Li, T Kelestemur… - … Conference on Robot …, 2024 - openreview.net
Scene representation is a crucial design choice in robotic manipulation systems. An ideal
representation is expected to be 3D, dynamic, and semantic to meet the demands of diverse …

Dynamic-resolution model learning for object pile manipulation

Y Wang, Y Li, K Driggs-Campbell, L Fei-Fei… - arxiv preprint arxiv …, 2023 - arxiv.org
Dynamics models learned from visual observations have shown to be effective in various
robotic manipulation tasks. One of the key questions for learning such dynamics models is …