Nerfstudio: A modular framework for neural radiance field development

M Tancik, E Weber, E Ng, R Li, B Yi, T Wang… - ACM SIGGRAPH 2023 …, 2023 - dl.acm.org
Neural Radiance Fields (NeRF) are a rapidly growing area of research with wide-ranging
applications in computer vision, graphics, robotics, and more. In order to streamline the …

Foundation models in robotics: Applications, challenges, and the future

R Firoozi, J Tucker, S Tian… - … Journal of Robotics …, 2023 - journals.sagepub.com
We survey applications of pretrained foundation models in robotics. Traditional deep
learning models in robotics are trained on small datasets tailored for specific tasks, which …

Perceiver-actor: A multi-task transformer for robotic manipulation

M Shridhar, L Manuelli, D Fox - Conference on Robot …, 2023 - proceedings.mlr.press
Transformers have revolutionized vision and natural language processing with their ability to
scale with large datasets. But in robotic manipulation, data is both limited and expensive …

Decomposing nerf for editing via feature field distillation

S Kobayashi, E Matsumoto… - Advances in Neural …, 2022 - proceedings.neurips.cc
Emerging neural radiance fields (NeRF) are a promising scene representation for computer
graphics, enabling high-quality 3D reconstruction and novel view synthesis from image …

Nifty: Neural object interaction fields for guided human motion synthesis

N Kulkarni, D Rempe, K Genova… - Proceedings of the …, 2024 - openaccess.thecvf.com
We address the problem of generating realistic 3D motions of humans interacting with
objects in a scene. Our key idea is to create a neural interaction field attached to a specific …

3d concept learning and reasoning from multi-view images

Y Hong, C Lin, Y Du, Z Chen… - Proceedings of the …, 2023 - openaccess.thecvf.com
Humans are able to accurately reason in 3D by gathering multi-view observations of the
surrounding world. Inspired by this insight, we introduce a new large-scale benchmark for …

Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation

W Huang, C Wang, Y Li, R Zhang, L Fei-Fei - arxiv preprint arxiv …, 2024 - arxiv.org
Representing robotic manipulation tasks as constraints that associate the robot and the
environment is a promising way to encode desired robot behaviors. However, it remains …

Clip-fields: Weakly supervised semantic fields for robotic memory

NMM Shafiullah, C Paxton, L Pinto, S Chintala… - arxiv preprint arxiv …, 2022 - arxiv.org
We propose CLIP-Fields, an implicit scene model that can be used for a variety of tasks,
such as segmentation, instance identification, semantic search over space, and view …

Se (3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion

J Urain, N Funk, J Peters… - 2023 IEEE International …, 2023 - ieeexplore.ieee.org
Multi-objective optimization problems are ubiquitous in robotics, eg, the optimization of a
robot manipulation task requires a joint consideration of grasp pose configurations …

Distilled feature fields enable few-shot language-guided manipulation

W Shen, G Yang, A Yu, J Wong, LP Kaelbling… - arxiv preprint arxiv …, 2023 - arxiv.org
Self-supervised and language-supervised image models contain rich knowledge of the
world that is important for generalization. Many robotic tasks, however, require a detailed …