Creative robot tool use with large language models

M Xu, P Huang, W Yu, S Liu, X Zhang, Y Niu… - arxiv preprint arxiv …, 2023 - arxiv.org
Tool use is a hallmark of advanced intelligence, exemplified in both animal behavior and
robotic capabilities. This paper investigates the feasibility of imbuing robots with the ability to …

A framework for tool cognition in robots without prior tool learning or observation

KP Tee, S Cheong, J Li, G Ganesh - Nature Machine Intelligence, 2022 - nature.com
Human tool use prowess distinguishes us from other animals. In many scenarios, a human
is able to recognize objects seen for the first time as potential tools for a task and use them …

ToolTango: common sense generalization in predicting sequential tool interactions for robot plan synthesis

S Tuli, R Bansal, R Paul - Journal of Artificial Intelligence Research, 2022 - jair.org
Robots assisting us in environments such as factories or homes must learn to make use of
objects as tools to perform tasks, for instance, using a tray to carry objects. We consider the …

GOALNET: Interleaving Neural Goal Predicate Inference with Classical Planning for Generalization in Robot Instruction Following

J Gupta, S Sharma, S Tuli, R Paul - Proceedings of the AAAI …, 2024 - ojs.aaai.org
Our goal is to enable a robot to learn how to sequence its actions to perform high-level tasks
specified as natural language instructions, given successful demonstrations from a human …

Abstraction in data-sparse task transfer

T Fitzgerald, A Goel, A Thomaz - Artificial Intelligence, 2021 - Elsevier
When a robot adapts a learned task for a novel environment, any changes to objects in the
novel environment have an unknown effect on its task execution. For example, replacing an …

“Do This Instead”—Robots That Adequately Respond to Corrected Instructions

C Thierauf, R Thielstrom, B Oosterveld… - ACM Transactions on …, 2024 - dl.acm.org
Natural language instructions are effective at tasking autonomous robots and for teaching
them new knowledge quickly. Yet, human instructors are not perfect and are likely to make …

Fixing symbolic plans with reinforcement learning in object-based action spaces

C Thierauf, M Scheutz - 2024 IEEE/RSJ International …, 2024 - ieeexplore.ieee.org
Reinforcement learning techniques are widely used when robots have to learn new tasks
but they typically operate on action spaces defined by the joints of the robot. We present a …

Effects of Robot Competency and Motion Legibility on Human Correction Feedback

S Wang, A Wang, S Goncharova, B Scassellati… - arxiv preprint arxiv …, 2025 - arxiv.org
As robot deployments become more commonplace, people are likely to take on the role of
supervising robots (ie, correcting their mistakes) rather than directly teaching them. Prior …

An approach to task representation based on object features and affordances

P Gajewski, B Indurkhya - Sensors, 2022 - mdpi.com
Multi-purpose service robots must execute their tasks reliably in different situations, as well
as learn from humans and explain their plans to them. We address these issues by …

GoalNet: Inferring Conjunctive Goal Predicates from Human Plan Demonstrations for Robot Instruction Following

S Sharma, J Gupta, S Tuli, R Paul - arxiv preprint arxiv:2205.07081, 2022 - arxiv.org
Our goal is to enable a robot to learn how to sequence its actions to perform tasks specified
as natural language instructions, given successful demonstrations from a human partner …