Intuitive Control of a Social Robot Using Natural Language with a Large Language Model and Error Correction Capabilities

In this paper, a novel framework for interacting with the social robot NAO using natural language commands is proposed. The motivation for this work arises from the needs of operators at the PASCIA Center (Heart Failure Care Program, Childhood Heart Diseases, and Those at Risk), a specialized department at the Polyclinic of Modena focused on childhood heart diseases. They aim to use the NAO robot during cardiology visits with young patients affected by autism spectrum disorders. To overcome the barrier that prevents unskilled operators from independently controlling the robot, a new interface has been developed. By leveraging the capabilities of Large Language Models, particularly ChatGPT, operators’ requests expressed in natural language can be transformed into concrete actions executed by the robot. Additionally, a mechanism has been implemented to assist doctors in adjusting and correcting ChatGPT’s responses based on a feedback loop. The results have been evaluated using accuracy metrics to compare successful and failed tasks performed by the robot. The findings demonstrate the framework’s efficiency and usability.