基于场景语义感知与大语言模型推理的行为树生成
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

基金项目:

国家重点研发计划 (2022YFB4700400)


Behavior Tree Generation Based on Scene Semantic Perception and Reasoning with Large Language Models
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    具身智能(embodied AI)需要能够与环境进行互动和感知, 并具备自主规划、决策和行动等能力. 行为树(BT)由于其模块化和高效控制的特性, 已经成为机器人技术中广泛使用的方法. 然而, 现有的行为树生成技术在处理复杂任务时仍面临一定的挑战. 这些方法通常依赖于领域专业知识, 生成行为树的能力有限. 此外, 许多现有方法在语言理解方面存在不足, 或者在理论上无法保证行为树的成功, 从而导致在机器人上的实际部署难度较大. 本研究提出一种新的行为树自动生成方法, 该方法基于大语言模型(LLM)和场景语义感知, 生成包含任务目标的初始行为树. 本文的方法根据机器人的能力设计机器人动作原语和相关条件节点, 并以此设计提示(prompt)使LLM输出行为规划(generated plan), 然后将行为规划转化为初始行为树. 虽然本文以此为示例, 但该方法具有广泛的适用性, 可以根据不同需求应用于其他类型的机器人任务. 同时, 本文将这种方法应用于机器人任务中, 并给出具体实现方法和示例. 在机器人执行任务过程中, 行为树可以根据机器人操作失误和环境变化动态更新, 对外部环境变化具有一定的鲁棒性. 本文进行了初始行为树生成验证实验, 并在仿真机器人环境中进行了验证, 展示了本文方法的有效性.

    Abstract:

    Embodied AI requires the ability to interact with and perceive the environment, and capabilities such as autonomous planning, decision making, and action taking. Behavior trees (BTs) become a widely used approach in robotics due to their modularity and efficient control. However, existing behavior tree generation techniques still face certain challenges when dealing with complex tasks. These methods typically rely on domain expertise and have a limited capacity to generate behavior trees. In addition, many existing methods have language comprehension deficiencies or are theoretically unable to guarantee the success of the behavior tree, leading to difficulties in practical robotic applications. In this study, a new method for automatic behavior tree generation is proposed, which generates an initial behavior tree with task goals based on large language models (LLMs) and scene semantic perception. The method in this study designs robot action primitives and related condition nodes based on the robot’s capabilities. It then uses these to design prompts to make the LLMs output a behavior plan (generated plan), which is then transformed into an initial behavior tree. Although this paper takes this as an example, the method has wide applicability and can be applied to other types of robotic tasks according to different needs. Meanwhile, this study applies this method to robot tasks and gives specific implementation methods and examples. During the process of the robot performing a task, the behavior tree can be dynamically updated in response to the robot’s operation errors and environmental changes and has a certain degree of robustness to changes in the external environment. In this study, the first validation experiments on behavior tree generation are carried out and verified in the simulated robot environment, which demonstrates the effectiveness of the proposed method.

    参考文献
    相似文献
    引证文献
引用本文

鄢龙武,郑王里,林云汉.基于场景语义感知与大语言模型推理的行为树生成.计算机系统应用,,():1-10

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-06-14
  • 最后修改日期:2024-07-18
  • 录用日期:
  • 在线发布日期: 2024-11-15
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号