Boston Dynamics’ Spot robot dog can now talk and act as a tour guide, thanks to ChatGPT. The engineering team was inspired by the ability of Large Language Models (LLMs) to role play, replicate culture and nuance, form plans, maintain coherence over time, and their Visual Question Answering (VQA) ability.
Put simply, they turned Spot into a robot tour guide capable of walking around, looking at objects in the environment, use a VQA or captioning model to describe them, and then go into detail using an LLM. That’s not all, it can also answer questions from the tour audience, and plan what actions it should take next.
- ✔✔【12 Different Types of Robots】: 12 different building robots that can move on land or water for your child to build, two levels as to...
- ✔✔【Upgraded Larger Solar Panel Boosts Power】: Powered by the Sun. This robot kit has been upgraded with a large solar panel, which Provides...
- ✔✔【Support STEM Education】: These science toys will teach children how to engineer a robot that uses solar power while helping to develop...
This sort of demo plays to the strengths of the LLM—infamously, LLMs hallucinate and add plausible-sounding details without fact checking; but in this case, we didn’t need the tour to be factually accurate, just entertaining, interactive, and nuanced. The bar for success is also quite low—the robot only needs to walk around and talk about things it sees. And since Spot already has a robust autonomy SDK, we have the ‘walk around’ part pretty much covered already,” said Matt Klingensmith, Principal Software Engineer.
