Boston Dynamics’ Spot robot dog can now talk and act as a tour guide, thanks to ChatGPT. The engineering team was inspired by the ability of Large Language Models (LLMs) to role play, replicate culture and nuance, form plans, maintain coherence over time, and their Visual Question Answering (VQA) ability.
Put simply, they turned Spot into a robot tour guide capable of walking around, looking at objects in the environment, use a VQA or captioning model to describe them, and then go into detail using an LLM. That’s not all, it can also answer questions from the tour audience, and plan what actions it should take next.
- 🎁 Ideal Gift for Kids & Teens: This STEM solar robot kit celebrates child’s growing skills and important milestones. Whether for birthdays,...
- 📚 STEM Educational Toy: This solar educational toy brings science to life! The fun DIY building experience sparks children's curiosity in...
- ☀️ Powered by the Sun: Enjoy outdoor play with solar power or switch to a strong artificial light source indoors, such as a flashlight, ensuring...
This sort of demo plays to the strengths of the LLM—infamously, LLMs hallucinate and add plausible-sounding details without fact checking; but in this case, we didn’t need the tour to be factually accurate, just entertaining, interactive, and nuanced. The bar for success is also quite low—the robot only needs to walk around and talk about things it sees. And since Spot already has a robust autonomy SDK, we have the ‘walk around’ part pretty much covered already,” said Matt Klingensmith, Principal Software Engineer.
