University of Toronto computer scientists have developed Stargazer, an interactive camera robot designed to help you film how-to videos. The system consists of a single camera on a robot arm, along with seven independent motors that can move along with the video subject by autonomously tracking regions of interest.
Its behaviors can be adjusted based on subtle cues from instructors including body movements, gestures and speech that are detected by the sensors. A user’s voice is recorded with a wireless microphone and sent to Microsoft Azure Speech-to-Text. This text, along with a custom prompt, is then sent to GPT-3, which then labels the instructor’s intention for the camera. For example, a user could have Stargazer adjust its view to look at each of the tools they will be using during a tutorial by pointing to each one, prompting the camera to pan around.
- Capture. Charge. Get a Highlight Video Automatically
- Revolutionary New Image Sensor
- Unbelievable Image Quality
Photo credit: Matt Hinse
The robot is there to help humans, but not to replace human. The goal is to have the robot understand in real time what kind of shot the instructor wants. The important part of this goal is that we want these vocabularies to be non-disruptive. It should feel like they fit into the tutorial,” said Jiannan Li, a PhD candidate in U of T’s department of computer science.
