Should you can’t seize what you wish to seek for with only a image, Google Lens will now allow you to take a video — and even use your voice to ask about what you’re seeing. The characteristic will floor an AI Overview and search outcomes primarily based on the video’s contents and your query. It’s rolling out in Search Labs on Android and iOS at present.
Google first previewed utilizing video to look at I/O in Could. For instance, Google says somebody curious in regards to the fish they’re seeing at an aquarium can maintain up their telephone to the exhibit, open the Google Lens app, after which maintain down the shutter button. As soon as Lens begins recording, they will say their query: “Why are they swimming collectively?” Google Lens then makes use of the Gemini AI mannequin to supply a response, much like what you see within the GIF beneath.
When talking in regards to the tech behind the characteristic, Rajan Patel, the vice chairman of engineering at Google, advised The Verge that Google is capturing the video “as a collection of picture frames after which making use of the identical pc imaginative and prescient strategies” beforehand utilized in Lens. However Google is taking issues a step additional by passing the knowledge to a “customized” Gemini mannequin developed to “perceive a number of frames in sequence… after which present a response that’s rooted within the net.”
There isn’t help for figuring out the sounds in a video simply but — like if you happen to’re attempting to establish a hen you’re listening to — however Patel says that’s one thing Google has been “experimenting with.”
Google Lens can be updating its picture search characteristic with the power to ask a query utilizing your voice. To strive it, intention your digital camera at your topic, maintain down the shutter button, after which ask your query. Earlier than this variation, you may solely kind your query into Lens after snapping an image. Voice questions are rolling out globally on Android and iOS, but it surely’s solely obtainable in English for now.
