Tech Desk 
Oct 04, 2024
Google is set to update its visual search app, Lens, with the capability to answer real-time questions about the surroundings.
English-speaking Android and iOS users can now start capturing a video through Lens and ask questions about objects of interest in the video.
Google Lens Director of Product Management Lou Wang claimed the feature utilising a “customised” Gemini model to make sense of the video and pertinent questions. Gemini is Google’s family of artificial intelligence (AI) models and powers several products across the company’s portfolio.
“Let’s say you want to learn more about some interesting fish,” Wang stated. “[Lens will] produce an overview that explains why they’re swimming in a circle, along with more resources and helpful information.”
How to use Google Lens new video analysis feature
If you want to access Google Lens's newly updated video analysis features, then you have to be signed up for Google’s Search Labs program, along with adapting to the “AI Overviews and more” experimental features in Labs. In the Google app, holding your smartphone’s shutter button enables Lens’ video-capturing mode.
Read more: Google Meet introduces new ‘Start a call’ feature for Android
The feature in Google Search utilises AI to summarise details from around the web. Wang said that Google Lens uses AI to identify which frames in a video are most “interacting” and salient, — and above all, relevant to the question being asked — and uses these to “ground” the answer from AI Overviews.
Wang said: “All this comes from an observation of how people are trying to use things like Lens right now.”
“If you lower the barrier of asking these questions and helping people satisfy their curiosity, people are going to pick this up pretty naturally,” Wang added.
Google Lens on Android or iOS perceives a product and showcases information about it, such as the price and deals, brand, reviews, and stock.
