Gemini Live’s camera and screen sharing technology is called Project Estra. It is available as an Android app for reliable examiners, and Google today unveiled agental capabilities for Project Estra, including how it can control your Android phone.
Google removed the new project estra capabilities on a Pixel 9 Pro. In this example, a person is having a live interaction while repairing his bicycle. It begins with the user’s request and finding a PDF manual online to Estra. When opening, Estra is present as a floating bubble, when you see its touch input and a screen glow as it scrolls the file on the desired page.
Next time we see it opening the YouTube app to find the search results, scroll for videos and then play. The app then discovers the user’s Gmail for information from the conversation. A view is shown with several buckets filled with parts, highlighting the right one on the Estra screen (which was teased on I/O years ago for some Google lens.)
Another impressive capacity is being able to call which receives specific information on your behalf. In this case, it is asking for a bike shop asking whether any part is available, including being able to call back to confirm that the cyclist wants it.
We see that Estra is in the middle of a reaction when it is interrupted by another person. Estra automatically prevents and ignores that irrelevant interaction. The user then asks the reaction to resume, with the session continues this entire time. Meanwhile, when they ask for a bicycle basket that can fit a dog, Estra’s memory ability cites the dog’s name.
Google calls this “Action Intelligence”, with Gemini’s advantage content recovering, interface control, agent highlighting, call aid, knowledge grounding, reference-geopolitan dialogue, personal shopping and domestic audio dialogue.
With Project Estra, Google wants to create a “Universal AI agent that may really help in everyday life.” This project is an Estra app how Google tests testing capabilities that will eventually debut in Mithun Live for consumers.
Our final vision is to convert the Mithun app into a universal al -assistant that will demonstrate everyday tasks for us, our worldly administrator, surface will take care of delightful new recommendations, making us more productive and enriched.
Discover more from AM TechHive
Subscribe to get the latest posts sent to your email.
