Zhi Dongxi reported on August 9th that Google announced on August 6th the launch of Google Home features based on Gemini AI. With Google Home gaining powerful AI support, Google Assistant on Nest smart speakers and displays will receive a significant upgrade in late 2024, and Nest cameras will soon have recognition and display capabilities.

Although chatbots tend to attract more attention, generative AI does indeed have the potential to make smart homes more intuitive and user-friendly. Amazon announced plans for an enhanced Alexa in June this year, and Google is also stepping up its pace, promising to launch a smarter and more powerful Google Assistant.

Advertisement

1. Google Home introduces Gemini, and new features such as smart cameras are about to go online.

Before the fall hardware product launch event next week (August 13th), Google announced three new Gemini smart experiences, which will be available on the Google Home platform later this year. These include:

A new camera intelligence feature that can automatically generate descriptive captions for video clips from Nest cameras;

A natural language input feature for creating programs in Google Home;

A smarter Google Assistant that will be suitable for Nest smart speakers and displays, and equipped with new voice options.

In addition to the new voice options, most of these features will be provided through Google's Nest Aware subscription fee - the video recording subscription for Nest cameras starts at $8 per month ($80 per year). These features will first be limited to Nest Aware subscribers in Google's public preview test program and will be expanded to more users next year.Before the launch event, Anish Kattukaran, the head of Google Home products, mentioned in an interview with The Verge that this is just the beginning of Google's smart home platform stepping into a new era of intelligence, "This paves the way for the next era of Google Home."

II. Gemini Enhances User Experience, But Does Not Replace Google Assistant

This update is good news for Google Home users who have been frustrated for a long time. Many users are already tired of the underperforming, aging smart displays and the trouble of common functions being canceled. In addition, they have also experienced a difficult transition from the Nest app to the Google Home app. The launch of the Google TV Streamer 4K (Google Home Hub) and the new Nest Learning Thermostat this week, along with a smarter Google Assistant, heralds that Google's smart home ecosystem is improving.

Google Assistant will obviously continue to exist. Google did not directly transplant Gemini to Nest smart speakers and displays to control smart homes, but deployed Gemini intelligence behind the scenes. Kattukaran explained: "Gemini is a series of models, and we are optimizing it for various functions of Google Home."

III. Three Major New Features Demonstrated in Practice, Smart Home Development in Multiple Aspects

The development of smart homes is not only about the improvement of technical functions, but also about paying attention to enhancing functions while protecting user privacy and security. The following will demonstrate the new features of smart cameras, simplified operations of home automation, and the upgrade and new voice technology of Google Assistant.

1. Camera Security Alerts with Automatic Subtitles Generation

Google is introducing Gemini intelligence to Nest cameras, enabling them to understand the images and sounds captured by the cameras and convey key information to users. This means that Google Home will not only send alerts about a person or package in the future, and users must watch the video to understand what happened, but also directly add a detailed description of what the camera sees. These models will learn and train in the cloud specifically for the user's home data, and over time, they will become more and more intelligent, more accurately understanding the dynamics around the user's home.Kattukaran shared an example of a video showing a person unloading groceries from a vehicle, with the title:

A young person dressed in casual attire stands next to a black SUV parked in a garage, carrying shopping bags, with the surrounding environment appearing very calm.

In addition to providing explanatory details, the video title also contains rich background information. This not only helps users find the footage but can also assist the system in linking to other automated functions. For example, if the camera recognizes an animal and determines it to be "a dog digging a hole in the garden," the system might automatically perform the "turn on the sprinkler" action.

Google Home's activity tab will add the ability to search for videos with text. For instance, this feature will be very convenient when a user's cat sneaks out of the house after dark. Users can directly search for the last time the cat was spotted, without having to browse through all videos labeled with animals to find it.

2. Natural Language Parsing Automation Program

The new "Help me create" feature in the Google Home app allows users to describe what they want to happen, such as "locking the door and turning off the lights when going to sleep," and the system will automatically create a corresponding execution program.

Users need to input text or voice through the Home app on their mobile phone (this feature is currently not supported through Nest smart speakers), but Kattukaran points out that it will have all the existing features of the Google Home app. This includes all current triggers, conditions, and actions, as well as access to all devices connected to Google Home, including devices that support the Matter standard. Although this feature is not as precise as Google's script editor, it is simple enough for any user to easily create automation features.

3. Google Assistant with New Voice Lines and Natural InteractionGoogle is launching new voice options for its Google Assistant. In addition to camera intelligence and easier natural language input, Google also mentioned improvements to the "core experience" of its Google Assistant—such as playing music and setting timers on all current Nest smart speakers and displays.

Furthermore, Google Assistant will add new voice options with different styles, tones, and accents. The company released a demo video of the first new voice option, which retains a female tone but sounds lighter and more natural.

Google Assistant not only sounds more natural but also interacts more naturally. Kattukaran said that it does not require specific command patterns to fulfill user needs and can handle pauses, "um" and "uh," and answer follow-up questions. This is similar to the Alexa feature announced by Amazon last fall (which has not yet been launched).

Kattukaran mentioned that the new Google Assistant will be able to retain the context of user conversations and gradually learn and understand the user's home situation. He revealed that the Gemini-powered feature will provide services for users' homes in the cloud based on Google's privacy principles.

"This feature is tailored to the user's home and data model. We deliberately slowed down in the home field because the tolerance for error in the smart home field is very small, and we don't want to make mistakes," he said. Google Assistant's goal is to gradually understand the user's home situation, such as the rooms and devices the user has, and become more intelligent based on this.

Conclusion: Gemini helps Google Home upgrade, Google is optimistic about the future of smart assistants.

These changes aim to push digital voice assistants closer to the vision that Google and its competitors have been striving for for years: a truly useful digital assistant.

"When we started launching the first generation of assistants, we promised it would be a furniture intelligent assistant like in 'The Jetsons,' and our vision is to create a super useful assistant that can proactively help you solve problems," Kattukaran said, "We have made a lot of progress, but then we stagnated—not just us, but all 'assistants.' We encountered a technical bottleneck, but with the emergence of large language models and multimodal language models, this bottleneck has been resolved."

As Kattukaran pointed out: "The home is a behemoth," it is complex and chaotic, with multiple roles and scenarios. Managing it is not easy for humans, and it is an even greater challenge for computers. But Amazon, Google, and Apple all seem to be competing to create a future where our homes have an intelligent, emotionally perceptive assistant to respond to our needs.

The future progress is worth looking forward to.