Zhi Dongxi reported on August 14th, in the early morning, at the ninth Made by Google event, AI became the key word for the debut of Google's hardware suite, mentioned nearly a hundred times, Google has seamlessly integrated Gemini into various hardware such as folding screen phones, smartwatches, TWS earphones, etc.

This time, the global large model and AI giant Google has thrown out its AI hardware suite deeply combined with the large model, ahead of Apple's official release of the iPhone 16 series equipped with Apple's intelligence, making Android the first mobile operating system equipped with large-scale equipment built-in multimodal AI models.

Advertisement

It is worth mentioning that, in terms of terminal-side AI, Google's thinking is basically the same as Apple's and Samsung's, that is, to rebuild the operating system with AI as the core, and redefine the functions of the phone.

The main updates of Google's AI hardware suite are:

1) Gemini Live, which is comparable to GPT-4o's voice function, has been implemented on mobile phones;

2) Gemini can be called directly regardless of applications and services, and multiple applications can be called with one click to meet the needs;

3) Google released four AI phones: Pixel 9, two sizes of Pro models Pixel 9 Pro, Pixel 9 Pro XL, and the folding screen phone Pixel 9 Pro Fold;4) The Pixel 9 camera has been fully upgraded, with AI capabilities providing 8K resolution videos, automatic framing to save failed photos, and it can also instantly become a concert magic tool;

5) The Pixel Watch 3 can use AI to detect the disappearance of pulse and actively issue an alarm;

6) The Pixel Buds Pro 2 is the first to carry the Tensor A1 chip, with noise cancellation effect twice as good as the previous generation.

It can be seen that Google is currently focusing on making users' AI experience smoother, which is also a specific manifestation of its core advantage: from the research and models to achieve new functions and capabilities, to the Android operating system that allows billions of people to use applications and services every day, and then to various high-end hardware made with custom Tensor chips.

As Google enters the market, the smell of gunpowder in the AI mobile phone war is getting stronger and stronger. However, the latest data from the global market research institution IDC shows that the market share of Pixel phones in the United States in 2023 is about 4.6%, and the variable value that AI brings to Google Pixel phones is still uncertain.

I. Deep integration with Android, all Google applications do not need to be called, Gemini can take over with one click

Google's most important release this time is undoubtedly the new capabilities and comprehensive integration of the AI assistant Gemini.

Previously, as the "imperial smart assistant," Google Gemini has already appeared on Samsung phones. Today, its own Pixel not only finally deeply integrated Gemini, but also brought many updates.

Firstly, Gemini Live, which is comparable to the GPT-4o voice function, is now fully open to Gemini Advanced subscribers in English, and will be expanded to iOS and more languages in the next few weeks.

Gemini directly becomes a chat partner, and the range of topics discussed with users can range from what they think in their hearts to solving complex problems and discussing innovative ideas. Users can have a free conversation with Gemini Live, not only can they interrupt at any time during the answer, but they can also pause the conversation and come back later to continue, and they can also use it hands-free or in the background.Google demonstrated a live conversation with Gemini, which was very fast overall, with almost no latency.

To make the conversation more natural, Google has introduced 10 new voices, allowing users to choose the style and tone that best suits them.

Gemini is now deeply integrated with all Google applications, making the experience of calling Gemini on Pixel very smooth.

Users can summon Gemini by long-pressing the power button or saying "Hey Google," making it available for help anytime, anywhere, such as by bringing up Gemini's overlay at the top of any app and asking questions about the content on the screen.

For example, when watching a video on YouTube, you can click "Ask about this screen" and ask questions about the video content, or let Gemini extract information such as addresses from the video into a list and add it to other apps like Google Maps; you can also summon Gemini while writing an email, have it generate an image, and directly drag the generated image into the text box.

Gemini can also be integrated with all Google applications and tools currently used by users, without the need to switch between apps and services.

For example, when a user wants to host a dinner party, they can complete it in one step: let Gemini find the lasagna recipe sent by a friend in Gmail, ask it to add the ingredients to the shopping list, and then have it create a "playlist that reminds us of the late '90s."

In the future, Gemini will also be integrated into applications like Calendar, allowing users to directly take a photo of a concert flyer and confirm their schedule for the day, set a ticket purchase reminder, and chat with Gemini without opening other applications.In terms of security, Gemini can, with the user's permission, link relevant personal data to all the knowledge organized and made public by Google, to provide assistance to users. For example, Gemini can help you create a daily exercise plan based on the emails of your personal trainer, or write a job resume using your resume in Google Drive.

With Gemini Nano, users can summarize audio from calls using call records, or save and organize images using Pixel screenshots.

But no matter whether this data is processed in the cloud or on the device, it only exists in Google's secure end-to-end architecture, ensuring the security and privacy of this information.

To make the user's mobile AI experience more intelligent and efficient, Google has introduced new models with higher response quality and faster speed, such as Gemini 1.5 Flash, and in the next few months, it will continue to launch deeper integration with Google Home, phone, and messages.

II. Google's largest and thinnest folding phone, a big increase in AI image and text capabilities

Starting with mobile phones, let's take a look at how Google has comprehensively upgraded hardware devices around AI today.

Google has released four new mobile phones, namely Pixel 9, two sizes of Pro models Pixel 9 Pro, Pixel 9 Pro XL, and the folding screen phone Pixel 9 Pro Fold, with prices starting at $799, $999, and $1799, respectively.

Pixel 9 Pro Fold has the largest mobile phone display in Google's history and is the thinnest folding phone. It uses a Super Actua Flex display, which is 80% brighter than Pixel Fold.The Pixel 9 smartphone features a brand-new design with the camera positioned at the front and center. It is equipped with a 6.3-inch Actua display, which is 35% brighter than the previous generation Pixel 8.

The Pixel 9 Pro model offers two different sizes for the first time, including the 6.3-inch Pixel 9 Pro and the 6.8-inch Pixel 9 Pro XL. Both phones are equipped with Super Actua displays.

All four phones are powered by Google's latest custom chip, the Tensor G4. Designed by Google DeepMind, the Tensor G4 has been optimized to run the most advanced AI models and is also the first processor to run the multimodal Gemini Nano model, allowing the phone to understand text, images, and audio.

In terms of AI capabilities, in addition to the deep integration of Gemini, the AI image and text capabilities of the Pixel have also been significantly upgraded.

The Pixel introduces its first image generator, Pixel Studio, which is a combination of a device-side diffusion model running on the Tensor G4 and the cloud-based Imagen 3 model, providing fast text-to-image capabilities on the phone.

Many people have a large number of screenshots on their phones, but it can be difficult to find the exact information needed quickly and accurately when required. The Pixel 9 introduces a dedicated app called Pixel Screenshots, which can save, organize, and retrieve important information from screenshots.

For example, if you are preparing a birthday gift for a friend who loves squirrels, you can first collect a large number of potential gifts or related information in the browser and save them as screenshots. Pixel Screenshots will analyze the content of all the images and provide further suggestions.Pixel has also enhanced AI capabilities in applications such as weather and calls. For instance, the Clear Calling feature can further improve audio quality, and the new "Call History" feature sends a private call summary and complete call record immediately after hanging up. To protect privacy, the call history feature operates entirely on the device.

III. Nine Major Camera Capability Upgrades from Hardware, Software to Algorithms

In addition to AI capabilities, the camera is another focus of the Pixel 9 release this time, with nine major areas of capability upgrades, including hardware upgrades such as lenses, HDR+ imaging pipeline, ultra-high-resolution zoom video, real-time augmented reality technology, Reimagine, automatic framing, panoramic night vision, zoom enhancement, and child-assisted shooting features for parents.

Firstly, at the hardware level, the two Pro models are equipped with a professional triple rear camera system, with a new 48MP 5x telephoto lens featuring an upgraded sensor that can improve autofocus, and a 42MP front camera that can capture clearer images even in low light conditions.

The Pixel 9 Pro Fold is equipped with an ultra-thin advanced triple rear camera system, including an ultra-wide camera that supports macro focusing for both photos and videos.

The Pixel 9 has the same main and ultra-wide lenses, which can collect more than 110% of the light, with a focusing distance close to two centimeters.

Secondly, the Pixel 9 series of phones use a new HDR+ imaging pipeline that optimizes exposure, tone mapping, sharpening, contrast, etc., to better capture elements such as textures, shadows, and skin tones in everyday scenes.

When shooting videos with the Pro model, users can use an ultra-high-resolution zoom capability of up to 20 times, which combines the details of the upgraded telephoto camera with machine learning to capture high-resolution videos with rich details regardless of lighting conditions - perhaps becoming another "concert artifact."

In addition, the Video Boost function utilizes AI capabilities to provide 8K resolution videos that can be magnified by 2 times in post-production and retain 4K resolution.Taking group photos and including the photographer has always been a challenge. Now, with real-time augmented reality technology, the "Add Me" feature can guide the photographer on how to compose a photo to match the first one after it has been taken, ensuring that no one is left out.

Google's AI image editing tool, Magic Editor, has added two new features: Reimagine and Auto Framing.

Reimagine can generate and edit images based on text, such as changing the appearance of grass, sky, trees, indoor decorations, and more in a photo.

Auto Framing is suitable for "saving bad photos." It uses AI to generate a better framing box, allowing the photo to be recomposed and generating background fill to highlight the main subject of the photo by filling in the blank areas around it.

The panoramic mode interface of Pixel is more user-friendly, and the shooting quality is higher, capable of capturing excellent low-light panoramas of landscapes and cityscapes.Based on the scaling enhancement feature, users can capture a portion of a photo and enhance the resolution intelligently with AI, thereby obtaining a high-quality zoom effect after shooting.

For children, the Pixel 9 Pro Fold has introduced the "Made You Look" feature, which plays interesting animations on the flipped side to attract children's attention, making it convenient for parents to take photos. To be honest, this function is quite suitable for pet owners.

IV. AI detects pulse disappearance, TWS earphones first equipped with Tensor A1 chip

The Pixel Watch 3 released by Google this time is called the biggest leap in the flagship smartwatch. In terms of size, the Pixel Watch 3 provides users with a new 45mm dial option.

In addition to size, AI is the focus of the update for this smartwatch.

Firstly, users no longer need to manually turn on sleep mode. The Pixel Watch 3 uses machine learning to automatically detect sleep and enable sleep mode, which can save power and turn off notifications and the always-on display. Based on the same technology, it can also automatically detect when the user wakes up and exits sleep mode.

Secondly, like Apple and Samsung, Google's smartwatch focuses specifically on the user's fitness and sports. To help users find a balance between fitness and rest, the Pixel Watch 3 has introduced three new tools: Daily Readiness (analyzes the user's readiness for training each day), Aerobic Load (tracks the heart load during training), and Target Load (plans training based on personalized goals, cardiopulmonary load, and physical condition).In terms of running, the Pixel Watch 3 also specifically offers new features that help users plan their runs, guide them through completing the run, and reflect the progress of the run. In the advanced subscription model of Google Watch, Fitbit Premium, Google AI will also provide automatic daily running suggestions. These suggestions are designed to help users achieve their target load and can be personalized based on previous running conditions.

The third major highlight of the Pixel Watch 3 update is the pioneering pulse loss detection function. This function combines the sensor signals of the Pixel Watch 3, AI, and signal processing algorithms to detect pulse disappearance events.

The so-called pulse disappearance refers to the sudden cessation of the heartbeat, a condition that can affect people of any age and may be caused by various issues inside and outside the heart, such as primary cardiac arrest, respiratory or circulatory failure, drug overdose, or poisoning.

The heart rate sensor of the Pixel Watch 3 will detect the user's pulse. Once it fails to detect a pulse, it will simultaneously activate the motion sensor to search for a signal, and then combine the pulse and motion signals based on AI to confirm the pulse disappearance event.

If it is, a check is triggered to see if the user responds, and measures such as misjudgment or issuing an alarm are taken.

Finally, Google has also moved the AI spam call filtering from Pixel phones to the Pixel Watch 3.

Looking at the TWS earphones, Google calls the Pixel Buds Pro 2 earphones that are specially designed for Gemini.

The most prominent feature is that users can talk to Gemini without a phone while wearing the Pixel Buds Pro 2. In other words, with Gemini, users seem to have a hands-free, eyes-free virtual AI assistant. It will actively ask users if they need directions to the nearest public transportation station? Do they need a reminder for an upcoming appointment? Do they need song recommendations?In addition, with the help of Gemini Live, Pixel Buds Pro 2 can provide a mobile conversation experience, allowing users to have long conversations with Gemini.

Users can even use Gemini Live on the go, simply by unlocking their phone and saying "Hey, Google, let's chat in real-time" while wearing the earphones.

In terms of hardware performance, Pixel Buds Pro 2 is equipped with a new chip called Tensor A1, which is specifically designed for audio processing. This is the first time Google has expanded the Google Tensor chip beyond smartphones and tablets.

Tensor A1 supports Google's next-generation active noise cancellation technology, which can adjust up to 3 million times per second based on the user's environment, thereby eliminating twice as much noise as the previous generation of Pixel Buds Pro.

Conclusion: The AI smartphone battle has begun, and the race for large model deployment is on.

Following Apple and Samsung, Google is also making a comprehensive effort in AI smartphones, undoubtedly bringing new vitality and competition to the tech industry. With Gemini at its core, Google is using AI to fully reconstruct a complete range of hardware from smartphones to smartwatches and earphones.

The battle for AI smartphones has already begun, and Google is leveraging the Android market advantage to expand the deployment range of Gemini, competing with OpenAI's ChatGPT for a foothold.

In addition to deploying on its own Pixel, Google has also cooperated with Chinese smartphone brands such as Xiaomi and OPPO to introduce Gemini into more models. This not only reflects the position of domestic smartphones in the global AI smartphone market but also demonstrates the advantages of Google's AI capabilities.