From Add Me and Pixel Screenshots to Call Notes, the Google Pixel 9 series has a bunch of AI upgrades, and none of them would be possible without the new Tensor G4. Tom’s Guide sits down with Jesse Seed, group product manager for Google Silicon, and Zach Gleicher, product manager for Google DeepMind, about what the Tensor G4 chip can do and how it stands out from Apple’s A series and Qualcomm’s Snapdragon.
Category
🤖
TechTranscript
00:00The Google Pixel 9 is here, and a lot of people
00:02are going to be talking about new features like the design
00:04as well as the display, battery life.
00:06But what really makes this phone stand out
00:08is the Tensor G4 chip and all the AI experiences
00:11that it enables on this new wave of devices.
00:14And to help us walk us through some of those scenarios
00:16and how it can actually help your life get
00:18a little bit better is Jesse Seed,
00:20who is on the team that is behind the Google Silicon,
00:24as well as Zach Gleicher, who is at DeepMind.
00:26And they have a deep collaboration
00:28with the Tensor G4 team.
00:29So Jesse, what do you think makes the Tensor G4
00:32chip stand out in a sea of smartphones?
00:35I think the biggest innovation that we made this year
00:37was being the first Silicon, the first phone
00:39to run Gemini Nano with multimodality.
00:42And that unlocks some very cool use cases, one of which
00:45is Pixel screenshots.
00:47So that's very handy if you're trying to remember things.
00:52I'm sure you got a chance to play with that.
00:54And another feature, not related to the Gemini Nano model,
00:57but I really love is also the Add Me feature.
00:59And so those of us that are photographers of our family
01:02or our crew definitely appreciate
01:04being able to go back in and dynamically
01:08add the photographer in.
01:09And that's something that we worked a lot
01:12on to tune over 15 different machine learning models
01:15and also using the Google Augmented Reality SDK.
01:19So yeah, I think those are my top two favorite Tensor-enabled
01:23Pixel experiences this year.
01:24So how do you get something like Gemini Nano
01:27to fit on something that's as compact as a phone?
01:30At DeepMind, we collaborate with a whole bunch
01:34of teams across Google.
01:35And we want to make sure that we're
01:37building Gemini models that meet the needs of all Google
01:40products.
01:41So as we were developing Gemini in collaboration
01:46with Android and Pixel, of course,
01:49we realized that there was this need for on-device models.
01:53So we saw this as a challenge because on the server,
02:01everyone was pushing for more capable models that
02:04were potentially bigger.
02:05And we, on the other hand, had all these interesting
02:08constraints that weren't present before on memory constraints,
02:11power consumption constraints.
02:14So in partnership with the Tensor team and Pixel,
02:18we were able to come together and understand
02:20what are the core use cases for these on-device models, what
02:24are the constraints for these on-device models,
02:27and actually co-develop a model together,
02:31which was a really exciting experience,
02:34and made it possible to build something
02:36that was so capable and able to power these use cases.
02:39For someone who hasn't upgraded their phone in,
02:41let's say, like three or four years,
02:42what do you think is going to stand out for them when
02:45it comes to the Tensor G4 chip?
02:46Improving what we call fundamentals,
02:48like power and performance, are very important for us.
02:52So the Tensor G4, which is our fourth generation chip,
02:54is our most efficient and our most performant.
02:58And so we believe that users will
03:00see that in everyday experiences,
03:02like web performance or web browsing,
03:05as well as app launch, and just overall snappiness
03:07of the user interface.
03:09So definitely, I think they'll be able to experience that
03:12in hand.
03:13And what about gaming performance?
03:14Because we know that's really important these days
03:16for people who are buying a new phone.
03:18In our testing, we actually have seen
03:20improved both peak and sustained performance
03:22in gaming and common games that run on the platform.
03:25So yeah.
03:25So I feel like we're almost past the phase where people are
03:28no longer afraid of AI, and they're
03:30more interested in terms of how it's going to help them.
03:32So what are some of the features within Gemini Nano
03:34coming to the phone that you're most excited about?
03:37Some of the main motivations that we
03:39see for the Tensor team and Pixel team
03:42coming to us for on-device use cases is,
03:46one, is better reliability.
03:49So the fact that you don't have to rely
03:51on an internet connection, the experience
03:54can be reliable and work no matter where you are.
03:58Another is, as we think about potentially privacy,
04:01if developers don't want the data to actually leave
04:05the device and be fully processed on-device,
04:08that's possible with having an on-device LLM.
04:11I think that some of the features that I'm excited about
04:15is I think the Pixel screenshots is a really great one.
04:18I think that really showcases how
04:21we are able to get these multimodal features that
04:25are working on the device.
04:27As you can see in the demos, it was really snappy,
04:29low latency, going to be reliable in how it works.
04:33But it's also a super capable model.
04:36And then all this information is data
04:38stored locally on your device, can be processed locally.
04:41So we're really excited that it can
04:43enable experiences like that.
04:45I think we're seeing traction for summarization use cases
04:50and smart reply and some of these common themes
04:52that are happening.
04:53And those are some of the use cases
04:55that we're really trying to make sure
04:57that the model works especially well for.
05:00So I think the models are continuing to get better,
05:02more capable, and we're going to just see
05:04the possibilities of what's possible
05:07on-device continuing to expand.
05:09So now that the G4 chip is in all of these phones,
05:11how do you balance the higher performance versus thermals
05:14and battery life?
05:16So something like thermal performance and indeed
05:18even battery life, they're full system design challenges.
05:21It's not just about any one component, like only the chip
05:24or only something else.
05:26It's about the entire system.
05:27So what we're so lucky to have is control of the full stack,
05:31everything from the silicon all the way up to the higher level
05:33user application and everything in between.
05:36So that means that we can tweak and refine year over year.
05:39And so, yes, as you mentioned, the addition
05:41of the vapor chamber, that's one concrete thing
05:43that we did in the Pro line this year
05:45to really give a little bit of extra headroom
05:47in those high sustained use cases
05:50where you're burning more power.
05:52But yeah, that's the way we think about it.
05:54It's really like the full system design
05:56and how do we improve that year over year.
05:58I think to a certain degree, sometimes users
06:00get intimidated with AI on phones,
06:02especially since we're still at the early stages.
06:04So how do you make sure with the Pixel 9
06:06in particular that people are excited
06:08and that they actually find these features to begin with?
06:11So I'm sure you've used a Pixel phone
06:13through all this process.
06:14There's this very cool thing called Pixel Tips,
06:16which I love to use when I get my new Pixel
06:19and will actually guide you through some of the new
06:23applications or new use cases or new ways
06:25that a particular app will work.
06:27So I think that's one way that we can help communicate
06:29to users what's the new cool stuff to play
06:32with on your new Pixel phone.
06:34I think we saw with Microsoft and Recall, which
06:36they had a Recall themselves, that people
06:39are a little bit nervous about their phones knowing
06:41everything about them.
06:42But I think screenshots is a little bit different
06:44if you guys can go into that.
06:45Because I know it's more manual.
06:47You're deciding what you want your phone to capture.
06:49But at the same time, it can still not know a lot about you.
06:52So how do you make sure that that information stays
06:54private and only on the phone?
06:56So I mean, one of the ways we do it
06:58is indeed by having a capable on-device model, right?
07:02So that means that the analysis that's
07:04being done on that screenshot, none of it
07:06leaves the device.
07:07So that's one way that we're able to address
07:09that privacy concern.
07:10I think the other thing is just empowering users
07:13to decide what they want to do, like how they want to use
07:16something like Gemini, right?
07:18And what use cases they feel comfortable interacting with
07:22and what they don't.
07:23So I think it really comes down to user choice.
07:25But in the case of Pixel screenshots in particular,
07:28that is a fully on-device use case.
07:30So yeah.
07:32So I don't think third-party benchmarks are going away.
07:34Because we're going to use them to test these phones
07:37and the Tensor G4 chip.
07:38But at the same time, I think we have
07:39to start thinking about performance a little bit
07:41differently now that the AI era is here.
07:44So from your perspective, how should
07:46we be thinking about performance now when it comes to this chip?
07:49That's a great question.
07:50I think it really all comes down to real world use cases, right?
07:53Like, how does this thing actually perform in hand,
07:55the way you're actually going to run it?
07:57So I do think that things like how fast the web browsing
08:01response is, how fast apps are launching,
08:04the quickness and the responsiveness of the user
08:06interface, those are all sort of for everyday use cases.
08:09Those are good standard things to look at, right?
08:11And then also things like how fast
08:13can you capture a picture?
08:15These are all reasonable things that people really
08:17do through the course of the day.
08:18And I think those are much more representative.
08:20Just a few to mention.
08:22Those are much more representative
08:23than the sort of oftentimes semi-synthetic benchmarks
08:28that we see out there in the industry.
08:31Yeah, so when it comes to the Gemini Nano model
08:33on Pixel phones, from your perspective,
08:36when does that phone pass the test in terms of performance?
08:39As we think about benchmarks for LLMs and Gemini,
08:44and especially as we think about Gemini Nano,
08:47we've seen in industry a large focus on academic benchmarks.
08:53And academic benchmarks like MMLU
08:56are great as it gives a common metric,
09:00but they could be gamified and people
09:02can optimize for them.
09:04And it might not capture what you really care about.
09:08So for example, MMLU is a popular academic benchmark
09:13that is going to ask some of the very diverse set of questions
09:17that it asks the LLM to answer.
09:20But maybe some of the questions that it asks
09:22is just information about history, history questions.
09:27For an on-device model, we don't really
09:29care that it knows it can answer history questions.
09:32We think that's probably a better use case for a server
09:35side model when we care about use cases like summarization,
09:39where it's not important whether you know when Rome fell
09:44or something like that.
09:45So what we try to do is we work closely
09:49with the partner teams that are building
09:51these on-device experiences.
09:53And we really try to gather benchmarks
09:56that they care about and the use cases that they care about
09:59so that we can evaluate against those.
10:02And that's how we think about quality.
10:03But again, this is what also becomes super important
10:07as we think about Gemini Nano versus our server side flash
10:10models and Pro is we also have to think about constraints
10:14like battery consumption.
10:16And we have to make sure.
10:17So we work that the model performs well
10:20and doesn't consume too much battery
10:22and that also the latency is good.
10:25So we actually partner with the Tensor team
10:28to profile our models as we're co-designing these models
10:32together to make sure that we are getting an architecture
10:35that works well and meets their efficiency, power consumption
10:39constraints.
10:39And then we collect data for the use cases that they care about
10:42and make sure that we can hill climb on those use cases
10:45and make the model as good as possible.
10:47Yes, MMLU and other metrics like that
10:50are great for us just to make sure
10:52that we have automated metrics that we could hill
10:54climb against because creating good evals is often
10:58a very difficult task.
11:00But we do a lot of that co-development together.
11:02That's great.
11:03And I would also just add, I think
11:04it's actually onto something here.
11:05It's also something to be said for it's not just
11:07about traditional maybe metrics of performance,
11:09but also quality.
11:10So if you look at things like the quality of responses
11:12coming out of the model or even things like quality
11:15of the photo, those are more like that's
11:18what real world users in hand are
11:19going to care more about than some number
11:22on the side of a bottle.
11:23It's more subjective.
11:25That's true.
11:26And as we think about quality, sometimes we
11:30have human raters who are evaluating quality.
11:33But what's, I think, really exciting
11:35about the development of Gemini is we could actually
11:39use larger Gemini models, which we call auto-raters,
11:42to evaluate the quality of the model.
11:45AI rating AI.
11:46Yes, self-rating.
11:47That can be a very powerful way for us to iterate more quickly
11:52and make sure that we are getting
11:53the model to perform well.
11:55Of course, these have their mistakes and issues as well.
11:58And that's why doing actual sanity checks with human raters
12:01can be helpful too.
12:02All right, so I just wanted to say thank you to you
12:04both in terms of taking the time out
12:05to talk about this new chip and what's
12:07happening with DeepMind behind the scenes
12:09and how this is all coming to life.
12:11We're going to test out these phones
12:12to see how good they are.
12:13But now we know a little bit more
12:14about how much smarter these things are getting.