Vinay AWASTHI, Managing Director, Greater Asia, HP Dahai LI, CEO, ModelBest ST LIEW, Vice President, Qualcomm Moderator: Jeremy KAHN, FORTUNE
Category
🤖
TechTranscript
00:00I think it's going to be an interesting panel because all three of our panelists are in
00:04some way involved in the attempt to kind of bring AI to the edge as one potential solution
00:12for the hardware dilemma that AI poses.
00:16I'm actually going to start with ST. I'm curious what strategies you guys have in place at
00:20Qualcomm around this and how important it is for you to sort of integrate AI at the
00:25edge into the solutions that Qualcomm is working on.
00:29Well, thank you for the good questions and it's really nice to be here.
00:35We believe that to harness the full potential of AI, and AI has been adopted in all industries
00:43in almost all walks of life, increasingly so, it needs to be intelligence everywhere.
00:51By having intelligence everywhere, we use the terminology of hybrid AI, so you will
00:57share the AI workloads, whether at the edge or at the cloud.
01:02As such, the full potential of AI can be gathered and also there are a lot of good benefits
01:10that come from it because, for instance, AI that's been performed at the edge can ensure
01:15a lot of personal security and confidentiality, not to mention it frees up the traffic going
01:23up the cloud all the time.
01:25On top of that, there's a huge economy to this as well because edge devices are many,
01:31many things, cell phones, smart cars, PCs these days, and thousands and millions of
01:38IoT devices, and it helps to make sure that that distributed way of processing AI will
01:45also tailor for the specific AI where it is needed.
01:50So we are committed to that, and that's why one of the things we have done recently is
01:57we actually launched our Snapdragon X Elite chipset platform for the PC.
02:03We believe 2024 is when the PC industry, the PC, is being reborn, and PC is a pivotal portion
02:11of the entire edge AI, and not to mention, of course, smartphones and XRVI and so on
02:18and so forth.
02:19So we believe this is the right thing and the important things to go, and there is a
02:22big need for a lot of the consumers, enterprises, small, medium-sized companies as well, to
02:30upgrade a lot of the devices to be able to do these AIs at the edge.
02:36So I think business opportunity is there.
02:38It also helps to involve more people into the digitized economy of the future.
02:47So that's what we are well-suited, Qualcomm is well-suited to do that.
02:51Excellent.
02:52Zahi, can you tell us a little bit about Model Best and about Mini CPM, which is this model
02:57you developed, and how this kind of represents a step in the direction of these kinds of
03:01models that can actually run on edge devices?
03:03Okay.
03:04Thanks, Jeremy.
03:05I'm very happy to be here to represent Model Best.
03:10It is a premier AI startup in Beijing.
03:14Model Best is all about creating the top-level big models, and we really value efficiency.
03:24We believe that efficient model means better performance, smaller size, and less cost,
03:34but with the same capabilities.
03:38And even some folks in the media class Chinese mistrust, because the automated pursuit with
03:47high efficiency is the thing we have only in common.
03:54Instead of just comparing the parameters and performance, we focus on the knowledge density
04:02of the big model.
04:05That's the first thing we are very, very focused on.
04:12Okay.
04:14The star product, our star product is Mini CPM.
04:19It is a bunch of super light models that you can run on your phone, but they pack a punch
04:30with bigger ones, such as GPT 3.5, GPT 4V, or even some future, maybe, GPT 4.
04:41These models have been popular all over the world in the past half years.
04:47Yeah.
04:48Yeah.
04:49And you said we were speaking before, and Dahi was saying you've released five different
04:54versions, updated versions, of Mini CPM just in the last six months.
04:58Is that right?
04:59Yes.
05:00We move very fast.
05:02We have released four, five versions of Mini CPM in the past half years, and we will release
05:12two versions additionally in the next month.
05:19Oh, wow.
05:20So this is obviously technology moving very fast, and I think what you can do on device
05:24is rapidly catching up with workloads that before you could only run in a data center.
05:30So, Vinay, I want to turn to you and HP.
05:32You're known for PCs.
05:33You're known for printers.
05:35Where do you see AI kind of fitting in with that kind of hardware?
05:39Well, I think, first and foremost, great to be here, and thank you for the opportunity.
05:44If you look at the AI story so far, I mean, it has mostly been data center and cloud centric,
05:50right?
05:51Now, we do know that while it's a very powerful story, it has limitations in terms of how
05:56much it can reach.
05:58What we see is the proliferation of AI in a massive way over the next three, four years.
06:04Every person in this room and everywhere in the world who is using a personal computer
06:09right now will find that in order to use AI over the next three, four years, as many of
06:15the use cases emerge and perfect themselves, your current PC generation, they will be all
06:19obsolete.
06:20And for us to really take advantage of AI to every individual, every small and medium
06:27business, every large business, governments, this is going to be a massive effort to provide
06:33AI PCs.
06:35Because at the foundation of AI is the need for driving a very order of magnitude higher
06:42computation.
06:43So, you know, today's PCs, there is a term now we call trillions of operations per second,
06:48TOPs, right?
06:49Today's PCs are anywhere from six, eight, ten.
06:52And we are now talking about 40 TOPs, 55 TOPs, even higher than that.
06:57So our drive is to be able to provide a huge portfolio of products, PCs, printers, communication
07:05equipment that will allow people to be able to use AI models at the edge in whatever use
07:11cases they have.
07:12Now, that's interesting.
07:13One of your rivals at Microsoft, they kind of released this early version of an AI PC,
07:20and it had to sort of record everything you did on the PC.
07:24And some people were worried about that, even though they were trying to sell it as more
07:27private than having to upload data to the cloud constantly.
07:31There was still this concern about security.
07:32How are you thinking about this at HP?
07:34Well, I think security is going to be extremely important if you look at it.
07:37I mean, why is security important?
07:39Because as any computational power, any software, it's a tool.
07:44You can cut it both ways, right?
07:46If good guys are not going to use security, and AI and the latest tools for security,
07:51the bad guys are going to do that.
07:53So what we are doing is developing models that actually learn from the various attacks
07:58that continuously happen.
07:59And those models will be AI-trained.
08:01So we have a suite of security products from BIOS all the way up to the OS.
08:06We call it Wolf Security.
08:08And that is really based on AI.
08:10So it's just one example about how we have to be, as responsible technology providers,
08:16provide technology that can keep people safe first, and then allow them to enjoy the benefits
08:21of the new technology that's going to come.
08:23Great.
08:24I'm going to ask for questions from the room in a minute, so please think of your questions
08:27and raise your hand, and I'll get a mic to you.
08:30But first, I want to ask Dahi, you talked about how popular MiniCPM has been.
08:34You also have this interesting case where some researchers at Stanford plagiarized the
08:40model or sort of built something that seemed very based on MiniCPM.
08:44Can you talk a little bit about what happened in that case?
08:46And I'm curious whether this is sort of a risk in general as we move to smaller open-source
08:51models that there will be this proliferation, it will be very hard to control what people
08:55do with them.
08:56Okay.
08:57It's really interesting.
09:00We released a new multimodal version of MiniCPM that's comparable to GPT-4V on the multimodal
09:09side on May 20.
09:14It's very cool, and the feedback from the open-source community has been awesome.
09:20But then on May 29, some people on the GitHub homepage pointed out to us that there was
09:30a Stanford project that has copied us, and the team claimed to have built a better model
09:39with fewer parameters, which was 1% of GPT-4V, which is true, and the little training costs
09:50only $500, which is not true, and as smart as the GPT-4V, which is true.
10:02After comparing, we found these models are almost the same.
10:07The American one could even understand the Chinese ancient text, which was merely found
10:15in the public training data.
10:17So as you know, what happened next, the Stanford team apologized publicly after getting caught
10:27out for the plagiarism, and they deleted the program, the project on the GitHub.
10:35This incident, we think it's not, does not represent Stanford officially, because it's
10:43just initiated by a few undergraduate students, and we forgive them and remind everyone to
10:54give credit to where it's due in the open-source community, and the incidents just make us
11:03appreciate the open-source community even more.
11:07We are thankful for the support to resolve the issue so quickly and fairly, and ModelBest
11:16has always been active in the global open-source community.
11:21We have created the OpenBMB community, which has been the biggest open-source community
11:30in China, and we continue to work on projects, open-source projects, and cooperate with other
11:39communities to advance model technologies.
11:44Great.
11:45I want to get a question from the audience in a minute, but I'm going to ask ST a slightly
11:49awkward question, which is, right now, there's a lot of concern about running these large
11:56AI workloads in data centers on the energy consumption, and a number of companies have,
12:01Microsoft and Google in particular, have said they've been thrown off their track to net
12:05zero because of the data center growth that they've had.
12:10Some people are looking at on-devices, oh, we can avoid all that, but then if you look
12:14carefully at where some of that additional carbon footprint has come from, it's not actually
12:19the energy used to run computations in the data center.
12:22It's all the energy that went into the production of the chips that are in the data center,
12:28and then there's also this concern about, well, where are you running these models?
12:32If we're pushing everything onto device, my question for Qualcomm is, what's happening
12:35in terms of the energy intensity and carbon intensity of the manufacturer of Qualcomm
12:39chips?
12:40Then, also, if we're pushing everything out to consumers, and the energy is being used
12:45when they charge their PC or their phone, isn't there an issue there because a lot of
12:50those consumers might not have access to renewable energy as their source of home power
12:54or office power?
12:55Well, I think, first of all, I think that we shouldn't say that everything is pushed
13:01to the edge.
13:03There is a very close collaboration between the cloud and the edge.
13:07In fact, if you look at the entire ecosystem, I would say that Qualcomm has always embraced
13:13new challenges and new technologies from 3G, 4G, 5Gs, and because of the connectivity
13:19foundation that 5G has brought, it has enabled us to provide the options of processing some
13:28of this AI at the edge, but you still need the cloud to do certain, you know, it's very
13:33efficient to do, like, massive learning, development stuff on the clouds, but then there are minute
13:39little things that make sense to do it at the edge.
13:43I think the tricks and the intelligence and the innovations is what to do where.
13:50So you have things that will happen on the edge, on my little cell phone, my PCs in my
13:56car that are not even an option not to do it there because you need that latency.
14:03You need that short latency like on smart cars.
14:05You need to have that response immediately.
14:08So there are these things that are just absolutely make sense to do it there, and then as you
14:13grow it bigger, the cell get bigger and bigger, you go do the intelligence at the edge, and
14:18then more edge, and then do the clouds.
14:21So I think there is that intelligence that with the liberalization of the technology
14:26and know-how and empowering so many smart people to do the right thing, there will be
14:31a good balancing point where I think the industry and the developers and companies
14:39will find the right balance to where it is.
14:43That's why I think that the model that will really reap the benefits of AI is something
14:49called the hybrid AI, which is going to be cloud-edged devices and so on and so forth.
14:55Questions from the audience?
14:56Does anyone have a question?
14:57If not, I've got more, so don't worry.
14:59I'm going to ask, I'm actually going to turn to Vinay also and ask on the sustainability
15:03question.
15:04I mean, you just said everybody's going to need a new PC in the next few years, and maybe
15:09we're all going to need new phones, too, because we're going to need that latest Elite X chip
15:13from Qualcomm.
15:14Doesn't that create also a sustainability issue?
15:16You're going to have all these people discarding these old devices.
15:18You're going to have an e-waste issue.
15:20How is HP kind of looking at that?
15:21Because I know you have a firm commitment to sustainability, but does this throw you
15:24off of that?
15:25It does not.
15:27If anything, we are accelerating our commitment to our sustainability goals.
15:30Because our sustainability framework is very broad.
15:37We look at how to make devices more efficient, which means today we are working on devices
15:42that can actually have truly 24 hours, 48-hour battery life.
15:47You make them more efficient because there is less loss of power when you are using those
15:51devices.
15:52That's one part of the framework.
15:54The second part of the framework is using sustainable materials as we build our devices.
16:00Every PC, every printer today now has recycled material, not just plastic, but recycled material
16:10that goes into the chips, recycled material that goes into the casing.
16:14We are also making our factories more efficient, so zero-waste factories is our goal.
16:19Last but not the least, it's a very important part for us to also work with what we call
16:23renewed supply chain, which means we can take devices back, renew them, and put them
16:27back into the market.
16:30Because there are a lot more people in this world today who need computational devices
16:35than those who actually have computational devices.
16:37For us to work on making our products more sustainable and also our supply chains more
16:41sustainable, making renewed products back into the market, I think that's our framework
16:46and we are very committed to that.
16:48Does AI have a role to play in doing that?
16:50It does.
16:51Is it making your operations more efficient or trying to find energy savings throughout
16:54your operations?
16:55Absolutely.
16:56If you look at today, we are talking about lights-out factories.
16:59Those factories will run because they are enabled through AI.
17:03We are talking about devices that will be optimized, whether it is eliminating the processes
17:08that are unnecessary and thereby increasing the battery life.
17:11It's all being driven by AI.
17:13It has a lot of use now.
17:15I see ST nodding.
17:16Is it the same for Qualcomm?
17:17I'm nodding, yes.
17:18You're saying the same things?
17:20The chipset we just introduced, because of our heritage in very efficient computing and
17:28very, very efficient power consumption, you are going to be able to do more things in
17:33a shorter period of time.
17:35Imagine, you can use these AI models after you really allow the thousands and thousands
17:42of developers to think about how to solve some of the problems you just posed.
17:46I think AI with the right tools, with the right liberalization and open up of the tools
17:54will enable a lot of solutions to solve the problems, some of them that you just mentioned.
18:00That's fascinating.
18:01While we have Dahi here, I want to ask, it's slightly off-topic, but one of the amazing
18:05things about Mini-CPM and some of the models you've worked on is their ability to, I guess,
18:11translate these ancient Chinese calligraphy, these Tsinghua bamboo strips.
18:14I don't know if you can talk a little bit about that, because that was also one of these
18:17tests for the model.
18:19I think it's also one of the ways they caught that, as you said, the Stanford plagiarism
18:22cases.
18:23Their model turned out could do this too, which was very strange if it hadn't been using
18:26your training.
18:27Yeah, the Tsinghua bamboo split is just a Chinese ancient replica that a Tsinghua friend
18:44just sent to Tsinghua.
18:50There is no digital information on the internet.
18:54We just take a photograph and use it as a fine-tuned picture to build it into our model.
19:06We make sure that any other model should have not the ability to identify the text character
19:18on the Tsinghua bamboo split.
19:20That's the reason we quickly identify the model as a theme.
19:26Right.
19:27Excellent.
19:28Well, we're out of time, but I want to thank ST and Dahi and Vinay for being here.
19:31It's great to have you.
19:32Thank you so much.