On this episode of The Six Five On the Road, hosts Daniel Newman and Patrick Moorhead welcome Chris Moore, VP of Marketing for Micron’s Mobile Business Unit for a conversation on Micron’s newest mobile storage solution announced at MWC 2024, the future of on-device mobile AI and why semiconductor innovation is key to enabling AI at the edge.
Their discussion covers:
- Insights into Micron’s new mobile storage product announced at MWC 2024 and what this enables for the mobile ecosystem
- The role memory and storage play in enabling on-device AI for consumers’ smartphones
- The benefits and use cases for computing at the edge for consumers
- Chris’s thoughts on mobile trends, the future of mobile innovation, and challenges for the mobile ecosystem when it comes to AI innovation
Be sure to subscribe to The Six Five Webcast, so you never miss an episode.
Want to learn more about Micron’s announcements at MWC? Check out their announcements here:
Micron Collaborates with Samsung on Galaxy S24 Series to Unlock the Era of Mobile AI Experiences
Watch the video here:
Or Listen to the full audio here:
Disclaimer: The Six Five webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five is on the road here at Mobile World Congress in Barcelona. Dan, it’s been an exciting, exciting trip so far. I mean, the hallways have been bustling with excitement, basically talking about, quite frankly, the benefits that generative AI can bring to the whole value chain here in Telco World. Whether it’s the device at the edge, the edge network, the core network and everything in between. It’s pretty exciting.
Daniel Newman: Yeah, we’re back. I mean, we are back. We’re walking the halls. You’re bumping into people. And again, we kind of went through this period. Everybody stopped traveling and stopped being on the road. And by the way, we are on the road. I mean, getting here, the traffic, by the way, that’s why we stayed close this time, so we could walk over here because it is just so chaotic. But you’re right, generative AI takes the stage. I went to a session this morning in the big stage and watched it. Everything was AI focused. You go down the halls, you look at the booth, AI, AI… But there’s still a lot of network transformation going on. There’s still a lot of mobile next generation devices, Pat. So we’ve got a lot to talk about.
Patrick Moorhead: Yeah, we do. And one of the biggest themes that we’ve had is the importance of semiconductors and how it is literally transforming everything. And I know that just sounds like a throwaway line, but the truth is, it is. I mean, you and I got into semiconductors before it was cool. I spent a few years at a semiconductors company, and it is amazing to see that importance come to the table. And it’s not just about, oh, I can’t get enough. It’s literally the thing that is making this latest AI trend, generative AI, a reality. And I think we’ve also talked about, hey, it’s not just about the compute, it’s not just about the NPU, the GPU. The importance of every system to have a balanced architecture with the right storage and memory that supports the use cases.
Daniel Newman: I mean, is this memory pretty important for doing parallel compute?
Patrick Moorhead: Oh, last time I checked, yes. And as we found on even handsets and PCs, more storage, more memory, faster interaction, and it took that not-so-subtle introduction, we’re sitting here in the Micron Executive suite, great to see you again.
Chris Moore: Nice to see you. How are you doing?
Patrick Moorhead: Yeah, much better in person than over Zoom. It’s great to see you.
Chris Moore: Yeah, much better. I agree with everything you were just saying. Even before asking a question, jumping right into it. I mean, when we were sitting here last year, we were starting to talk about generative AI, but it was because a certain vendor showed Stable Diffusion. And I was getting asked all the time, why do you need AI at the edge?
Patrick Moorhead: Right.
Chris Moore: Right? But it’s clear that you need it at the edge now, right? I mean it’s not even a question anymore. So yes, AI is growing in the cloud, all these LLMs running in the cloud and now you’re seeing companies launching LLMs on phones, right?
Patrick Moorhead: Right.
Chris Moore: And that’s just really exciting. It’s a major inflection point.
Patrick Moorhead: Well, we get less, used to be the big question was, again, why do we need that? And now as the use cases and quite frankly, announcements that we were both at on the West coast, United States, plus some of the announcements here, make it very clear. Now, can it be clear? Absolutely. Is this the beginning? Absolutely. Is it going to get even better and more ‘oh my gosh’ as we get in there? A hundred percent.
Chris Moore: Absolutely.
Daniel Newman: Yeah. It’s been quite a year, hasn’t it, Chris?
Chris Moore: Yeah.
Daniel Newman: I mean, I do remember last year we were at the show and we started to see demonstrations of on-device AI, with Stable Diffusion, I think we actually had a sit down there.
Chris Moore: Right.
Daniel Newman: It’s been great having you on.
Chris Moore: Didn’t you make a stuffed bunny with a rainbow behind it on a beach?
Daniel Newman: Yeah, one of us. It was like a leprechaun in a field full of weeds or something like that.
Patrick Moorhead: I just gave it something that I thought I couldn’t do that would be completely ridiculous. And it actually did it in about 12 seconds.
Daniel Newman: Yeah. We learned a lot from last year. I mean, you see what Soros… I mean, it’s just absolutely crazy. And by the way, this is such a good moment for a company like Micron because with the amount of memory required to do this and with all the energy and interest in the on-device world, so Micron made an exciting product announcement here at Mobile World Congress. Can you tell us a little bit about the announcement and talk a little bit more about the overall announcement, what’s going on, partner, ecosystem, et cetera?
Chris Moore: Thank you. Yes. Today we announced our new Next Generation UFS 4.0 storage device in the world’s smallest 9×13 package. The smallest thing, it’s about the size of my pinky.
Patrick Moorhead: Tiny.
Chris Moore: Yeah. You can store up to a terabyte in this thing. So it comes in 256, 512, and 1TB capacities. It’s got some great new features. What we can talk about in a little bit, one called One Button Refresh, which is a custom thing that we came up with that helps the device feel like it’s new even after you’ve been using it for a while. And we also have a feature called High Performance Mode, which measures the bus and prioritizes the activity that’s going on in the device to make sure that the end user gets the benefit they’re looking for. We actually see applications can load 25% faster based on this feature.
It’s great. We work with all the great OEMs that are out there, including where our product lineup is in the Samsung Galaxy lineup. Every major OEM you can think of as well as the Honor Magic 6, which was just announced. Our storage and memory devices are in there as well. It’s great. So this product is still the world’s fastest, the fastest speeds in the world at the lowest power, as well as these new custom features, and in the world’s smallest phone part. So we’re really excited about it.
Patrick Moorhead: Yeah. So we’ve talked a little bit about the smartphone and what I’m hearing is, there is a scenario where each user will have to download 20 to 30 different models, not only on their smartphone, but also on their PC because of the variation of the benefits you get. So for instance, one model might be best for images, the other model might be best for parsing text and doing summaries, as an example. One might be the best at being context-aware of what’s around you to be able to tee that up.
So I hope you’re making enough of this in the future. I felt pretty confident on the PC side to call an inflection point. I’m still doing my calculations on when smartphones are, but I have to tell you, I’m seeing better things than I had expected early on because it does take some time for the software to catch up with the capability of the hardware.
Chris Moore: Yeah. And we consider it our job – I personally consider it my job – to keep the hardware out of the way of the software. I mean, our job is to make sure we have this platform that the really smart people that are coming up with the next generation software can run on. And up until now, we’ve been able to do that pretty well. As we’ve come out with faster LPDDR4 and LPDDR5, people could go to from a 4K display to an 8K display. You can get a hundred megapixel camera, but kind of incremental, right? It’s just incrementally better. Certainly not enough to go drive my wife to buy a new phone every year. With AI, that’s changing. The hardware is now behind the needs of software.
So we just talked about an 8 billion parameter file. Well let’s just say that’s about an 8GB file, right? Let’s just do some quick math. If you’re running your storage and you’re saving a few of those on your storage at a time, but you can only read at 4GB a second, it’s going to take two seconds just to load up the file, not even execute it. So you have to take it from the storage, load it into the DRAM, then you have to execute it across the NPU and the DRAM. You’re talking seconds before you get things actually running.
Patrick Moorhead: So it’s not just capacity, it’s the performance, increased performances.
Chris Moore: We’re truly seeing a need now for the first time in a long time to go drive new innovation and that’s just really, really exciting. It’s why I became an engineer in the first place. It’s why a lot of us go to work every day. That said, even before going into AI, with today’s phones, you ever use your phone and after six months or so it’s running slow?
Patrick Moorhead: Absolutely.
Chris Moore: So on the device we just launched today, we have a new feature called OBR One Button Refresh. We’re old enough to remember DOS. Remember Defrag Tools and DOS? As a file sits on a device over a long period of time, it gets fragmented and then you’d have to go type it, right? Your Defrag. And there’s some smartphone vendors today that have their own, where you go into the opt and you-
Patrick Moorhead: Clean up.
Chris Moore: Clean it up, exactly. Well, we do that on the fly. So this new One Button Refresh, you don’t actually have to press a button. It actually happens automatically in the background.
Patrick Moorhead: That’s great.
Chris Moore: And it’ll make your phone look like maybe you’ve only had it for six months. So that’s the kind of innovation that we’re driving, those types of features that make it better for the end user. That’s really what we’re focused on at Micron.
Patrick Moorhead: It’s funny, just when you think you know everything as an analyst – and sometimes we think we do – you learn. I actually did not know that capability was placed. And also the performance required to do this all. So I love it.
Daniel Newman: Yeah, when you started talking about semi, just in the beginning of the show, I put my hand on my heart. I know we want to dig into all the applications, but we said it four or five years ago, semiconductors would eat the world. That’s why everybody wanted to say software. It’s like try running all these apps now without it.
Chris Moore: Well think of the software is the car, right? And you opened up with traffic out there for sure. I know I was late to my first meeting today as I’m sure quite a few people were, because you’re used to running at, I don’t know, 140 kilometers an hour on these streets and we’re doing 5 to 10 because of all the traffic.
Patrick Moorhead: Exactly.
Chris Moore: Well, quick little tie-in to another new feature we have called HPM, High Performance Mode. So what we’re actually doing in the storage is monitoring the bus and we can tell whether it’s an operating system, read or write, or whether you’re executing maybe an LLM, or maybe you’re loading up your favorite video game. And so what we’re doing is parsing that data and then prioritizing, again, automatically in the background on the storage device, making sure the reads for what the user wants to see are prioritized and come out.
We’re seeing actually a 25% faster app loading time based on this new technology that we put out there. So this is based upon the applications today. We’re really excited looking at the applications that are coming tomorrow, but you think, oh, you’ve got this bus and the bits are going back and forth and you’re just always going to be operating at 4GB a second. Truth is, you’re not. You’re operating a system, there’s a lot of stuff out there that’s traffic. And so if you can make sure that you’ve got that police officer making sure your car gets through the red lights faster, your end user is going to be happier.
Daniel Newman: It’s a great feature and I’m glad I set you up for that one. That was like a bonus.
Chris Moore: I know, right?
Daniel Newman: Interestingly enough-
Chris Moore: You didn’t even take notes.
Patrick Moorhead: He’s got this, he’s one of these photographic memory guys.
Daniel Newman: Well, so let’s double click a little bit. Because you’ve kind of alluded to this, but I wanted to ask you, so we’re unlocking more compute at the edge and that’s beneficial for consumers, but something when we were talking in the green room, you talked a little bit about refresh cycles and then I heard you say something earlier in this conversation about getting, you mentioned your wife, but I mean everybody out there buying phones more regularly, buying PCs more regularly, and I think even with the AI PC trend and the device trend, there has been a bit of like a, what’s going to be the reason to go buy the next iPhone? What’s the reason to go buy the next Android phone? By the way, you’ve announced great new partnerships here with a couple of vendors, right? Honor and-
Chris Moore: Big one is Samsung.
Daniel Newman: Samsung. But talk a little bit about what are the benefits of the consumer? What do you see as the drivers to get the consumers to go to the store and upgrade, take that cycle from four years to… What would the difference be? If it went to three, would be massive incremental growth.
Chris Moore: Yeah. And again, the users are why we come to work every day. And so what I see, I see a couple of things. One is, we’re going to start with basic applications as you’re saying, right? You’re going to download 20, 30 different apps of a couple billion parameter files to do your favorite memes, Stable Diffusion, things like that. My kids are playing with that today to be able to do that on device so that you’re at this sporting event where you don’t have great network connectivity, you can still get it done and send it to your friends, do it at dinner in the background and things like that. But what really gets interesting to me is when the way you interface with your device changes, and actually Honor announced intent-based UI, which I was really excited about, I know we’ve talked about contextual awareness before and you mentioned it.
It doesn’t take any artificial intelligence to have your phone know that you’re approaching your car and it’s time to go to work and it’s going to tell you there’s a traffic accident on the way you’d normally go to work, go a different way. You don’t need AI for that, but to be able to go and look at your phone. Have you ever been to any kind of dinner or a social event where you’re in a conversation and someone talks about something that reminds you of a picture that you just want to show them, or some file that’s on your device?
Patrick Moorhead: All the time.
Chris Moore: It takes you a minute to go find that file. By the time you find the file, the conversation’s moved on. Well, maybe I’m the engineer here who that happens to and…
Patrick Moorhead: Dan usually, on dinners, he’s on his phone all the time.
Daniel Newman: Hey. Be nice. Although you bring up cars also. Ah, let me show you that one.
Patrick Moorhead: Yeah. Exactly.
Chris Moore: Right. But it takes a little while to find that file. But with AI now, you can actually just type in and search for the picture you’re looking for as opposed to actually going through and streaming through and getting 96 pictures and then finding the right one.
Daniel Newman: So give me an example. It could be like me and my orange car, my name, in this year… Because it’ll have been trained on a model, it’ll know your date. It’ll-
Chris Moore: Just like you can build a picture based upon what you type, you can go search for a picture that already exists based on what you type.
Daniel Newman: So get much, much more-
Chris Moore: Much more granular of what you’re looking. Didn’t even take notes.
Patrick Moorhead: You guys are connected.
Chris Moore: But again, over time, intent-based over time, the phone can even get to know you and what pictures you look for more often. So you may not even have to type in the whole thing, it can start automatically looking for what you’re looking for.
Patrick Moorhead: So it knows to put a little bit more effort maybe into training on that subset, just to be ready, maybe caching it somewhere where it has a higher likelihood you’re going to be looking for it.
Chris Moore: Or the concept of federated learning. So federated learning is like when you talk to your favorite OS, hey, so-and-so, and it knows that it’s her voice and it’s not your friend’s voice that’s standing next to you. Oftentimes that’s done with federated learning and that’s where learning on the device is customized towards you. Well now imagine that learning on the device is customized towards what you pick on a menu, how you work out, when you do go to work, when you pick up your kids, what sporting events you’re interested in.
All these little things that we parse day to day, if your device actually started understanding what you enjoy doing and what you do for health, what you do, today’s my wife’s birthday, so I better do something for that. Right? What does she enjoy doing? As your device gets to understand you better, it’s going to be able to customize the data that you are looking for and it’s going to be able to make your life easier and better as well. So the entire concept of your device understanding you I think is really the next generation.
Not only that, imagine just being able to talk to your device and it understands the words that you’re saying and it’s not like how many times have you tried to talk to your device and then it goes to the cloud to try and search for something and didn’t search what you’re actually looking for. With generative AI, the ability to parse the language, understand you, understand what you’ve done in the past and precisely find what you’re looking for, it’s going to completely change the way things evolve.
Patrick Moorhead: No, it’s exciting. So Chris, we’ve talked a little bit about longer-term. We’ve talked about kind of what’s right here in front of our face. You do a lot of planning. I mean, quite frankly, you have to make decisions three years before you ship a product, architecturally before that, to do that. And yeah, you could make some packaging changes here and there that they can get closer in. But what are you expecting for the next year? What are some of the challenges related to AI that the industry, or with Micron, needs to address to help this move forward?
Chris Moore: I think, Tera Operations Per Second per watt, TOPS per watt, is one of the most important things. How much compute can you get done without killing your battery here? Because all of these models are going to drive a lot of TOPS, which are going to naturally drain your battery. And they’re also going to drive a lot of interactions as we already described, between the storage and the memory device itself.
Patrick Moorhead: Yes.
Chris Moore: So as we can take the storage and the NPU and the memory subsystem together and optimize that for not just performance, but performance per watt, that’s the main focus right now. Another example of a next generation. Wouldn’t you like to be able to, I would like to be able to just take my phone, not have it attached to the cloud, and record meeting minutes, and then actually go through and take those meeting minutes and parse them and tell me what the action items were and the major things.
So you can do that today attached to the cloud, but if you’re a small and medium business owner, you don’t want that necessarily going to the cloud. You want that data to be close to you. But if someone’s talking about social security numbers, there’s just all sorts of security reasons you want it done in the edge next to you. We want to really focus on how we can optimize the storage and memory system with our partners in the SOC world to make sure that this is all optimized and some owner of some small business can just put their phone out there, record it, and then not have to have another employee go write it all up or have to spend hours themself writing up what the meeting minutes were and following up on it.
Those types of applications, again, are really going to change how people use their phones. People may not think it as revolutionary, it’s just going to be a new, I mean, people kind of look at this stuff evolutionary, but in order to make that happen, we got to make sure the battery doesn’t die halfway through the meeting. So that’s again where we’re really focused. How can we make sure that we can load this thing up and not kill the battery over time?
We do this sometimes through standards, standards bodies do a lot of driving that, but there’s also a lot of innovation like what I just described for the One Button Refresh. A lot of that innovation is proprietary that we do ourselves, and I got to tell you, we’re running Llama 2 right now and we’re trying to figure out how all this stuff works and, what can we put in our firmware that no one else will have to make it better next year and year after? So it’s a lot of fun stuff.
Patrick Moorhead: So, to put an extra exclamation point in that, I think the future operating system, and I think we’ll probably see in the next year, we’ll be taking snapshots of everything that we do and users are not going to want to pump that all over the cloud. And the benefits you can think of, there’s only four or five major ecosystems today. They don’t work together, but this could be the glue that pulls together the context across all of those major ecosystems to deliver value to the user.
And that’s going to take an incredible amount of storage capability. It’s going to take an incredible amount of performance at low power because we want everything, we want to have our cake and eat it too.
Chris Moore: That’s right.
Patrick Moorhead: So you and the industry, we’ve done great work, there’s a lot of work to be done, and this is one of the most exciting times I’ve seen in a long time, where you have to buy a new phone not just because it breaks or you don’t like the color or something about the camera’s still important, but it’s something besides the camera that gets you in to get a new phone and not wait four or five and sometimes six years.
Chris Moore: It’s a new set of tools within the tool belt, right?
Patrick Moorhead: Yeah.
Chris Moore: I used to work with someone who called the phone today a great Swiss Army Knife, does a lot of things kind of well. Well now we’re adding a whole new set of tools to it that’s going to actually work really well. And to your point, today we’re seeing an average capacity increase about 4 to 8GB on the memory side for an AI-based phone. So if you were going to have an 8GB phone, it’s going to be 12. It could go to 16, it could go to 24, you can receive 24GB phones out there. There’s no AI running, there’s no models running in an OS yet. Pretty soon there’s going to be at least two models running at a time in the OS on top of all these apps. So the need for more memory, faster memory, as well as on the storage side, it’s absolutely coming. It’s really exciting.
Patrick Moorhead: Totally.
Daniel Newman: It sounds like it’s setting up very well, Chris, and I think hammering it home, you got an efficiency and power opportunity. Then of course you have compute productivity gains, opportunity that comes with pairing the right memory to all this compute. Congratulations on all this success. Congratulations on the announcement and exciting times for memory. The boom bus. This is a boom cycle and we are very excited to watch Micron’s next wave of growth.
Chris Moore: We are too, and we’ve got the best engineers in the world. We figure this stuff out. So it’s going to be a real fun ride.
Patrick Moorhead: I love it.
Daniel Newman: We’ll be watching. Thanks so much for joining us.
Chris Moore: Thank you.
Daniel Newman: Hey everybody, hit that subscribe button. Join Patrick and I for all of our coverage here at Mobile World of Congress 2024 in Barcelona. For this particular show, it’s time that we say goodbye. See you all soon.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
