On this episode of The Six Five on AI, hosts Patrick Moorhead and Daniel Newman are joined by Samsung Semiconductor US’s President Jinman Han, head of Samsung’s US Memory Business Jim Elliott, and head of Samsung’s US Foundry Business Marco Chisari, for a conversation on Samsung‘s approach to artificial intelligence.

Our discussion covers:

• Samsung’s overarching AI vision and how it’s shaping the future of technology.
• The strategic approaches Samsung is employing to navigate the AI landscape.
• The target markets Samsung is focusing on within the AI sector.
• Challenges Samsung’s customers encounter in AI implementation and how Samsung addresses these issues.
• An overview of Samsung’s specific AI products or offerings and what makes them stand out in the market.

Learn more at Samsung Semiconductor.

Watch the video below, and be sure to subscribe to our YouTube channel, so you never miss an episode.

Or listen to the audio here:

Disclaimer: The Six Five webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.

Transcript:

Patrick Moorhead: The Six Five is back and we are talking about AI with Samsung. I’m joined by my incredible co-host, Daniel Newman. Daniel, does it seem like we have been researching, and pontificating, and doing videos all on AI for the last nine months?

Daniel Newman: It does. It feels like it’s been a focus for you and I for much more than that, but it feels like over the last nine, even maybe 12, sometime around November of 2022, it became the only thing. And Pat, it isn’t the only thing, but it is really important. And getting the stories and understanding how companies are approaching AI and talking to their leaders has to be something that you and I are doing to make sure the market knows what’s going on.

Patrick Moorhead: Exactly, and there’s so many ways you can take advantage of generative AI, machine learning, deep learning from the software to the services, to the semiconductors, to memory, to doing fabrication for all the semiconductors, and everything in between. And it’s With that, I am pleased to bring on Samsung’s Jinman, Jim, Marco: welcome to The Six Five. Great to have everybody on the show. We have some first-time guests to The Six Five. But we also have some veterans with Jim and Jinman. It’s great to see everybody. Welcome to the show.

Jinman Han: Thank you for having us. Always exciting. Excited to see you guys in this great platform, Six Five.

Patrick Moorhead: I appreciate that. Is it better than F1? I don’t know. It was great seeing you there too.

Daniel Newman: I think we can do it all. Pat. I think F1 is a great place to hang out and have some social conversations. I also think that The Six Five is a great place to analyze the AI space and talk about what companies like Samsung Semiconductor are doing. And it’s great to have all you here. And Jinman, I want to start with you. I want to talk vision with you. Every company right now is responsible, if you’re in technology- really, if you’re using technology- to be talking about how AI is shaping the future of your business and your company. That’s no different for Samsung Semiconductor. Love to hear from you. What is your vision in terms of how AI is shaping your business?

Jinman Han: Well, the vision is a pretty big word. And you know what? I have a chance to attend APAC meeting, discussing with colleagues from Japan, USA, all different countries. And we were having a similar conversation, and I told them our vision, Samsung’s vision for AI is we really want to be a major hardware supplier, so to speak. For this AI industry. USA is very good at having great engineers for software, but all the great AI software also needs much, much better, efficient hardware. So, unless you have a great bedrock in hardware, it’s really tough to have the best performance out of AI software. So this division, we have memory, foundry, systematic side. We even have packaging capabilities which will be essential in providing great hardware platform to all the customers in USA who are right now working very hard to come up with great AI software capability. This is our vision. Very simple-

Patrick Moorhead: No, I love that. And it’s funny you hit on something that Dan and I talk about a lot as it relates to hardware versus software and services. You’ve heard the expression “software is going to eat the world”, which by the way it does, but what is it going to run on air, right? I mean it needs solid hardware to create all those incredible experiences for both a consumer and enterprises. And just as a follow up, Jinman, to the vision question: I’m a mission vision objective strategy person. And I’m curious, can you describe some of the more finite areas of your strategy and how you’re going to achieve that vision?

Jinman Han: Okay, so right now everyone’s talking about AI capability in hyperscalers. But these days a lot of our customers are starting to talk about on-device AI, especially on smartphone and PCs. And when we are talking about large language model for own device edge application, we also have to think about- it’s a kind of a oxymoron statement- but we really need small large language model so that they can be into the small devices like smartphones or PCs. And at the same time, we also need this challenging requirement from our customers. Hey, we’re going to put our small large energy model, which is about 10 billion parameters, which is not as big as 200 billion or 175 billion large models, but 10 billion model parameters are not that small, actually.

But to fit those model in smartphones or PCs, you also need to have a very low latency at the same time with very low power. So that’s the challenge we are seeing right now from our customers. So in that regard, it’s almost always about memories. So memory has to be very fast, it has to have a big density. So delivering low latency, very high-speed data to the AP or system on chip at the same time with low power: that’s the challenge we are now seeing. And in that area, I think Samsung is in perfect position to supply those technologies to our customer. So that’s the immediate requirements that Samsung is seeing right now.

Daniel Newman: Yeah, it’s really interesting, Pat, you and I have talked no less than it feels like a hundred times on this show about the symbiotic relationship between memory and the rest of the computer architecture and how effectively it doesn’t always get the attention it deserves, but try running these really advanced workloads without enough memory and without low enough latency performance, enough memory. And this is why we always end up turning back to teams like yours in this particular industry.

Because we know, and the world needs to know, the important relationship. Marco, I’d like to jump to you and after you, I mean I’d like to hear from all of you, because I realize you all have slightly different roles in the company, but where do you take this strategy to? What are the markets, what is the approach that Samsung Semiconductor has to be able to win? And what customers are you going after first and how do you see that evolving?

Marco Chisari: Yeah, that’s what Jinman said. The pillar of the strategy is to offer the best possible hardware to all these AI machines. And we do that really both providing a leading edge technology from a foundry perspective. The industry is today physically manufacturing a three, two millimeter with a roadmap to 1.41 millimeters for the most advanced and demanding processor unit. And then, of course, we offer that platform in conjunction with memory for HPM or advanced memory. And we stitch all together with advanced packaging, 2.5, 3D packaging or a combination of both, which is 3.5. In doing that, we address a variety of market and a variety of customers of course, first and foremost the data center AI. And we do that targeting the largest player and hyperscaler as well as a lot of AI startups, which are now developing a very differentiating basic solution and very promising ones that I think will come to the market in the next 12 to 24 months.

And then we do that on AI, as Jinman will say, we’re really very focused on AI on device because we believe that AI will grow as a market for interference on device targeting a smartphone, PC, and out. And these three markets and these three types of devices will completely transform in terms of user experience of AI. And so we offer all these super advanced technologies that stitch together and ultimately we’ll be able to give our customer and our end market the type of technology boost that they need to address the super demanding computing AI application.

Patrick Moorhead: Yeah, on-device AI is, it’s not a question of if it really is a question of when all the things, all the magic tricks we want to do with generative AI require this data to be on the device. And even in Western cultures who have no problem sharing everything, they’re not going to want to share every text, every image, every conversation that they’re having in real time, they’re going to want that to be on the device. And those 10 billion parameter models really come into play-

Jim Elliott: I think that’s true for the device and also within the enterprise as well because if you think about on-device AI at the enterprise level: if you’re doing code debugging or having AI review your financial plans, things of that nature, you don’t want to use publicly available data. This is all going to have to be kept in-house. And so from that standpoint, you think about on device AI, it’s at the client level, PC and phones, and these devices are going to be smarter and smarter, but it’s also, you’re going to see it at the enterprise level on-prem with internal data sets to maintain that IP confidentiality. So it’s going to be very, very interesting. And especially as the data sets become larger and larger and heavier and heavier, the density of that data becomes very expensive to move all that around, which means you’re going to need potentially faster and higher storage capacities at the edge as well as at the client level.

Patrick Moorhead: So Jim, as analysts, right, we don’t actually have to execute any of this stuff. I did for 20 years, so I had a real job. Analysts hate when I say that by the way- but it is hard to do it. And I’m curious, what are some of the challenges that some of your customers are having getting to the stage of taking advantage of AI for their businesses or for their customers’ Customers?

Jim Elliott: Yeah, I think at the end of the day, in the short term, it’s all about LLM, large language models, in the cloud, right? So there it’s all about bandwidth and higher capacity to be able to build out and utilize these models. I think longer term that’s going to shift over to more of a edge device on-device AI type of model where you have to, in addition to storage size, and capacity, and bandwidth, you’re going to have power requirements as Jinman was touching upon. Now also, a lot of these big LLMs in the cloud are limited by the amount of power that a cloud company can get access to. So that’s also something that’s coming into focus.

But I think, ultimately, the way I see AI sort of building out over time, it’s sort of like the democratization of AI where an AI platform will be opened up to hundreds of thousands or millions of potential external third-party developers that will be able to create new AI-based applications for consumers that will be as ubiquitous today or in the future as the app platform is that kind of burst onto the scene like 15 years ago. I think we’re going to see that same type of activity with the democratization of AI as this starts to become monetized in the future.

Jinman Han: And also one issue that keeps coming up during the discussion with our customers is all about power consumption. They are really concerned about power consumption of this powerful GPU or even memory high bandwidth memories. So, one time I asked the executives in AI GPU firms asking how I can solve the problem, they said, well Jinman, that might be the reason why we have to build this large language model to ask questions, how to save power for humankind in the future.

But it’s really a real problem right now with the we are facing. So, to address those issues, our customers are talking about all different kinds of new memory architecture combining with GPU or SOC design. So, in the future we might be able to see totally different novel memory architecture combined with GPU or CPU so that we can address those power issues at the same time increase performance without putting too much power.
So it’s going to be a very, very interesting period, especially for us, for that industry.

Marco Chisari: And look, maybe I’ll add, Jim and Jinman brought up a critical point with respect to product performance and power performance, which of course now are front and center and the most critical points to address for the industry. But there is an increasing sensitivity now for customers with respect also to geopolitical aspects, because of course AI is going to be a very transformative technology. And I think customers are also getting increasingly sensitive on how to source the hardware component. They want increasingly diverse supply. You want increasing diversification from a geopolitical point of view. And that’s also where Samsung has a lot to offer. And when we feel the customer, of course are very responsive, extremely responsive to that, to those to the component as well.

Daniel Newman: It’s very prudent that you brought up all of these particular points, Jinman, starting with sustainability, we are hearing the same thing talking to enterprises and large organizations. It’s a problem. There’s shareholders and stakeholders expect the companies to be making this part of the calculus in their decision making process. And then of course the geopolitics are very substantial and it’s in the news, it’s a headline every single day. We want to be working with trade partners that share value and that is something that is going to continue to be of the utmost importance.

And in the process of doing all this, we still need very performant applications and computing architectures that enable our businesses to keep their technology and global market leadership. And so, there’s no sacrifices here. And I’m hearing from you that there’s no sacrifices on your end in what you’re building and developing. So, let’s talk about that because kind of talked roundly about what you’re doing, vision and strategy. We’ve talked about the challenges of your customers, but let’s talk about what you’re offering. Jim, I’d like to direct this one to you, but if you could share a little bit about what is the Samsung portfolio, what is the offering that you’re bringing to market that is going to win for you to execute against this vision?

Jim Elliott: Yeah, so from a memory standpoint, it’s all about having a very, very wide portfolio. And I think as Jinman touched upon, you’ve got CPU architectures, GPU architectures, TPU architectures, and as well as custom asics that are being driven for these sort of purpose-built AI applications and opportunities. So, from our standpoint, we want to have HBM, HBM three, HBM 3E, which is the fastest speed and better power footprint, being able to be able to accommodate high bandwidth type needs, high-capacity DRAM like DDR5. And we just announced a 32-gigabit monolithic device. And without getting into too much detail, really what that allows us to do is build a higher capacity DRAM module that uses less power.

So this allows us to produce at scale for performance applications that aren’t going to need to use as much power. We also have graphics, GDDR6, GDDR7, and then LP5 as well as looking at UFS 4.0 as you get out into these clients or edge-type devices. But in addition to having that wide portfolio and leading edge technology portfolio that Samsung is always well known for and renowned for, it’s all about the ability to scale production for these sort of hyperscale level and mass markets that we’ve been talking about. And that’s really where Samsung is able to leverage our financial, our technology, and our production wherewithal to bring these AI products to market for both cloud enterprise and client products.

There’s another point to be made there on the 32-gigabit monolithic. So today to build a 128-gigabyte RDIMM DDR5, you have to use a 16-gigabit and then utilize TSV or through silicon via, which is, you guys may have heard, a big bottleneck for HBM production. So, if we can alleviate that or mitigate some of that by producing a 32-gigabit monolithic: number one, you get less power. But number two, you get better overall production efficiencies because you don’t have to utilize TSV technology and you can free that up to be used elsewhere. So, it’s kind of a win-win solution from an efficiency, from a cost, and from a power perspective.

Patrick Moorhead: Yeah, I appreciate you going through what I call the diversity of memory. I mean, years back it was one type of memory and this is what you got and now-

Jim Elliott: Now it’s more complicated.

Patrick Moorhead: Yeah, I mean we’ve got mobile memory, we have GDDR, we have HBM, we have specific memory that operates better in the data center and everything in between. And this pervasiveness of memory types I think reflects how hard it’s been for the industry as a whole-

Jim Elliott: That’s right.

Patrick Moorhead: To get these gains from everything. And, quite frankly, what we’ve done is we’ve gone to accelerated computing, is not only have we pushed some of the burden onto the programmer, but we pushed the burden onto the memory makers, which by the way is a good thing for you.

Jim Elliott: Your point, I think that the major arc here is going from that general purpose compute to this very sort of application-specific compute, be it in the cloud or at the client. So yeah, a hundred percent-

Jinman Han: Actually a long time ago, I remember buying memory modules when I was going to fried electronics. Don’t say it’s goner, I’m afraid.

Patrick Moorhead: Yeah, I mean a lot of specialty stuff out there, but it’s important to keep those innovations going regardless of what might be happening with some different type of architecture. So Jim, you outlined the different types of pieces that you have and I’m curious, and it’s funny, I’m, I’m going to ask, I’m going to put Marco on the spot here. How is it differentiated? I hear some of the basic terms from other companies and how is Samsung differentiated?

Marco Chisari: Well, as we were saying, I would say the biggest first differentiation is the fact that we can offer so many different technologies as a one-stop shop, right? GRAM, the most advanced DRAM and HPMI. I offer the foundry services. We have an entire new team that will offer advanced packaging. So, you can find a place where you can really put together all these different technologies under a single umbrella. I would say it’s the most important differentiation as AI is going to go chip for economic reasons and you want to manufacture different parts of the chip and different technology notes, you will keep the most advanced and leading-edge technology on the part of the chip that really needs to go at the most advanced performances.

And you will keep everything N-minus one or N-minus two for cost reasons. And same for memory. We also differentiate ourself from a shared point of view of process technology. We have been first in the market with a gateway around. With a gateway around we hope to restart voltage scaling, which is very important for power, which as you know, is critical for the industry as we discussed. And we hope to restart the 20-30% power improvement scaling between nodes. We get around, we have been first to get around, we are already with our second generation in the market and we’re ready to launch our third generation next year.

We think that would be really critical for data center and power sensitive application. We finally provide also, in terms of differentiation, packaging. And packaging we believe is going to be a core differentiation because the type of packaging we’re doing, it’s a clean room type of packaging now. It’s still extraordinarily advanced and stitching these chips together on top of each other in three stacking, it is going to become really extraordinarily complex.

Daniel Newman: Well, gentlemen, I want to say on behalf of The Six Five that we really appreciate you all taking the time. For Patrick and I, we set out to do The Six Five on AI because we felt that every day we were being pitched by companies from chips to SaaS, as we like to say, all trying to tell us why what they’re doing is unique. But what we were feeling was we were hearing a lot of the same things on repeat.

So we said we’re going to go deeper, we’re going to talk to the senior leadership and we’re going to make them really explain what they’re doing. And I think the three of you have done a fantastic job of breaking this down for us. We really look forward to sharing this with our extended communities because what Samsung Semiconductor is doing is very important and we believe the company has a really important role to play in the advancement of AI. So, thank you all so much. Hope to have you back on our show again soon.

Marco Chisari: Thank you.

Jinman Han: Thank you. Thank you very much.

Jim Elliott: Thank you guys.

Daniel Newman: All right, everybody hit that subscribe button. Join us for all of our episodes of The Six Five on AI and our other insiders on the road and of course our weekly show for this episode of Six Five on AI. For Patrick Moorhead and myself, it’s time to say goodbye. We look forward to seeing you all really soon.

Author Information

Daniel Newman

Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.

From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.

A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.

An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.