On this episode of The Six Five Webcast, hosts Patrick Moorhead and Daniel Newman discuss the tech news stories that made headlines this week. The handpicked topics for this week are:
- CrowdStrike Global Meltdown
- Meta Won’t Do GAI In EU Or Brazil
- HP Imagine AI 2024
- TSMC Q2FY24 Earnings
- AMD Zen 5 Tech Day
- Apple Using YouTube To Train Its Models?
- NVIDIA Announces Mistral NeMo 12B NIM
For a deeper dive into each topic, please click on the links above. Be sure to subscribe to The Six Five Webcast so you never miss an episode.
Watch the episode here:
Listen to the episode on your favorite streaming platform:
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: The Six Five weekly show is back and we are unencumbered by any CrowdStrike issues. We are live. Did not get blue screens when we went into our PCs this morning, but yeah, that is just one of the topics we’re going to be talking about, but I got to introduce Daniel. I mean, which bunker? So are you in Hawaii hobnobbing with Michael Dell and Marc Benioff or are you on some Caribbean island hanging out with CEO of JP Morgan or something? Where are you?
Daniel Newman: Oh, man, I wish I could tell everybody, but when you’re flying at these altitudes, you got to make sure you protect your location, but I might be somewhere where the water’s warm, where your soul feels free, where the sun shines on your face and turns you red like a tomato. I’m somewhere nice. I’ve been on a board offsite, but I am getting a little bit of vacay in here, been playing a little golf, getting a little water and soaking in the sea. But Pat, I appreciate you pointing that out. It’s been a nice week, and I can’t believe it’s already Friday. It’s been a week since I left.
Patrick Moorhead: I know. It’s great. First of all, I’m really happy to see you actually take time off. I mean, I didn’t take time off until, I don’t know, two years ago, but it’s nice to see that. We all have our different ways of working. For some people, it lowers their stress to come in and get a little work done. Even when I was in Europe for two weeks, I did email triage a couple of hours a day, come in and get stuff done, but the one thing I didn’t do was do any video calls. But Dan, we made it 224 episodes without getting canceled. I’m really excited about that. And we’ve got some great topics for you. We’re going to start off-
Daniel Newman: Getting closer, getting closer to getting canceled.
Patrick Moorhead: No, not at all, dude. I mean, you’re, I don’t know.
Daniel Newman: We did make it though. It’s 224, man. You know what? I just want to say to everyone out there, there are some things that add stress to my life on the weekly, and there are some things that take stress off weekly. This always kind of takes stress off. I enjoy these conversations. I enjoy doing this pod with you. It’s kind of the best part of every week. And I’ll be honest, when we’ve had to miss it, it’s felt like the missing part of my week, my completion and bringing it together and talking to all of you. We really appreciate the community. By the way, buddy, there’s a lot that happened in the last 12 hours because you and I were like, “Do we do this? Do you take off?” And then I don’t know, the whole freaking internet went down, which seems like something that we need to jump on and talk about.
Patrick Moorhead: Yeah, let’s talk about it. We got a great show for you. We’re going to lead off with the CrowdStrike global meltdown. Microsoft applications were a part of that too. We’re going to be talking about Meta pulling back on GenAI in the EU and Brazil due to regulatory concerns. We’re going to break down HP Imagine AI 2024 event. We’re going to be talking about our favorite thing, which is chips with TSMC coming out with earnings. They’re really a bellwether for most of the industry. AMD had a Zen 5 Tech Day, and we’re going to break that down. A lot of conversation about Apple potentially using YouTube to train its models, and Dan’s going to hit that. And then I’m going to end the show talking about NVIDIA and Mistral coming together to create their own small model and NIM.
So let’s dive in here. I’m going to call my own number here. So unless you’ve been living under a rock, I mean, literally on X, literally 75% of all the content I saw this morning when I woke up at 6:00 A.M. was about it being down. Now, what’s interesting is it appears there’s two independent things going on here. First of all, Microsoft had an apps outage, a lot less information on that, but a Microsoft executive did comment in the Wall Street Journal about it in fact being an issue, but the biggest of all issues is CrowdStrike. So CrowdStrike is a broad-based security company, and one of the things they do is they secure devices, and one of those devices is PCs, Macs. Apparently, what happened is a corrupted .sys file was automatically updated, and a .sys file, if you’re unfamiliar with Windows, is basically the core elements or the most important part. It’s about as low level as you can get and directly talks to the operating system.
What this did is it created a bunch of blue screens of death, lovingly referred to as BSODs, and it has taken down major airlines, railways, hospitals, financial institutions, even news. Sky News was down, apparently, and I saw just a ton of flights that were canceled in India, in Europe, and in Asia. So yeah, this is a freaking meltdown. New information is coming out, but my first question is, how does an enterprise do a global update on a .sys file without doing an air gap test, and is that for deployment, speed? I had a couple people respond to me in X and said, “Listen, this is for zero day where you have to be almost immediate.” That’s question number one.
And the second question I have, because many of these systems are still online, modern enterprise Windows PCs have BIOS-based tools, where through a management console, the IT teams can do updates as long as they have access to the BIOS. Long as the BIOS is not corrupted, they can go in and they can do updates to files on the fly. In fact, they can even reboot systems remotely. So knowing that these PCs aren’t alive, it leads me to believe that, A, they don’t have this management capability in place and it’s likely they’re using older PCs. We saw this with a lot of the zero days where they were using Windows 98, Windows 8, actually, yeah, Windows 8 that did not have many of these software-based capabilities. Again, we’re going to get more information as it comes out. But my overall message here is you always have to have an OSHBT plan when your first line of defense goes down. You need a way to very quickly find a way to go back to a known good state out there.
By the way, that’s just not for PCs, that’s for enterprise SaaS applications that get updated. Most enterprise SaaS applications platform have a major update that happens every month. Sometimes it happens every quarter when I look at the enterprise SaaS players, but you need a way to get back to a known good state, whether it’s a PC, a server, a router, whatever. It’s interesting, Dan, it really puts these companies like Cohesity, as an example, into play that have air-gapped and Commvault, who have these air-gapped capabilities of being able to get back to that known good state. We’ll be following this as it moves forward. Dan, what are your quick takes?
Daniel Newman: I mean, look, you and I would probably be the first to admit that we’re not cybersecurity experts. We have people that we employ on our teams that really focus on this. This seems to be somewhere crossed over between the impact of CrowdStrike’s massive deployment and the number of endpoints and systems that it secures on a global basis and the extraordinary impact that can be had from a small mistake that could be made on the developer side. Someone that’s pushing code out. To your point, we are always up against this immediacy of handling the threats. Again, for people that are out there in our community here that aren’t really into security, the threat environment is so much more substantial than most people tend to understand, the number of threats, the speed of which threats are being created and the risk that companies, enterprises have. We hear about how companies, banks, healthcare systems, these data breaches, these are not companies that have unsophisticated and a lack of resources working on security. These are companies that are targeted daily, every minute by nation states and bad actors.
Now, ironically, from what we understand, and again it’s early, the full news, the story will evolve and we’ll hear more about this, is that this wasn’t something that it wasn’t done by an attack. This wasn’t someone breaching or breaking CrowdStrike’s perimeter or shields or getting access to its device servers or some core application. This was a self-inflicted wound. So to your point, Pat, when that happens, it is really fascinating to me this … Remember the old triangle? Was it speed, quality and price or whatever it is pick any two?
Patrick Moorhead: Yeah, it’s a product management thing or at least that’s the way I always looked at it.
Daniel Newman: To the point of people who spoke to you, it’s like, “Yeah, I get the point of speed over backup and double,” but really, as you are developing, pushing these next updates and codes, you don’t have an environment to test something in very quickly just to make sure it doesn’t break anything, it seems to me like we’re going to … You’re going to find out that there was some error made that did not follow protocol or chain of command that created this breakdown, and the breakdown was substantial. To your point, look at how many things it broke. Pat, it’s incredible. I mean, it really is incredible that a mistake in losing access to all these endpoints, losing access to all these systems that one company can secure can literally take down industries. It can take down entire industries. I mean, literally, planes are not going up in the air right now. People are not able to check in for their flights, they’re not able to get access to their bank accounts. Broadcast companies like Sky aren’t even able to broadcast the news right now.
By the way, it’s taking hours to bring it back up. This also shows the complexity and the significance of our systems that even after we know it’s wrong and we work to push the fix, it’s not just flipping a switch. You and I have lived our lives resetting. Everything that breaks, you just reboot it. In this case, a reboot is not enough. This is fascinating. We’re going to keep following it. I think Will Townsend on your team-
Patrick Moorhead: Anshel Sag will be on the analysis too.
Daniel Newman: Anshel, Will, and then on our side, Shira Rubinoff and Krista Macomber lead our security. I’m going to be looking for them to be posting some more in-depth as to what happened from a security side, and then of course, our developer folks will talk about where developers potentially went wrong, look for more notes out of our team over the next couple of days.
Patrick Moorhead: Good stuff. Good adders there, dude. Let’s go to the next topic here. So regulation has been a big topic related to generative AI, how that data is captured, how that data is processed and how that data is used here. Some countries have put what at least I consider some onerous halt or penalties so large you wouldn’t want to take the risk to do that. And as a natural byproduct of that, you have corporations who decided, “Okay. Let’s not do this.” So Dan, what is going on with Meta in the EU and Brazil?
Daniel Newman: Look, I think this is the first of many, not the last of an issue. So we talked about the security and speed. We have another issue that’s going on that’s kine of these two parallel paths of innovation and then privacy. So as we’ve seen in the past as we’ve launched social products, data-driven products, advertising products, the EU in particular, but also other countries around the world, in this case Brazil, have been somewhat steadfast to create what they communicate as a policy to protect their citizens and constituents from technology that maybe goes too far, that takes too much data, doesn’t clearly articulate how data is being used, potentially gives monopolistic powers to companies inside of certain regions, prioritizing products and services. Well, now, we’ve got the same situation going on with AI. So as we know, the US and a number of other countries, but specifically the US is aggressively wanting to have the technology leadership position in AI on a global scale. The vast majority of the large language models are being developed here in the US. The implementation of AI into these services, whether it’s Apple intelligence, Meta’s Meta AI, whether it’s OpenAI, these things are all aggressively challenging, things that have not yet been regulated or no policy has been defined around.
For instance, we really don’t have use rights policy defined yet around large language models and training. And we’re going to talk more about that actually when we talk about Apple, but the net of this is that basically Zuckerberg and Apple or, sorry, Meta is saying, “We’re just not going to launch our multimodal models in these regions.” And that’s, by the way, the point. They’re not saying, “We’re not going to do anything.” They’re kind of saying, “We’re not going to do the most cutting edge and make the most leading capabilities that we’re developing available in these markets because what we feel as a company,” and this my paraphrase of Zuckerberg, “is that we feel like we’re basically walking into a mine, a minefield,” that basically the EC, European Commission on competition that tends to regulate and has a revolving door of fines and taxations on big tech companies, no actual intent to really stop them, just find them as a fundraising activity is going to basically wait for them to launch these things and then they’re going to try to throw the book at them.
So he’s basically saying, “Look, we’re not going to do it.” And also in this case in Brazil, we’re going to hold out, we’re going to wait, we’re going to deploy these in markets that are more innovation friendly like here in the US, and we will grow that way and we will continue to give what I would call degraded or less viable technology to the EU into those markets in order to protect themselves from what could be massive fines. In some cases, these fines that the EU regulators put on can be percentages, up to 10 and more percent of all turnover of these companies if they don’t follow these rules. Now, that’s never happened. It’s never landed there. But in settlements, you’re talking about Microsoft, Meta, Qualcomm, Google, Amazon, have all seen multi billions of dollars over the last two decades of fines out of the EC for basically pushing innovation that is seen as somehow anticompetitive or breaking the rules of privacy and data.
So this is probably a first of its kind though, Pat, of a company actually coming out and just saying, “You know what? We’re not even going to do it.” In the past, it’s like they always do it and then they wait and see what happens. I think this is going to come down to a situation of what is the constituents, what are the people of the EU want in terms of not being behind on technology versus companies saying, “We don’t want to take the risk.” So it’s a really interesting kind of inflection and maybe the first of many where companies are going to say, “We’re just going to degrade the product and roll out less, and if you’re going to keep regulating us, we’re just going to give less technology and good luck building it there,” because as we know, nobody, Pat, is building this kind of technology in the EU.
Patrick Moorhead: Great breakdown there, Dan. This is classic barbell type of stuff that we’ve seen historically where there’s one end who is moving really quickly, and I don’t know if you want to call them decels or, I mean, all regulation is not bad. Imagine, we had no regulation of our highways and our airplanes and stuff like that, but classic historically, even when you’re looking at electricity and the safety and the value that that brings in, there were a ton of discussions about even AC versus DC. Thomas Edison was involved. I mean, a lot of historical things to look at here. I think some decels might say the only people that should be doing this should be the government like nuclear weapons and the potential damage that they can cause. So here we are, but I’m glad we’re having this conversation. It’s going to give bureaucrats more things to think about. Obviously, it’s giving tech companies more things to think about. One thing on a reputation is the EU is really getting a reputation that not only are they hindering innovation and they have very high tax rates and don’t really incent for innovation, companies, if you start your startup, that’s not where you want to put your house up. There’s a lot of other-
Daniel Newman: And if you do, buddy, you don’t stay, meaning the biggest ones ultimately start there and then they defect to somewhere where it’s more favorable. I looked at some charts on this. I mean, there’s nothing going on there. There’s no unicorn companies. It’s a handful.
Patrick Moorhead: And there was even discussion during the Figma meltdown with Adobe where it’s like if you’re a startup, do you even do anything for sale in the EU or what country do you go into? And then you’ve got UK Brexit it out and their CMA, right seems to be wanting to, I don’t know, lead with even more regulation than the EC. So anyways, this is natural. This is the discussion. I sit on the camp of, A, let’s accelerate. Increasing innovation has always had an effect on lowering prices and providing more things for society that are positive. Let’s jump into our next topic. HP had an event in New York City called Imagine AI. Unfortunately, you and I were not able to attend.
Daniel Newman: We had our teams there, right?
Patrick Moorhead: Oh, we did. Anshel was there and got the full download. I know he’s got a write-up coming, but wanted to break that down. What were the news? So first and foremost, HP brought out a new AI PC Copilot+ PC called OmniBook Ultra. Now officially, I don’t think it’d be called a Copilot+ PC because it doesn’t have the operating system support. The AMD Ryzen AI 300 series does not have an operating system. It will likely at the end of this year. Some of the highlights here is AMD’s new Ryzen AI 300. It has 12 cores. Interestingly enough, similar to Qualcomm’s high-end. Has 55 tops versus 45. So I don’t know what exactly we’re going to do with all those tops yet, but when it does get an operating system, potentially it could make recall work better when that gets out, potentially faster image creation, potentially more accurate speech-to-text and translation, potentially faster RAG capabilities, which by the way, kudos to HP for putting a built-in RAG capability. It currently has 100 megabyte limit, which shouldn’t be limiting for Word docs and stuff like that, but very limiting for stuff like presentations and images on device, which is really cool.
Battery life for this, 21 hours. That’s compared to 26 hours from Qualcomm. HP also brought some stuff out for their Workstation platform. One thing I’ve been really impressed with is HP’s ability to create together not just a hardware platform with HP AI studio for Z, but also its end-to-end workflow for developers. And what they did is they added, really leaned into trust framework, security, and collaboration. So other developers working on GenAI projects can collaborate with people and not just on the local area network, but it also plugs into things like Microsoft GitHub. So the last thing I’m going to hit on here is they did roll out some collaborations with some pretty cool apps people, beautiful.ai for presentations, Locus for data management and analysis, OmniBridge for integrating different AI tools, one called Polymer that’s all about productivity and one called Virtual Sapiens, I love that name, which gives coaching for communications. So pretty cool. Wish I could have been at the event, but we had that fully covered.
Daniel Newman: Yeah, and I mean, there’s a lot there. Look, I think HP’s following the path of being able to address a different set of markets. One of the interesting pieces I read about HP Imagine AI was how the company’s really focused on AI for the persona, meaning, look, we’ve got this kind of inflection with AI devices where it’s like where’s the value. You and I have talked a lot about this. We’ve got new devices, they’re all Copilot+ or they’re AI PC or you kind of name and define them. What are people doing with them? What are they using? What’s the thing that’s really important to them? Does everybody care about high tops? What about long battery life? That seems to obviously matter to some people. Which applications do they need to have access to? One of the things I read was that HP is really focused on sort of hitting those different personas. They’re focused on kind of saying, “Look, we’ve got expert types that are technical. They’re going to need a certain high performance type of device. We have other people who are hybrid-focused. It’s going to be all about driving longer battery life.” I think this provides the platform to start the sell-in process, Pat, because the thing is what we want to really understand is how fast does this cycle move, how quickly do companies feel obliged to get in on these new next generation devices, and as they put these next generation devices in play, are they able to get value that makes it worth the spend and what they’re spending. So Pat, I think that overall, my take is that HP is moving in the right direction. So it was a positive event for the company.
Patrick Moorhead: Yeah, I agree, and I am really enjoying my OmniBook.
Daniel Newman: Hey, hold on one second, Pat. Just get us started.
Patrick Moorhead: Yeah, you got it.
Daniel Newman: All right. I’m ready again. Let’s do this.
Patrick Moorhead: Yes, it was your topic, so I didn’t want to get into something-
Daniel Newman: Sorry, I have people at the door, people knocking at the door. They want to know if I was ready to go for a cruise into this magical ocean.
Patrick Moorhead: That’s great, man. It’s real life, dude. Hey, let’s jump in here. TSMC bellwether for CHIPS in the industry. Dan, what is going on? By the way, if you’re in Taiwan, you do two earnings, very similar to what they do in South Korea, but this was the real earnings, not the preview.
Daniel Newman: They give a preview. They gave it about a week and a half ago on the revenue, and then of course, they came out all in. Look, I mean, Pat, would you have been surprised to find out that they beat their expectations and that they slightly raise their guidance and that they are telling the world that they’re having a very hard time dealing with the outsized demand for AI? Any surprises for you there?
Patrick Moorhead: None, buddy. That’s not a shocker.
Daniel Newman: So I mean, the first thing is that their preview on numbers looked positive. What did C. C. Wei have to come out and really have to say here? Look, he said he’s never basically seen demand like this. Their customers want to put AI in everything they’re building right now. Of course, we saw the whole tech market fell a bit on its head the last couple of days. The former president Trump came out and talked about some sort of tariff tax for the protection that the United States is giving to Taiwan. Look, it’s interesting because to me, it’s a bit of a nothing burger. I mean, look, there is some real truth to what he’s saying. The US and the importance between the US and Taiwan relationship does definitely create more of a pause for China, which has been doing drills in the South China Sea for how long preparing itself for some sort of potential attack on Taiwan. Taiwan manufactures over 90% of the leading edge chips, which basically means that if there was some sort of change of control or power that was to take place in Taiwan, it could completely break the supply chain, it could disrupt all the innovation. And no matter how many great companies like NVIDIA and AMD are designing chips and Qualcomm here in the us, we would not be able to manufacture those chips.
So I know this isn’t the earnings conversation, but there is a tie together as to why these great results have not yielded a higher tidal wave of growth in the stock price because there are some concerns as to what policies might come into place, and as it’s looking more and more probable that there could be a second term of Trump in office, he did live his word. When he said he was going to do it with China, he did it, and the question is is now what he’s saying about Taiwan. What does that mean? Now, I will put on the other side of this pat that I do think that the outsize demand and the sort of insatiable demand for AI, if we ended up putting some tax or tariff on exports out of Taiwan, it would just get absorbed. Right now, look at the margins and what people are paying. They could just charge more. I mean, I really do say that, and I know I’m a bit cynical and it’s probably a little different on the handsets and PCs than it might be on a data center server, but I do think most of the price would get absorbed. The problem is who actually ends up paying for it. I think right now, just a quick political thought on this is what Trump is basically saying is, “I’m going to bring jobs back to the US.” That’s a commitment that he’s made, and this is one of those things that has been completely offshore, he’s saying. By the way, the CHIPS Act was all about this already. This isn’t some new thing that he’s all of a sudden started. We wanted to bring more-
Patrick Moorhead: Well, Trump started what was the CHIPS Act? In fact, and this is public, I can talk about this, I did consulting for the DOD when Trump was in office the first time on all of this stuff.
Daniel Newman: So as you and I are running through those, the metrics though, is look, we need to bring I’ve said 20 to 30% of the leading edge out of Taiwan. It needs to move elsewhere, and ultimately, it would be pretty good if we got to parity. That doesn’t necessarily mean Taiwan has to shrink. This doesn’t have to be a lowest common. It can be a highest common denominator where the chips base and the market for everything from HP high bandwidth memory all the way to AI processors and XPUs. I think the rumor yesterday that Broadcom is going to make one for OpenAI. This scale means we need capacity. We need it in the US, we need it in Europe, we need it in Israel, we need it in other parts, maybe Japan and other parts of Asia. So anyways, long story coming all the way back around TSMC here, the demand is outsized. 50% now is at the leading three and five nanometer. They’re considering moving some of their five nanometer capacity to three because so much of this AI silicon is being built on three and the demand for that is sold out for over two years now. CoWoS sold out for over two years now. Basically, any concern about this sort of chip AI thing, this infrastructure thing being a fad that’s going away isn’t showing in any infrastructure commitment.
So TSMC is indicating that NVIDIA should have a good quarter. AMD, its numbers are going to be a little bit interesting based upon some of the concerns on order cancellations, which we’ll probably have to come back and talk about at some point, but the AI infrastructure build out is real. When it becomes a consumption layer that’s real and measurable and driving productivity, that ingestion period isn’t the digestion, ingestion period isn’t fully understood, but this spend on infrastructure build out doesn’t seem to be slowing and TSMC’s numbers were very indicative of that.
Patrick Moorhead: The thing that really stood out for me is literally we’re saying we are sold out for 2025, and I look at the specific words used. My takeaway is we could be sold out for 2026. This has to be a positive sign for Intel, absolutely 100% has to be a sign for them, and it’s got to be a positive sign for Samsung and their foundry services business. There is just no possible way for us to not conclude that that’s the case. You are going to have to work with Intel, Intel, Intel Foundry Services, and you’re going to have to work with Samsung Foundry. You’re going to have to spend the money to put that in there and that could be tens of millions of dollars. There are design oddities and flow changes you have to make in a design. You can’t just do a copy and paste between founders. I wish it were that easy, but it’s not. One great proof point for that is when you look at cadence and synopsys and their tools, they have to be specifically optimized for a certain foundry and a certain node and flow. Anyways, positive sign for-
Daniel Newman: By the way, really interesting if I could say just that you pointed that out. I mean, C. C. basically said no joint venture, which again, that could be just posturing, but maybe, I mean, I don’t know if they need it, but two, Pat, that where does that excess demand go because even if TSMC goes all out on building more, it’s years. So what I’m saying is the immediate demand, the Samsung and Intel opportunity you pointed out is really, really substantial. So while I know a lot of people have been down on Intel, I mean, if they can just get it right, if they can just get it right in terms of the outsized demand and take all that in, there’s a huge growth opportunity for them on that side of the business.
Patrick Moorhead: Think of it takes three to four years from the decision and I am going to start building a foundry to pumping out wafers. Packaging takes less time, but if you want something like that, you’re looking at three to four years. That’s the timeframe, I think more like three years we see from TSMC. If it’s a new city like Columbus, Ohio with Intel could take four years. Anyways, let’s move to the next topic. Imagine this, more chips. AMD had its Zen 5 Tech Day and they made some announcements that to me were expected. They were a drill down. So Computex AMD rolled out Zen 5. They talked about some parts like the Ryzen 9000 series and the Ryzen AI 300 series that would leverage it, and I just want to go in that. So if you recall, my biggest question was, okay, 16% IPC improvement and that’s instructions per clock. So put frequency aside, for example, what architectural changes do you make to something to increase its single core instructions per second? And as you’d expect, they made some enhancements to the front end. They made some improvements to the execution units itself, which the purists would say, “This is the only way you can get to an IPC improvement.” That’s like increasing instructions per cycle up going from six to eight as an example. There were some back-end enhancements, and this is all about IO increasing the data bandwidth, stuff like that, even improving your AVX implementation, and this time with 512 helps on vector performance and some other things, which it’s interesting.
I can debate whether it’s IPC or not, but optimizations made with four nanometer and even three nanometer processes also can play a role, particularly if you can improve. Now, that’ll improve performance. I’ll debate on whether that’s an IPC improvement, but it is a positive sign. So I’m going to dive into this Ryzen AI 300 series. Again, this is the chip that the HP OmniBook Ultra is going to be using. What’s interesting is AMD says it has 50 tops, and that’s the XDNA II architecture, but HP said it has 55 tops. I didn’t have time to figure out what the differences were, but it’s something I’m going to do. One big announcement they made that was unrelated to the parts itself was the appointment of a new leader, which is Rahul Tikoo. Now, I knew Rahul when I worked at AMD with him and then he had a leadership role at Dell with PCs, but he is now the new senior vice president and general manager of the client business unit. Ironically, I had lunch with him yesterday here in Austin, and I think he’s going to be an amazing addition to the leadership team at AMD. AMD was running so lean on the amount of people. By the way, running lean is good, but sometimes running so lean that you’re not getting stuff done and have the inability to get stuff done, that’s the flip side of it. So again, mostly pricing reactions. There’s always negative like, “Oh, the price is too high with the Ryzen 9000,” or cash latency. I love it when non-architects get into architecture discussions, those who have never actually been an architect. So anyways, those are my thoughts. Dan, what do you got?
Daniel Newman: I didn’t have a chance to attend this one, so I don’t have a ton to add, Pat. I think you hit a lot of the high notes. I mean, look, AMD is already working on Zen 7, so this stuff moves very, very quickly. I do agree with your point on new appointments, new leadership. Right now, it’s really contentious, and our lab, our Signal65 lab at our second beautiful child that we’ve had together has been busier than ever this year doing the assessments because we’re dealing right now with everything as it comes to these performance and efficiency claims and these constant new launches and releases. It’s what keeps the industry vibing and moving. So I don’t have much else to add to what you said, but I do think the innovation pace continues to speed and the ability for these companies to meet the performance and efficiency expectations is going to be critical. It looks like AMD is continuing to do what it does here, and I’m eager to see how this stuff lands in the market. So I’m going to keep that one short and sweet.
Patrick Moorhead: Sounds good. Let’s move to Apple potentially using YouTube to train its models. Apple has what some would say is a pristine reputation for not stealing stuff and using it for themselves. Others might say Apple steals an incredible amount of stuff, like IP doesn’t pay its vendors, like Qualcomm tries to make them go bankrupt, but this is an interesting one, Dan. Break this down for us.
Daniel Newman: They only done that a couple of times and never to anyone quite the size of Qualcomm. I mean, look, there was this investigation that came out that was making claims that Apple Intelligence had been trained on YouTube data. Something like 170,000 videos is the number that’s running around the internet. A lot of these are from these super influencer types, MKBHD and Mr. Beast. I think it was 170,000 videos and it’s focused on the subtitle was really what it was trained on. So it’s the subtitles, which was the language. Now, long and short, Pat, is, is Apple doing this? Yes. Is Apple doing it for Apple Intelligence? And what they’re saying is no. This is really important because Apple launched what it called the OpenELM model, and that was a research contribution. So they were starting to be involved in this LLM development and advancing open source. So Apple calls that a state-of-the-art open language model.
Now, when Apple talks about its OpenELM, they tell the markets that this was only done for research purposes. So this wasn’t designed to power Apple Intelligence. So I guess the first question, Pat, is … By the way, there’s a group, and again, I’m not sure if you’re super familiar with it, it’s called EleutherAI. Now, EleutherAI is basically a large collection that’s kind of I guess as a sort of a nickname of the pile, and this is supposedly a pile of sort of real data. Now, one of the sub issues that’s going to come out as we continue to proliferate a large language model and model development is going to be we’re going to run out of real human-created data to train models on. So what’s going to start to happen is we’re going to start creating and training models on synthetic data or data that’s already been created by AI. So EleutherAI, this pile has been created is to be a pile of real human-created content and data that can be used for research and for the purpose of companies like Anthropic, Apple, NVIDIA, and others to train models on.
That was a little bit of an audible from the play about Apple, but the long and the short of it is, yes, Apple did train on YouTube. It took subtitles from 170,000 videos that were done by high-profile influential voices on YouTube, but they used it to train a research-focused open source model that is not being used to feed their Apple Intelligence platform. So does that make it okay? I mean, I think the question mark there is that I don’t know that that makes it okay. I don’t know that all these companies that are using this content for free without any royalty or licensing agreement for the benefit of research makes it okay. But, Pat, look, this is a fundamental issue that sits at the very top of the LLM development cycle where were all these models created, right? We all remember the very, very popular video of the OpenAI CTO talking about whether or not the Sora model was trained on YouTube to which was probably one of the most awkward interview interactions that I’ve ever seen when you know somebody’s looking at you just lying straight to your face, but I guess by you could call that a lie of omission.
So long and long is that Apple’s not not guilty, but they might not be guilty either. They’re kind of following the protocol that everyone seems to, which is trying to find high value real human-created content to train models on to then deploy into research ecosystems to try to create better models. But Pat, here’s my caveat. Do we really know? Do we ever really know? I mean, they can say they’re not using it in Apple Intelligence. How’s the average person ever really know that that’s true to be the case? I think right now, there’s a lot of trust that goes on, and we found out whether it’s been Apple, it’s been Adobe, it’s been OpenAI, it’s been Microsoft, it’s been Google, that there is a lot of gray area as to what data can be used, what should be used, and what we’re seeing in the results that we’re getting.
Patrick Moorhead: Listen, I think Apple is brilliant in how they responded to this, right? I mean, when in doubt you blame it on your research group, you blame it on junior people, you blame it on somebody. You put together a process that you fired. I mean, it’s-
Daniel Newman: Pat’s about to go hard. I can feel it.
Patrick Moorhead: No.
Daniel Newman: Was I too nice? Was I too nice?
Patrick Moorhead: No, no, no. You were actually, I think, quite balanced in that.
Daniel Newman: Weird.
Patrick Moorhead: No. I still have PTSD from working in corporate America. I have seen every excuse. I’ve seen the blame game. I don’t know if you’ve heard the two envelope joke about the new CEO comes in and one of them talks about, “Hey, be sure to blame it on me if things go south.” I mean, we see responsibility here, but it is, to be fairer, I mean, blaming on your research people. I don’t think there are and should be any restrictions on research groups being able to come in and hoover data, as long as none of that gives value to something that ends up to be a commercial entity. I have a hard time believing that what Apple’s Research Group is doing and the insights they’re gaining won’t help Apple sell something they’re going to make money and margin on. I mean, it’s kind of like OpenAI is a non-profit corporation that’s worth a hundred billion dollars, right? I mean, it’s kind of farcical and it’s kind of funny, right? Anyways, I think we’ve drained this topic as much as we can.
Daniel Newman: Listen, man, I thought we’d launched into that one though, that you were just going to come just crushing down on them, but basically, is the TLDR, if I wanted to give the sound bite here, is that they’re all equally as full of or not full of (beep) as the others when it comes to this.
Patrick Moorhead: Beep.
Daniel Newman: Sorry, I just ruined this one. We’re going to get a censorship on this one.
Patrick Moorhead: We’re going to get explicit. We’re going to be explicit.
Daniel Newman: We hope that everybody out there will stay with us and realize that I was just saying that out of love for the industry because I am a techno-optimist, Pat. I like to think it’s all good.
Patrick Moorhead: You’re a decel. We know the real you. We know it. We know it. Hey, just kidding. Dan, we need to get you out to the golf course here, my friend. Let’s go into the final topic. That is the NVIDIA announces Mistral NeMo 128B NIM. What is that gobbledygook? So first of all, Mistral is a model company. We all know what NVIDIA is, and they co-developed a 12-billion parameter NVIDIA inference microservice together. Well, that’ll be out later, but you can get it on the AI service today. So essentially, what they did is they came together, and this model was trained on the NVIDIA DGX cloud AI platform, and it leveraged the NVIDIA tensor RTLLM and the NVIDIA NeMo development platform to do this. So what does all this mean or, actually, let me give you some of the deets here. You can run this model locally. This is targeted for enterprises. It’s very small. You can even run it on classic, what would be considered NVIDIA accelerators for machine learning, not for large language models. So you can run this thing on an L40S. You can run this on a consumer RTX 4090, an RTX 45, even an RTX 4500. It is distributed via hugging face with an Apache 2.0. It’s available now as a service from ai.NVIDIA.com, and the NIM is expected soon.
So what can this model do or, actually, what’s the benefit of having a smaller model with higher accuracy? First of all is you don’t have to run something on a $30,000 card. You can run it more like on a $5,000 card. And what can you do with this? This is for chatbots, conversational agents, multilingual translations, co-generation and summarization, and basically reasoning and world knowledge type of stuff. So this might be something you would want to use for customer service or if you wanted to put a front end in human resources. So pretty cool, but first and foremost, by the way, it’s FP8 as well, which means it takes less resources. Obviously, you want to dial. That’s not as, let’s say, accurate as FP16, but uses around half of the resources. NetNet, we talked about software being the real biggest mote that NVIDIA has. I’m convinced that somebody can create very competitive hardware. We’ve seen it from AMD and I’m expecting that from Intel, but when you look at the entire solution and going from low-level drivers to libraries, to machine learning frameworks, to LLM models deployed over NIM, you have a very, very large mode.
Daniel Newman: Yeah, Pat, it’s such a large mote as I’ve had to talk to a number of media outlets about it that their ability to outinnovate the market by years is creating this vacuum of pressure, but I mean, is it really their fault for getting it right? I don’t know. I mean, look, in the end, we need to be able to deploy models that can commingle public data and private data, and they need to be able to do so efficiently to create text and chat and generative content and assets. And the bottom line is that they’ve done it in a way that’s more effective and efficient, and this is just one example of that. This is the way these complex, high-technical debt enterprises that are full of data that want to be able to write software to a GPU to create an application to benefit from AI. This is the package, dude. This is what we’ve got here. So look, the NetNet is what you just said. I mean, look, they’re doing a lot of things right. They’re making it easy, they’re making it accessible. By the way, they’re creating forces of stickiness that are going to outlast the innovation of competition. So that’s it. That’s my take. They’re on a roll. Time to call it a day and get out to the golf course, buddy.
Patrick Moorhead: Yeah, here are the famine knocking. I want to thank everybody for tuning in. We need to get out early so Dan can go golf and show with the family. Dan, that is not a knock or a criticism. You work way too hard and you need to take a break. So with that said, thanks for tuning in. We will see you next week. Hit us up on X, hit us up on LinkedIn, give all the compliments to me and give all the criticism to Dan and we’ll call it a day. Have a great weekend. We really appreciate you. Take care.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
