On this episode of The Six Five Webcast, hosts Patrick Moorhead and Daniel Newman discuss the tech news stories that made headlines this week. The handpicked topics for this week are:
- NVIDIA Siggraph Announcements
- Zoom AI ToS Dust-up
- GlobalFoundries Q2 2023 Earnings
- Salesforce Bring Your Own Model
- Groq Meta LLAMA-2 70B Param 100 tps Milestone
- Qualcomm Wins First All-Electric Cadillac SUV Design
For a deeper dive into each topic, please click on the links above. Be sure to subscribe to The Six Five Webcast so you never miss an episode.
Watch the episode here:
Listen to the episode on your favorite streaming platform:
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Daniel Newman: Hey, everyone. Welcome back to another episode of the Six Five Podcast with my bestie here, Patrick Moorhead, Chief Poobah, Moor Insights & Strategy. It’s episode 179. Can you believe that episode 179? We’ve done like a thousand of these because we’re awesome.
Patrick Moorhead: But in all seriousness, we don’t count the summit in that total. We should. This is episode 1,500.
Daniel Newman: This is episode 1,435. And by the way, but you’re going to be really old. Because we’ve been doing this like what, three or four years now?
Patrick Moorhead: Yeah.
Daniel Newman: Yeah, yeah. We’re probably not going to get to that. But it is Friday. It’s Friday morning. Friday mornings are the best time of the week because this show is the best thing… Let’s just say it’s one of the best things we do. We’ve got a big week ahead of you, ahead of us, I shall say. We’re going to talk about a lot of things that, by the way, not a crazy week of just earnings, which Pat, you know we love, but we don’t love. So it’s nice that we’re going to be back talking about some other stuff. We’re going to be talking about cuGraph and some big NVIDIA announcements. We’re going to talk about Zoom GlobalFoundries. Salesforce says, “Bring your own LLM model.” Groq says to NVIDIA, “We’ve got a challenge for you.”
And then we’re going to talk a little bit about a new all electric gas guzzler. What do I mean by that? You’re going to have to wait and find out. If this is the first time you’ve ever hung with us here on the Six Five Podcast, we want to let you know that this show is for information and entertainment purposes only. So while we will be talking about publicly traded companies, don’t take anything that we say as investment advice. And by the way, if it is your first time, we always have to ask the question why, why would this be your first time when this is the best tech podcast.
By the way, I checked out the rankings, Pat, on a worldwide basis. We’ve been in the top 20 in a number of different markets and around the globe at different times, but we would like to be number one. So share it with your friends and make sure everybody knows that we’re here because we’re doing the analysis. We actually get beneath the news which few people want to do because it’s risky, but we’re interesting. So we’re swimming. We’re waiting in those deep waters, buddy. So before we get started, Pat, how are you doing?
Patrick Moorhead: Hey, man, I appreciate you letting me talk. No, I’m doing great.
Daniel Newman: I took two minutes, 15 seconds.
Patrick Moorhead: No, I know. No, I swigged my electrolyte drink and I’m slugging down two cups of coffee. And yeah, I’m trying to get healthy. I decided to show up for the show angelic white. It’s going well. I haven’t traveled in a week, so I’m literally a different type of human and… Yeah.
Daniel Newman: Can I show you something? My daughter just brought me breakfast.
Patrick Moorhead: That is so sweet.
Daniel Newman: So disappointing that it came in at the exact moment that I couldn’t eat them, but I just want everybody to know my kids are awesome. My 17-year-old woke up, made eggs, sausage-
Patrick Moorhead: So nice.
Daniel Newman: … cheese, avocado, salsa, rocking out a breakfast. You know what I have to do? I’m going to sit here and smell it for the next hour as we go through this podcast, Pat. But you know what, that’s going to motivate me to go fast. Remember when the Six Five was six topics, five minutes each?
Patrick Moorhead: Yeah.
Daniel Newman: That was the idea. Yeah, that’s why it’s called the Six Five. People always ask that question. It’s really not that tough. But I guess the fact that we’ve never done a single topic in five minutes ever. I don’t think we’ve ever done a five-minute topic.
Patrick Moorhead: I think we did one. I think we had an Earnings-palooza where we dove through that.
Daniel Newman: Was it like, “Yeah, we’re at 58 minutes in and we’re only five topics?” So this one is going to be the unfortunate victim of a short coverage.
Patrick Moorhead: Exactly.
Daniel Newman: All right. Well, listen, we’ve got a lot going on and we do have to get done by the top of the hour, because Mr. Moorhead’s very important. In case you didn’t know that. And you got very important people waiting to talk to him. Me, on the other hand, I’m going to the beach. I’m not going to say much more about it than that, but this is the last thing I’m going to do today before I’m going to take a half day today and a half day on Monday, and by the way, that’ll be the one full day off and that’ll be in the longest amount of time I’ve taken off all year. So everybody-
Patrick Moorhead: Buddy.
Daniel Newman: … feel sorry for me. This is my martyr moment. Look at how much I work and how little I rest. Anyway, story of our lives. We’re building an empire here. We’re telling stories. We’re spinning tall tales about the tech industry, and let’s start with the first one of the week, NVIDIA. Doing what NVIDIA does, disrupting itself because no one else is even close.
Patrick Moorhead: Oh wow, that’s tee up. Yeah.
Daniel Newman: So I want to challenge them, dude. I want to kick them in the, but they make it really hard.
Patrick Moorhead: They do.
Daniel Newman: They’re making it hard.
Patrick Moorhead: So, hey, let me talk about if you’ve never heard of Siggraph before. So first of all, this event is 50 years old, okay? And it’s all about research artists, developers, filmmakers. It’s all about graphics and visualization and think workstations on the hardware side and then think in the cloud doing workstation like stuff up, up in the cloud. Now, the interesting thing about it is that NVIDIA’s biggest announcements had nothing to do with visualization at all. I scratched my head and I’m like, “Do I not understand the product called the GH200?” Which is basically a super chip, which is a combination of Grace, which is a… By the way, GH means Grace Hopper, and Grace is the name of the CPU and H is the name of the GPU. So it’s a combo arm CPU, an NVIDIA GPU. It’s all about HPC and maybe about AI. So it had nothing to do with that.
I’ll get to that at the very end on why I think Jensen got up on stage there and talked about this upgrade to the Grace Hopper 200, which essentially what it did is it added memory and it gave higher performance memory on the GPU side. Interestingly enough, they pulled back the performance of the memory on the CPU. So a couple interesting things there. I don’t feel like NVIDIA’s trying to pull a fast one. I think it was that they didn’t need that memory performance on the CPU to get what they needed overall for high performance computing and AI training and inference.
So you might also ask a question, “Well, why is it CPU? Why not AMD or Intel as we see with DGX?” Well, interestingly enough, it’s not about the performance of the CPU, it’s all about the memory footprint, right? And I also think that NVIDIA can shave a tremendous amount of cost using this arm neo verse as well. So the other thing it does is it gives a more straightforward ability to connect the CPU to the GPU over NVLink. See, AMD and Intel are not the most motivated to put ports on the CPU to talk directly to NVLink, and they could also turn it off from generation to generation.
So anyways, I just thought that was important stuff. So the new increased memory footprint goes from 96 gigs to 141 gigs and it’s faster memory, and that’s HBM3e. And Dan, you and I, have done podcasts before on high bandwidth memory and what it means and what it doesn’t mean, and this is the latest memory. The CPU memory, like I said, actually got slower, which was interesting. Now, the standard, the form factor for this goes in, which is an MGX, which is a single server, which to me led me in the direction of why would NVIDIA bring this out. And the only thing I can come up with, which is bringing out an HPC and an AI chip at a visualization show, is all about AMD and the MI300 and the value proposition that AMD came out with, right? Because AMD with the MI300 came out with this massive memory footprint.
And let me explain why that memory is important. When you get into large language models specifically, and I also think there’s a lot of other foundational models, you can keep that workload, whether you’re training or inference sitting in GPU memory, that’s really, really fast. In fact, with HBM3e, the fastest memory that you can get in that form factor. So I think this is a competitive play. I don’t know what they’re expecting AMD to come out with. They came out with that big announcement about the MI300 and massive memory footprint. Oh, by the way, them not needing to have two cards to hit that memory footprint, they only needed one. So I’m trying to figure out what that means, Daniel. Does this mean that it’s between AMD and NVIDIA with large language models in 2024? Because by the way, this new system with HBM3 comes out in Q2 2024, which is around the same time that the MI300 comes out. What do you think, Dan? What are your thoughts?
Daniel Newman: I think they’re running predictive analytics and they’re predicting the future. And if you’re the company that makes the most sophisticated AI chips on the planet, I would say it’s a high probability that you’re probably running some site of data visualization that could understand the likely path in which AMD is going to take with its MI-series and where they’re going to need to be competitive. I think NVIDIA has to look at how it can protect its moat right now and how it can give confidence. So if I was AMD right now, I’m out running around and saying, “Look at what we’re doing with memory. Look at how capable we’re going to be.” And of course, there’s a lot of debate right now because the logical way someone’s going to tear this market apart and really create some disruption is going to be price. NVIDIA’s got this absolute foothold on the price and the margin market.
Patrick Moorhead: Yeah, we’ll talk about price a little later when we talk about Groq.
Daniel Newman: Yeah, and nobody wants to obviously kill the golden goose here because you start getting… There’s a bit of market falloff that I think is available on these high-end processors that is going to be had just because of availability, because of the ability to serve the customer. Meaning NVIDIA’s had so much growth so fast, I think AMD and even Intel is going to be able to compel some customers, “Hey, come with us. We’ll give you our focus and our attention. Spend with us. We’re going to support the heck out of you, your specs. We’re going to build around you.” And I think they’re looking at that. And like I said, so if I’m NVIDIA, what I’m saying is what are the most likely areas that we have some vulnerabilities? You have companies like Intel and AMD that are historic CPUs. We all know that for the biggest workloads, it is not just GPUs, it is GPUs plus CPUs. That’s the training plus inference formula for the future.
So you’ve got all these kinds of things coming together and are basically saying, “Look, we’ve got the market. We’ve got the customers. They’re tied in and hooked into our software and our frameworks with CUDA.” Let’s make sure that they see the roadmap and they have no real reason to want to consider changing. So I think that’s what’s happening. I think it’s ambition on the front end. I think the company’s going to continue to push the envelope and force the disruptors to have to be on their toes. If you want to disrupt NVIDIA, I think the easiest way to do it is with price. But we all know because we know this from CPUs, we know this from laptops, we know this from many things in the business is a zero-sum game. So you have to decide how quickly you want to erode the profitability of the business.
So I think the early hope is to keep value high, keep margins high in the early stages because we know downward pressures will naturally occur over time. So great technical insights, Pat. Hopefully I added a little bit of flavor there in terms of what I think maybe this all means for the business. And speaking of what things mean for the business, when you’re not understood and your strategy is sometimes not understood, the market tends to want to tell stories for you. And Zoom had a week, this week. Zoom, let’s talk topic two. Basically got absolutely eviscerated in the press media and by users online for something that I would like to say… I’ll just start with the end. Stephen Covey is one of my favorite authors. One of The 7 Habits of Highly Effective People is to begin with the end in mind. Well, the end is that this big story ends up being a nothing burger, but let’s just talk a little bit about what happened.
Patrick Moorhead: I almost put that in the headline.
Daniel Newman: Nothing burger?
Patrick Moorhead: Yeah, it was good.
Daniel Newman: I put that in my-
Patrick Moorhead: Oh, I know. I’m sorry. I changed it. I probably should have left that in there. It was good.
Daniel Newman: Yeah, sure. Then it would have been able to give me credit for awesomeness.
Patrick Moorhead: I’m giving you credit right now, buddy. If I were there with you, I would patch you in the back.
Daniel Newman: Can I get a high five? Everybody? And that’s just listening. We just high-fived. It’s important sometimes that you walk people through what’s happening when they’re not able to watch the video. The video’s so good, you should watch it. We’re very fun to watch. Sometimes we’re typing or tweeting, often not paying any attention to one another. It’s really funny. I’m kidding. I’m kidding. I digress.
All right, Zoom. So here’s the thing, it’s weird when the outrage of people starts to consume over things that to me seem pretty obvious. Now, basically Zoom came out with a new term of service. And in their new term of service, there was some opaqueness in the language of how it would be able to utilize data from meetings to be able to train AI. Now, I’m going to come back to that in a second. Before I even come back to that, I just want to explain, none of you ever read your term of services. So I’m very interested in who this person was actually that read this in the first place. Kudos to you for taking the time. secondly-
Patrick Moorhead: It was Hacker News by the way.
Daniel Newman: Second, now, please go read the term of service of the rest of the apps that you’re using to find out that the data that you are creating and using for your email, for your web browsing, for your productivity tools, probably your CRM is being used at least in some anonymized capacity to help train and improve the products that you’re using. It’s pretty universal in SaaS. This is why at least originally I was like, “Whoa, why are people so pissed off?” But I think what happened here is there was some big ambiguity in the language that made people believe that they were training their generative tools with the actual meeting content. Meaning it wasn’t clear of any anonymization. It wasn’t clear that this was specifically related to the generated outputs that were being used when people use generative AI tools from Zoom. It just read like, “Oh, if you’re using Zoom, then it’s recording your meetings and it’s training it with the specific data.”
No real clarity on whether your data was safe, whether you’re protected, whether your meeting data could be used or getting into the hands of competitors or somehow be used to train a large language model. I parallel this a little bit to some of the distress and outrage that OpenAI has faced in its early days because there’s a significant amount of clarity lacking as it pertains to when you put data into an LLM or you put data into a generative tool, how that data then gets captured. The overall consensus is don’t put anything in. You don’t want to be out there in the wild. And I think that’s created some paranoia including me, and I’m sure you, Pat, with what data gets put into a Bard or a OpenAI or a Zoom for that matter.
But in the end, what like I said, mostly happened here is it’s a detail and it is all this is about is Zoom using service generated data. Meaning when people are using ML and AI to create generative data, it can be telemetry, product usage, diagnostic or generated text that’s not the actual but generated summaries and stuff like that. It could then take a summary of data to help it train and improve that model. So Pat, I got to be honest with you, I could spend a little bit more time beating this bush. The more I looked at it, the more I realized it was a lot of foe outrage. It was a lot of, oh my God, this is the worst thing that’s ever happened. And it felt to me very targeted, very unrealistic. And anybody that’s using any app, whether it’s a social media app, a SaaS app or any other data app probably is allowing their data to be used for similar functions, features and utilities.
So suck it up people when you’re using these apps, this is what’s going to happen. If you want the apps to get better, they’re going to use your data. If you have a big problem with it, put the app on-prem and use some terrible on-prem based collaboration software where you have 100% control. There we go.
Patrick Moorhead: That was good analysis, Dan. And I want to take us back to 2020 when we were all remote working, doing our thing and there was a big dustup that Zoom had on security. Zoom said it had certain features. It wasn’t routing through China, it was inaccurate on the type of encryption it was using, that it said it was using, I could find a way how I could get there. It was a different type of encryption and people were really leaning into, including myself, how many bits were there. And then that turned into this giant issue. Now three plus years later, right? I mean in 2022, Zoom for Government has Department of Defense impact level four certification, right? And that doesn’t even come up. Security rarely if ever comes up. And when you attach the history of that to this new thing where quite frankly I do believe that there are writers out there who are looking for a scoop, even if they know that there’s no malintent.
Now, you and I have the distinct privilege of being able to meet directly with the most senior management of Zoom. I wasn’t able to make their analyst event, you were. I was there last year, spent a lot of time with Eric Yuan. And for this, I spent some time researching and meeting with Zoom’s chief operating officer and chief product officer to really get the insights scoop of what was going on. And it’s interesting, I’ve been a person when I first saw what Google and Microsoft came out with, there was really no talk about how your corporate data would be used to make stuff better. And I’m like, “Using corporate data is going to be better. This is what people actually want to do.” And I feel like Microsoft and Google held back and Zoom trudged forward into this.
The only reason, again, I brought up the security thing is they handled this so much better this time, right? They hit it very quickly. We’re not going to have Eric Yuan every week or whatever he did talking about security updates because this is a giant nothing burger, okay? And in the end, Eric came out and I put a LinkedIn in the notes that really shows coming from Eric Yuan what the deal is. So I think they handled the issue really well. There is not a follow-up with work. I just think that the company needs to now educate it, try not to over rotate it. And quite frankly, if Zoom could overcome what happened in 2020, 100%, they’re going to overcome this. My fear is that every Zoom AI story from the press for the next three years will cite back to. And back in 2023, right? Terms of service. But quite frankly, that’s just the world we live in. If technology weren’t so important, shaping the world and shaping economies, nobody would care.
Daniel Newman: Yeah, I love that. And thanks for the history lesson how quickly we forget. It didn’t even–
Patrick Moorhead: We don’t talk about Zoom insecurity. Exactly.
Daniel Newman: Well, the Zoom bombings, it was a headline story for a long time. There was some whitelisting that you mentioned that people have quickly forgotten. And to their credit, they made the fixes and they got the right security authorizations. I spent time with Eric last week. The company’s investing, he’s got a great track record he’s built. If you don’t remember, he was the brains behind WebEx as well. This is not the first rodeo and I think the company’s done a good job of continuing to make it easy. So this is a hill to climb, but I don’t think it’s one they can’t surmount.
So let’s jump to the next topic, Pat. The one earnings of the week, and it’s a big one because it does tell a pretty big story about the broader semiconductor market through the lens of the foundry side. So GlobalFoundries, Pat.
Patrick Moorhead: Yeah, so if you’re not familiar with GlobalFoundries, they’re a leading foundry that is global, okay? And they were global before it was cool. Back in 2009, they were a spinoff from AMD and I actually ran corporate marketing when we spun it off and created the corporation with Mubadala who was a huge investor. So boy have they come a long way, right? They’re one of the only foundries with a truly global footprint, but more importantly they do semiconductor foundry work. They’re very focused, right? They’re not a TSMC that is doing everything for everybody. I don’t want to call TSMC the “Walmart,” but they do have this top to bottom. GlobalFoundries is very focused on things like 5G, on power, and on automotive. They do a little bit of compute, a carryover from the AMD days, but not a whole lot.
They’re one of the leaders, if not the leader in silicon photonics. So how do they do? Well, they did pretty well, right? They hit on the higher end of their guide, which I think was really good. Now declines were in areas that quite frankly you would expect, right? What is down in the market? Smartphones. Yeah, smart mobile devices were down 19% year over year. Communications infrastructure, primarily comms infrastructure. 5G is down, data center is okay, but also down. But that was down pretty big. 38%. Personal computing, again, is a knit right now. It’s $52 million on $1.8 billion in revenue a quarter. It was down significantly. But on the bright side, automotive up 200% home and industrial IOT right now, industrial IOT, up home, it down knitted out to be a little bit of a push, but growth, right? About 3%.
So nothing that you wouldn’t expect. There was a lot of talk on the call about these long-term agreements. And when I read through the transcript, my takeaway was, “Hey, we’ve got long-term agreements,” but we’re not trying to make them onerous to the sides and squeeze them to the point where we’re having them. And this is my number– chew into 50% of the wafers that maybe weren’t delivered, right? Or that the customer didn’t need. But the key is that with these long-term agreements, the true view of the market is probably subdued a little bit, right? Because in these LTAs, for a long-term agreement, you have wafer agreements that you have to abide by. But anyways, solid work here. And I’m really interested to see competitively when Tower Semiconductor closes and Intel embraces that and what happens next.
Daniel Newman: Yeah, I think you hit it. This one can get pretty geeky, pretty quick because of the stuff they focus on. There is definitely some great stuff that was talked about that you didn’t necessarily dig into their Lockheed Martin partnership that was announced. They’re doing some interesting stuff on 3D or-
Patrick Moorhead: Hey, man, I had to leave you something, dude.
Daniel Newman: Thank you. Thank you. Thank you. I do appreciate that. But some 3D heterogeneous integrations. Look, Pat, this is probably the most interesting thing just to continue to talk about is we had this great run of tech for the last couple months and it’s sold off pretty hard in the last month. And again, we’re not a stock show, but the market is the market and these companies do very much emotionally react to the market in terms of how they do their business.
And so we had a lot of Q2 is the bottom, Q2 is the bottom. We heard from Pat. Lisa had made similar overtures. Jensen hasn’t had to make any comments like that, but GFS beat GlobalFoundries is beating. We saw Lattice had a good number. Are we out of the woods? Before I just dig, I mean just kind of the question here. How do you feel, are we beating expectations because we put the bar on the ground and now these companies are jumping over it and we set low expectations, these huge cuts in expectations, or have we turned a corner now? And even though some of the macro data is still a little rough, tech’s about to see another explosion.
Patrick Moorhead: Yeah. So I like to separate the market. You have to set memory aside, memory and storage. So memory and SSDs. Just because they’re so off and they’re so crushing. Well, the good news is GlobalFoundries doesn’t do any of that. So then you’re into RF, you’re into different impacts of what hits there. So I think though this shows the diversification of GlobalFoundries and also the impact of these long-term agreements that, by the way, and Tom brought this up on the call, CEO Tom Caulfield. Hey, remember when we couldn’t get chips, we couldn’t get that 1 cent chip to sell that $75,000 car. Heck, there are luxury cars that they couldn’t ship with a radio because they couldn’t get a MOSFET or some analog PMIC out there.
Hey, we’ve got a good question coming in from Jeff Rick. “Hey, one year post chips act impact.” Well, hey Jeff, the money hasn’t been distributed, the awarding hasn’t been done and quite frankly, doing what the government’s done, we’re sitting on our hands right now and companies are waiting to get that money. But some companies didn’t wait. Intel went in and made… Dan, what was it? 200 billion commitment globally, 100 billion commitment just in Columbus, Ohio with or without the CHIPS Act. And what Intel did to defray the risk is they did a couple JVs with some companies to be able to split the financing. So a risk return there. Great question. And we love the questions we get from our awesome audience.
Daniel Newman: We don’t always talk, because if we did, we’d have no time to talk about our topics. But I think since we were asking about the market that was really well-timed. I talked to a few different press outlets about that one year thing. And Pat, I got a slightly different view. Look, the government administration is just terrible at actually executing plans, whether it was Obamacare or the amount of years it took before any of that actually took hold or something as big as the CHIPS act. I think the expectation, at least for the chips part, the science part’s a little bit different because the science part’s a lot more small companies, universities, it’s innovators, it’s VCs, it’s accelerators, but the CHIPS Act is really big mega global companies with huge balance sheets that are going to largely be benefiting from that cash. And so, long and short here is that the science part will be delayed. The CHIPS Act part of it. I just think they know these companies can cover it and they’ll get around to it.
All right, let’s hit the fourth topic. Salesforce says bring your own model. And so, let me talk about this a little bit. What we’ve really had here is a bit of what I would call as a pinging pong of generative AI announcements. Microsoft makes one, Salesforce makes one, Microsoft makes one, Salesforce SAP makes one. Microsoft makes… And by the way, I keep saying Microsoft because they by far are making the most. Now, that’s a byproduct of a couple of things. They’ve got the broadest portfolio, they’re putting open AI and stacking GPT on everything. But at the same time right now, there is a definite aggressive architecture to be that, “Hey, we’re the company building AI into everything.” But Salesforce is the SaaS leader in CRM, and that’s been the case for a long time. The two companies have had a little bit of a debate on who put AI first into their CRM. I won’t try to make that call because they’ll both tell me I’m wrong. So I’d rather just play on the safe side on that one. But-
Patrick Moorhead: You play safe a lot.
Daniel Newman: I don’t play safe that much. Don’t read my market. You don’t read my market.
Patrick Moorhead: I’d love to see you take more shots, but you are younger than me.
Daniel Newman: Are you lecturing me on air? You’ve already lectured me in the comments. Are you going to lecture me on air? Is that what you’re doing? We’re not going to do that here.
Patrick Moorhead: No, buddy, we’re besties. No lecture.
Daniel Newman: We’re not going to do that here. But no, look, I can make the argument for both sides. That’s why I’m good at this. But here’s the thing. Multi LLM or single LLM, or better yet, multi-model or single model, I think we all know where this is going to end up. And I think one of the most interesting things, and going to be the most interesting debates at the end of this competitive story is going to be, did it make sense to go all in and build your own LLM? Was there enough value in it or is the real money in the ability to daisy chain stack and utilize a lot of different models in a very simplistic way? So Salesforce is saying, “Yeah, we’ve partnered with OpenAI. We partnered to have ChatGPT capabilities, but we are going to build our data cloud in a very open, bring your own model capacity.”
So Einstein’s Studio effectively is going to enable people to work across the ecosystem. Sagemaker, AWS, Vertex and others, Cohere, Anthropic, different models. And the idea is you’re going to be able to use the studio without requiring an ETL. You can deploy quickly. You can generate higher numbers of predictions. And of course, they’re saying if you can do these two things as a business, you’ll be able to increase your revenue and bring down the churn in your business. This is a pretty pivotal seminal moment for Salesforce. The company had launched its big GPT moment a couple months back in New York City. They came out with pricing. They’ve basically built this data cloud. They’re letting customers have quite a bit of governance in how the LLM and training is used. I think, frankly, Pat, if you remember six or seven years ago, Marc Benioff pitched the Einstein concept and this is how it went.
It just landed like that and whatever that is. But sometimes it’s like Jensen, he was wrong about enterprise AI until he wasn’t. And I think a little bit of what was here was the sophistication of AI, the models training. It was really in the beginning it was more advanced analytics and ML was really what he was touting. But now with generative tools, with where we’re at with AI semis, with where we’re at with software and cost management, the ability to implement AI into your business applications is very real. And so the claim to be stick is that they had this vision six or seven years ago and now we’re seeing it come to reality. I’m pretty excited about the idea of an open approach to bring your own model. A lot of people have been down on anyone that’s not sort of Google or Microsoft because they didn’t go all in and build it themselves. Those companies are going to be very successful in AI.
But I think in the end, like a hybrid cloud, AWS didn’t say hybrid cloud for a long time and then now they say hybrid and then multi. It happened over a period of time. I think that’s what’s going to happen here with AI. I think we’ve seen AI very monolithic in the models to like, “Oh my gosh, we’re going to need small models. We’re going to need foundational models. We’re going to need large models. We’re going to need micro models. We’re going to have different models for different things and they’re all going to be required to really truly benefit from AI.”
So in the opposite moment, that’s why a bedrock can work and that’s why a Salesforce bring your own model can work. And we’re going to see this ebb and flow between many models, single models and in the large languages even, is one model enough or do each bring something of value here? And then, of course it’s how you deploy it, how you make it easy, how you train. I like what they’re doing here, pat. I think it’s interesting. I think competition brings innovation. You and I talk about this all the time. My Forbes piece says this in the end, the more they innovate, the more they compete, the more we win.
Patrick Moorhead: Okay, we got nine minutes. Two and a half topics.
Daniel Newman: Oh my God.
Patrick Moorhead: So please stop talking. No, I’m just kidding. So first off, it’s hard to determine if Einstein using machine learning and deep learning was successful, okay? But any chatbot that came out six or seven years ago wasn’t that good. It solved a few problems but also created a few as well. Now, Salesforce did go from zero AI predictions per week to a trillion. So I think that is one metric that says it was successful. So a couple things I just wanted to hit that I don’t think may have been covered either holistically or appropriately out there.
So first of all, Deloitte’s not just a partner, they’re actually a customer. So they’re using AWS SageMaker to support the bring your own model Einstein studio. Now the interesting part is I got to do some research on this. I still can’t make the connection from Sagemaker to Bedrock. In fact, I met with the AWS folks last week and there is currently not a connection between those, but what I think in Einstein’s Studio is doing, they are making that connection between Sagemaker and Bedrock. The other thing that didn’t get a lot of play as well, is really the importance of the Salesforce data cloud. I just hired a new VP in principal analyst Robert Kramer, and I’m going to sick him on this. But essentially this gets back to– you have to have– it’s garbage in, garbage out, right? That’s been true for 50 years. And you have to have the ability to take real time and proprietary data, train those, but also toss out and basically tear up the data that you don’t want sitting around, that you don’t want others to look at.
And I’ve been pretty hard on Salesforce the last couple of years, but I do have to give credit where credit is due. They did a really good job showing in their AI day that I know you attended. I watched one video on how the Salesforce data cloud works and also the Einstein Trust layer. I need to do a little bit more research on the trust layer, but I’m hopeful that Salesforce does not make themselves the arbiter of what’s good speech and what is bad speech. They’re letting their customers be able to determine that otherwise they’re putting themselves in an absolute no in situation with the barbell of ideologies that we have out there right now.
Daniel Newman: Well said. All right, we’ve got a little bit of a speed round. So what we’re going to do for these last few topics because we have to get the show pad is I’ll let you hit this one largely without comments and then I’ll close off on the last one and hopefully I’ll leave you just a second. But I talked about this already. Groq made a pretty big announcement about a hundred tokens per user per second. Pat, what does that mean?
Patrick Moorhead: Yeah, so good question. So first of all, Groq is a company that was founded by the folks that did the Google TPU. So smart cookies. And in my vernacular, they’re creating an ASIC to tackle first inference and then training. As we talked about many times on this show, an ASIC is more efficient than a GPU at doing certain things. And then the challenge is putting a programmatic layer on top of the ASIC to make it programmable. And then there’s Llama 2. So Llama 2 is an open source model that came out of Meta that everybody but trillion-dollar companies can take advantage of for free. And essentially it’s all the rage, right? Open models, right? Because we don’t want one company to have their model.
And what do I mean by closed models, right? So OpenAI and ChatGPT is a closed model. Bard is a closed system as well. So now, you have in the enterprise world at least everybody’s saying, “Hey, it’s about a combination of proprietary and open models distributed through somebody like a hugging face.” And then the 70 billion parameter model where they were literally according to them. And I can’t find any data that says this is not, it’s the fastest performance on Llama 2 70 billion parameter at over a hundred tokens per second per user. And the reason tokens are important as tokens determine the amount of data that can go into the prompt or they can go into the grounding.
So this has a lot to do with the pricing as well. So the cool part is that the cost is just extraordinarily lower to do this. And Dan, you hit this on the NVIDIA piece. Groq says that on a workload like this you get three X lower total cost of ownership from the inception, which is really great value, right? Those are comparisons using an 80-node NVIDIA A100 SuperPOD is $27 million, and H100 SuperPOD is $39 million. And a Groq 80 node system is $18 million. So again, competition is good. Dan, that’s a theme on our show. We say it every day. Competition matters. And one final thing, current silicon is 14 nanometer. Imagine when they get to four nanometer or five nanometer, performance and power should be amazing.
Daniel Newman: Absolutely. So I’m going to keep running. I’ll just say in the press release I did comment availability, Pat. I mean, you can actually buy these things. I just want to point that out. These are actually available which and surprise people wouldn’t want to capitalize on that.
All right, last topic. Going to hit it pretty quickly. We’ve got to rock and roll, but Cadillac announces an all electric Escalade IQ looks pretty freaking cool and it’s powered by none other than the Snapdragon digital chassis, Pat. Look, Qualcomm’s had a bit of a shaky couple quarters because of the inventory, because of softer Android numbers, but this is the part of their business that is absolutely ripping, rocking and rolling as a $30 billion pipeline for their automotive design and their partnerships with GM, BMW to really help power these companies in their next generation.
We’ve seen Tesla take off, but electrification outside of Tesla has not really gone as quickly or as excitedly as the other. So Pat, this car looks gorgeous. It’s another example.
Patrick Moorhead: It really does.
Daniel Newman: Really does. I am super stoked. I might even get to see one when I go to the mobility show, but if not, I’ll definitely see one I think when we go to CES. So we’re going to look at that. But Pat, this is a quick hit. It’s just one more win, one more snack Dragon Cockpit platform. Look, that company went from zero to hero in the automotive space. They were not there. NVIDIA was all the rage. This is where Qualcomm has been crushing it. If they can do this in other parts of their business, like PCs, they have a pretty exciting story ahead, Pat. I don’t know, I know we got to run a quick add-on here.
Patrick Moorhead: Yeah, just this was a trifecta win, right? Its cockpit has 5G connectivity and ADAS, which has a lot of content in that car. Also add, my apologies if you said this, this is Cadillac’s first all electric SUV out there. It looks pretty good and it’s funny. I went back and I looked at a video I had posted back in December of 2021 where they were on stage giving a precursor and I love connecting the dots between the big win and the fanfare and then the cookie crumbs that go into that.
Daniel Newman: I just want a Six Five voiceover on the infotainment systems when you’re like, turn left ahead and it’s Pat’s going to be. Well, I believe that left could be the right turn, but it’s going to give you the analyst viewpoint. It’s like an analyst’s reply. But I’m going to tell you, if you make three rights, you might get there faster. So all right everybody, there you have it. That’s this week’s show. It’s 179. We covered it all. We went slow and then fast and we do that sometimes. Pat lectured me in public and in private. I won’t send the personal chats, but you can just be sure that that happened. Look, tune in, hit that subscribe button. Join us for all our shows. We love y’all. We’ll see you later. Bye.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
