The Six Five team discusses NVIDIA Updates Robotics Chip & ReDrafter
If you are interested in watching the full episode you can check it out here.
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Daniel Newman: So it’s funny, maybe as a bit of background. I don’t know if a lot of people realize this. So we had this crazy market run up over six months and guess what stock hasn’t really gone up at all. In fact it’s down over the last six months?
Patrick Moorhead: Well, AMD.
Daniel Newman: Well, NVIDIA.
Patrick Moorhead: Yeah.
Daniel Newman: NVIDIA. So NVIDIA had this meteoric rise and then they had this split and then they rose and I mean the company became the most valuable in the world and this and that. I mean it’s still over $3 trillion. But interestingly enough, as we’ve had this melt up, this big bubble of assets, Far Coin hits a billion dollars in market cap and the HAWKTUAH coin. But I’m saying the overall market, the broadening out, the cubit company that used to be a beverage company, it hit 1.56 billion in market cap, INQ, it went to $43. I mean Inquantum, on less than $100 million of revenue. Not a bad company, actually a very interesting company. But having said that, we’ve seen this frothiness in the market. Well NVIDIA, who’s actually grown into this really low multiple, has kind of gone quiet. And I think right now it’s an interesting moment it to kind of say, what is NVIDIA up to? And we all know Blackwell is ramping, but what else is NVIDIA doing? And that’s a really interesting thing.
So we have CES coming up and it seems with some of the events and some of the stuff that NVIDIA is talking about publicly about CES that there might be some interesting news coming. But NVIDIA also pulled something out of the oven this week. It was literally Jensen doing his thing. Pat, you and I both talk a lot on this show about AI and just edge computing as a whole. Well one of the big opportunities, when you think about where we’re going to head, whether it’s small robots in factories or humanoid robots, is going to be building these powerful AI computing platforms for the edge. And so there’s been a product from NVIDIA called Jetson Orin and they released a new one that’s about 80 times more powerful now. It’s a $250 edge device, comes in eight gig, four gig versions, 1024 core, 32 tensors, just kind of running down the specs. And basically it’s used for all kinds of different edge robotics use cases.
This is a big expansion opportunity. What I’ve been saying over the last several months is NVIDIA needs to start talking about what else, what are the other ways it’s going to make money? It’s not that the data center stuff’s not amazing, it’s just, are we going to see robots? Are we going to see more energy go back into automotive? Are we going to see the next wave of personal computing, gaming, et cetera from NVIDIA? Where does the next big moat come from? And of course all this stuff really ties together. So this announcement was incremental, but when you start looking at the volume of robot, the volume of edge AI use cases, it’s another really big opportunity for revenue stream. So that came out of the oven, literally. That’s how Jensen presented it. He pulled it out of his oven and walked it out and put it on the table, this $250 device.
And really quickly, another announcement, which I think is another one from strange bedfellows, Pat, was Apple and NVIDIA announced this week something which they’re calling ReDrafter was a collaboration between Apple and NVIDIA and using a novel technique for accelerating text generation in LLMs. What they’ve basically done is worked together to make an optimized, something that’s going to be done on NVIDIA hardware using beam search and dynamic tree attention. These are kind of technical terms, but it’s really about multiple text sequences, which is beam search. So obviously every time you put a query in, there’s endless number of options of the text that can be created. Speeding that up speeds up inference, improves performance, and then obviously reducing the redundancies. That’s the dynamic tree, is that not having redundant overlaps in those expected sequences, which is the text that it is used to generate.
So basically, they’ve integrated this into NVIDIA’s tensor framework and effectively what they’ve done is make this run 2.7 times faster in generating tokens versus the way it’s done today. Probably what’s most interesting though about this whole thing to me, Pat, is just Apple has been almost obstinate about doing anything with NVIDIA. They’ve stayed away from NVIDIA, any public kind of notoriety about utilization of NVIDIA hardware. They partnered with Google to do all the work they’re planning on doing with Apple intelligence on the hardware side. And they’ve recently announced, or at least, I don’t know if it’s publicly announced by Apple, I think it was just a rumor drop, but was that Broadcom is going to be helping Apple develop its next generation AI chip. So kind of a big win. Another thing is NVIDIA has perpetually felt it’s under appreciated for what it does in inference. And so these kinds of technologies, and of course a partnership with Apple probably should be seen as a positive catalyst.
Patrick Moorhead: So, Jetson is 10 years old and a lot of people have forgotten this and there’s been a lot of market thesis about when is the edge going to take off in a monumental way? And I think that NVIDIA, over the past two quarters, even on the earnings announcements, have been dedicating a portion of the time to robotics. You accurately said that it is one of the next big things that could hit, automotive could be another one, but gosh, Qualcomm is just giving it some heavy-duty competition there and they need to move this forward. So I think this is part of the reminding people that there’s a growth. We don’t just do data center, because there will be a time when that percentage growth goes down and there might be even a time that quarter on quarter, the actual number goes down. And this is just they’re playing the long game here and investing on what is going to be the next big thing. I think this is pretty aligned with what’s happening in China with robots. And by the way, China has about 100X more robots than we do here in the United States. It’s aligned with what some of the forward-looking thinkers, like Elon Musk are driving forward. Elon will probably use his own chip, probably a derivative off of Dojo, but we will see on that.
One thing on this Apple, NVIDIA thing, Apple did announce at Reinvent that essentially they had already gone big with AWS and they flashed, they’re testing out Trainium 2, they’re using Graviton, and they’re using multiple services from AWS. So this Google, Apple thing might just be on the training side, but one thing, while we’re here, there was also an NVIDIA announcement. I mean NVIDIA continues to invest to create essentially its own hyperscaler. Some of them have been by proxy, kind of a tier two CSP, look at the investments in CoreWeave and other companies. You look at AMD’s investment in Vulture. So, the market is aligning, NVIDIA is reacting to these companies standing up their own ASICs. So really, really cool stuff.
Daniel Newman: Yeah, absolutely, Pat. And yeah, it’s interesting to kind of think the partnership on the training side, I mean ultimately it delivers better inference, but Apple’s got a lot of work to do to build these models that are going to be compelling. The early readout on intelligence is pretty meh.
Patrick Moorhead: Interesting though, Dan, they’ve actually gotten accolades on the research but they can’t seem to convert that into meeting the expectation of what Apple intelligence is supposed to be.
Daniel Newman: Yeah, I can’t actually figure it out, Pat, and this is kind of an interesting segue to the next topic, but I just can’t figure out, Apple’s almost to 4 trillion. Smartphone shipments aren’t up, their AI stuff isn’t particularly impressive so far. I mean their sales are like multi years ago right now on iPhone. I don’t understand it.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
