The Six Five team talks about AWS re:Invent Compute.
Watch the clip here:
If you are interested in watching the full episode you can check it out here.
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we do not ask that you treat us as such.
Transcript:
Patrick Moorhead: But let’s move on, if we can, to AWS Compute. As Daniel said, we had a chance to actually sit down with Dave Brown and have a great conversation with him. He runs all of EC2 for AWS, and as most analyst do, we also got pre-briefed well in advance of what it was coming down the pike, so we could answer the questions and have some intelligent conversation in forums like these, and I wrote an article on Forbes which hopefully you can check out.
But net-net, what Amazon did is they raised the game on Compute again, and plenty of harken back 10 years ago, where there were companies, Andrew Horowitz saying software is eating the world, and my stacking response was always, “Well, software has to run on something.” The reason why people thought that Silicon was getting commoditized was because the industry was allowing it to happen, and now you have companies like AWS and you have Apple who are strategically using Silicon and to their competitive advantage.
So AWS made a few announcements. First of all, they’re expanding Graviton2 workloads with better networking, better storage, took a really good evolution on that. Also, AWS announced it’s Graviton ready program, which is, enables software partners to offer certified solutions. Those solutions from ISVs, as opposed to being those applications that enterprises are building itself. They also announce Graviton3, which is a brand new SOC, and while Amazon didn’t say it’s using, I believe it’s using the new arm, Neoverse V1, and Graviton2 is positioned in having a 40% price performance improvement versus what was available from Intel. Graviton3, AWS’s promising that it goes even 25% higher compute performance compare the Graviton2, and that’s just for general compute. They’re doubling the size of the SIMV unit, which drives machine learning and things like cryptography and they’re adding bfloat16 which you might remember that Intel added that to the last generation of Xeon’s scalable processors.
It just doesn’t stop there, I mean, 50% increased memory bandwidth that helps all applications like databases that are very IO dependent and put a lot of information in memory. The other thing that the company announced, they gave a little bit more depth on Inferentia. Inferentia just keeps moving along like a train, they doubled the networking throughput to 800 gigabits per second, which is pretty incredible. So that gets you to petabyte scale type of clusters, which I thought was pretty cool.
On the training side, the company brought out Trinium based instances, which was you got sneak peek up last year, and right now NVIDIA dominates in all train. What AWS is saying is they have the fastest and the lowest cost of machine learning training in the cloud providing 40% lower price to train deep learning on the A100, which is the P4D on instances. I mean, if that isn’t huge, I don’t know what is. Now, there have been a lot of companies that have made claims that they do stuff better than NVIDIA when it comes to training and it didn’t necessarily make a dent in the market, will this? I mean, if it does it’s going to be absolutely unprecedented.
I think we’ve seen a few you chinks in the armor with [Abana] in certain training workloads, outperforming Intel. But if this happens, I think it could be a first. The final thing, we just don’t have enough time to talk through this stuff dude, is AWS introduced custom SSDs where they’re modifying the firmware to pretty much go a hell of a lot higher performance than standard off the shelf SSDs, and they added multiple instances to support that.
Daniel Newman: No, that’s a lot to cover there, and obviously we went from AI to ML, now across to the instances. Pat, you covered it really well. I will just add a little bit of sparkle and color here. AWS is doing some strategic things on many fronts very well. They’re building in-house silicon that continues to push the boundaries, and again, going back to my comments where I talked about competition, I feel the same way here. It’s very democratized. AWS has teams that are focused on working with Intel, focused on working with NVIDIA and AMD, and there’s certainly a ton of users on X86, but they’ve also made some tremendous investments in their arm team to build out custom capabilities, whether that’s Inferentia, Trinium, now Graviton3, they’re making it very digestible. Of course, at The Six Five Summit, we launched the Graviton challenge. I hope that we will have another Silicon based launch that we can do together at the next Six Five Summit, that’ll be in June. By the way, we have a date for that, but we’ll have to wait and hold out, we’ll share that with you a little bit later.
But Pat, I have to say we had some really great time with some of their key players on Dave Brown’s team. We talked to things, of course, we can’t talk about a lot of the specifics, but what I can walk away saying is the company is very steadfast on making sure they’re meeting the customer where the customer is. A diverse set of needs, a diverse set of giving flexibility. Of course, like any company that’s building something in house and then selling something that’s not built in house, they’re pushing the boundaries and creating a product that’s as competitive as possible, higher performance, lower power. That’s a big part of the Amazon narrative with carbon neutrality, so that’s going to be a sub theme of what’s going on here.
But Pat, overall, like I said, what a great set of announcements from the EC2 side, from Dave Brown, and of course I’ve got to recommend again, watch our pod, read Pat’s article, and of course I had an article, two talking a little bit about all these announcements. So I promise you if you read the two and watch that one you’ll understand entirely, because Dave brown did a great job breaking it down for us.
Author Information
Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.
From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.
A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.
An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.
