Talking AI GPUs and Ransomware Detection – Infrastructure Matters, Episode 23

Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this webcast. The author does not hold any equity positions with any company mentioned in this webcast.

Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.

Transcript:

Camberely Bates: Hi, folks. It’s Camberely Bates with Infrastructure Matters. Again, I am here with my host, Krista Macomber. Welcome.

Krista Macomber: Thank you.

Camberely Bates: And of course, the great Steven Dickens.

Steven Dickens: Feeling less than great today, but thank you for the kind words. If I get through this without a coughing fit, that’ll be an achievement. That’s the bar for me today.

Camberely Bates: Okay, there we go.

Krista Macomber: It’s that time of the year.

Steven Dickens: Yeah. Well, I was at re:Invent, so with 60,000 other people, so I’m sure I caught something there.

Krista Macomber: Yeah.

Camberely Bates: I didn’t hear any really bad stuff coming out there, but I have been hearing a few colds here and there, but that’s normal, feels normal.

Steven Dickens: Put 60,000 people in an air-conditioned space together and pack them in for four days, there’s going to be some germs that go round.

Camberely Bates: So we’re coming into the holidays here. Things continue to kind of push forward. So we’ve got a few things to get covered, and then we’re going to talk a little bit about some ransomware research that Ms. Krista has been doing. This week, I was at the Marvell Analyst Day, and so for those of you who don’t know who Marvell is, they’re a semiconductor company, silicon company. They were best known many years ago before the executives that are in there, about seven years ago, this new executive team came in and really has changed the company phenomenally. They were very much so in the commercial side of the house, the gaming systems, et cetera, and they are now completely almost all enterprise-based servicing, enterprise as well as super clouds. They’re super hyperscalers. So what was really kind of cool is that Chris Koopmans got up and started presenting, he’s the president, and put up their vision statement that they had written seven years ago, six years ago. And that vision statement is so in line with where we are today, which was we develop and deliver semiconductor solutions that move process and secure the world’s data faster and more reliably than anyone else.

Krista Macomber: Bingo.

Steven Dickens: It’s good to see when a mission statement actually becomes true, and you can go back seven years and look at it. You see so many of these things thrown around, and it’s sort of marketing speak, and it doesn’t actually relate, but it’s always good when an executive leadership team can actually go back and say, “Hey, we nailed that.”

Krista Macomber: On top of it just with tech and how quickly everything develops and changes, six and seven years in the tech space is a long time.

Steven Dickens: Seven days, at the moment.

Camberely Bates: I’ve aged a lot since then, 20 years at least.

Steven Dickens: Six or seven days with the pace of AI right now is what it feels like.

Krista Macomber: Absolutely.

Camberely Bates: They did have their earnings announcements a little while ago. I’m not going to go through those. There is a link to the Marvell earnings that one of our folks had written. But the big area that they had some softness in what I would call the enterprise data center market as opposed to what they call their data center market, and we’ll talk about that. But what really drove the market this year was their networking business, and that gets into moving the data or moving all the information that has to. And where that’s coming from clearly is the same place that everybody else is coming from. It’s the GenAI, the AI noise that’s going on and craze that’s going on. So that’s the big space. The 800 PAM4 networking is driving the growth. The second thing really that they’re going to see driving the growth into 2024 is this custom compute. So they are heavy into servicing the hyperscalers, so think Meta, think Google, the big public cloud places and all of the social media folks and the big guys there that are building their own or want their own silicon built. And so they are building custom CPUs, custom TPUs, fill in the X CPU or whatever that is, as they can afford to have these custom pieces made for them to scale out these big huge environments. So the implication here is you’ll see some of the companies coming out with their own chips or their own silicon, but think of it saying that it’s probably a very highly collaborative IP effort with Marvell. So they didn’t say who, but they listed all the names. So I can’t claim, “Oh, that one has one of Marvell’s IP in it.” But there’s definitely ones that are coming out that are all Marvell IP.

So the other big thing they talked about is the need for an incredibly fast switching for these architectures, the AI architectures, and that’s because we’ve got this east-west design, and it’s spreading out so wide. You’ve got as many GPUs as can you link together, the CPUs require some very, very heavy duty Ethernet and optical. So we’re going to see them competing against some of the other folks that are out there with their Ethernet and optical as we go forward. Then the other, they deal in two big industries, the telco industry, which has slowed it down, so that market was slowed down for them, the auto industry, which is not super picking up because there’s a lot of complexity things that they have to answer. I will give you my coffee mug. Can you see my coffee mug here? Can you see? It’s a fancy-dancy car that they have here, so you can’t really see. But what they’re talking about, and this gave pause for thought, is that they talked about what these big, huge custom data centers look like and what’s in the custom data centers that the hyperscalers. They’re doing image and video processing, they’re doing machine learning acceleration, they’re doing network interconnect, they’ve got general purpose compute, et cetera. And guess what? That’s all what’s in this car.

Krista Macomber: Which is pretty wild.

Camberely Bates: Yeah. So I had an opportunity to sit down with a couple folks that’s all they did was auto, and that was the core analyst space that they talked through about what’s going on and how these things are designing. And it’s not just the Teslas or the new EV people that are coming out with these custom environments that they’re putting in, but it’s all the other folks that are coming out. So we’re going to be riding around in probably more data center stuff in your car than we ever had in the ’80s and ’90s, which frightens the heck out of me, guys.

Steven Dickens: Mobile data center is coming.

Krista Macomber: Even of the last 10 years, I feel like, nevermind going back to the ’90s or the ’80s.

Steven Dickens: I think the pace of that’s accelerating. I got a chance to spend some time with Qualcomm at Mobile World Congress, and I’ll probably do the same this time around. I think what they’re doing with these kind of looking at the car as a platform and attaching all of the diagnostics, the infotainment systems, where maybe they were discrete systems previously, taking a holistic platform style approach, they’re becoming data centers on wheels.

Krista Macomber: Oh, yeah.

Camberely Bates: They are, they are. And that completely frightens me when I just got a new computer, and I can barely get it up and going. So I’m in really big trouble.

Steven Dickens: At least the camera’s working this week.

Camberely Bates: I know.

Steven Dickens: That’s progress over the last week.

Camberely Bates: I know. Okay. So that was the Marvell piece of it. The big messaging on the current space here is the enterprise data center for them. They’re seeing still absorb whatever data storage they have, whatever network they have, whatever they have. The hyperscalers are tearing off anybody that’s doing, they’re latched onto AI. Kind of in the same space of where I’ve had that conversation about is how long is this going to last? Is this a flash in the pan? Are we going to hit the bottom here shortly or whatever? But same as us, they’re saying this is a long range kind of thing that’s here, that application area that’s being invested in long range. So all good, all good conversations with the crew there. So how about you? What have you been tracking, Steve, this week?

Steven Dickens: Yeah, so on the same sort of theme around custom silicon, a couple of big announcements this week. I didn’t get much chance to dig into the AMD announcements, but we’ll talk about those briefly. But the one I did cover, and there’s a research note out, as part of the big headline-grabbing launch this week from Gemini from Google, sort of their competitor to ChatGPT, if you will, as part of that kind of buried in there was the TPU, their tensor processing unit, the TPU v5p and the AI Hypercomputer that they’ve launched. So I sort of wrote a research note covering that. It gets very nerdy very quickly, 459 teraFLOPS of bfloat16 performance, backed by 95 gigabyte or high bandwidth memory. This stuff’s going to be what Google builds and puts all of its Gemini on, so I think it’s fascinating. I think there’s some cost versus performance trade-offs, which is going to be interesting. So that’s going to be something I’m tracking to see the adoption of this. But I think one thing I’m seeing with all of the custom silicon, and you were sort of touching on it a little bit with Marvell, is there’s going to be a lot more options for people to run this workload on.

We were out at AWS last week. What they’re doing with Graviton and Inferentia and Trainium, I think there’s a lot more optionality for people, and that kind of feeds into the AMD piece. I was kind of mildly tracking that. Daniel was out there from our team around the MI300X that they’ve sort of announced this week and really going in on that AI-orientated server piece. Lisa Su did a great job, I thought. I was listening in. Daniel’s probably putting some more improved coverage out there than I certainly would be able to, but you saw that reflect in the share price just yesterday. Obviously, Nvidia’s gone on a tear this year from a share price point of view, but when you look at the market’s reaction, I think they see AMD peeling off a part of that. I think where if we look at the start of the year, it was NVIDIA, NVIDIA, NVIDIA, I think there’s a lot more optionality now what Google’s announced with its TPUs this week, the new updates from AWS last week, what AMD is doing, Intel with Gaudi. There’s various things coming through that I think is making this space more competitive than it even was six months ago.

Camberely Bates: Which it needs to be. We all love competition because it benefits us all.

Steven Dickens: It’s good for the market in general, I think. I think we need a robust ecosystem of different options. NVIDIA is going to be a player in the space, and they still are going to be a player going forward. I don’t see that changing. People need options. They need public cloud options, they need on-prem options, they need various different sort of approaches. The other one that stood out for me was Kirk Skaugen, who was one of the best executives we get to work with, was on stage with Lisa Su. So it’s going to be interesting to see Lenovo on what they’re doing with AMD around that particular MI300X and see where that goes going forward. I think the server model for AI, I don’t think it’s all going to go in the cloud. I think there’s going to be a lot of on-premise deployment as well. So as I say, a lot of respect for Kirk, and it’s going to be interesting to see what Lenovo and that team are doing with AMD going forward.

Camberely Bates: Yeah. So right now most of the AI work is all very much so focused on the compute, whether it’s compute or GPU or whatever those areas are. That was kind of one of the conversations we had, because a key piece of business from Marvell is their work in the interconnect technologies that support data storage, whether it’s the HBA and RAID cards and that kind of areas. And that clearly is the data storage space right now on-prem and a bit in the cloud is still absorbing a lot of the stripments that happened in the past years, and we haven’t seen that uptick in the AI, but we did have a conversation about it. So the discussion point on there is that as we get into using multimodal designs, which means I’m adding video, an image. Right now today, it’s all very much so voice and language, which is very small in terms of data storage. As we get into some of the others, which will be nearing the end of probably 2024, we’ll see that increase.

To that end, somebody else that had a big announcement this last week was VAST Data, and VAST data is an up-and-coming company in the data market, I think they’re calling themselves a data platform now, so I’ll correct myself on there, but they had $118 million series E, very high valuation, $9.1 billion on it, but that’s because they’re linking themselves and they’re putting themselves into that AI market space. So that promise of where this is going over the long haul is where they’re going to be playing. So I believe it’s respective of where that’s going. Okay. With that, Krista, you’ve been doing some research on this concept of ransomware scanning analytics. I know there’s been a lot of announcements over the last couple of months, we’ll shift into another area, which is that area.

Krista Macomber: Yes, yeah. So a little bit of a deviation from the AI and kind of GPU/TPU discussion.

Camberely Bates: There’s another topic besides AI?

Krista Macomber: I know.

Steven Dickens: Something else apart from AI?

Krista Macomber: Shocker.

Steven Dickens: I know.

Camberely Bates: Shocker.

Steven Dickens: There’s probably going to be AI in there somewhere just because there’s got to be. Yeah, always.

Krista Macomber: Give me a minute.

Camberely Bates: Go for it, Krista.

Krista Macomber: Give me a minute. Yeah, yeah. No, yeah. So I was here in the office this week, and I decided to take a little time and read up your report, as you’re mentioning Camberely, on scanning and detection for ransomware, more specifically from the perspective of how we are seeing these capabilities become integrated into data protection software and data protection tools. I thought this was a good topic, because I think there’s quite a bit of maybe some gray areas in the market and maybe a little bit of confusion, because you have your tools, your more traditional anti-malware tools, like Malwarebytes, for example. You have tools on the endpoint detection side of things, and really just this ecosystem is pretty complex with just the number of tools and the fact that you really do need to cover the entire spectrum of your technology stack from the application and kind of the endpoint device all the way through the protection environment with being able to detect ransomware.

So I do, as I mentioned, I have this report. It’ll be coming out I’m hoping over the next couple of weeks, depending on our production schedule here. But really in the report, I kind of broke down the two key areas that we’re seeing be utilized for data protection vendors for identifying ransomware attacks. And the first area is really your more traditional scanning-based on ransomware file signatures and other what we call indicators of compromise. And that’s kind of based on known ransomware variants that, again, we can scan for. And what’s interesting here is this is kind of from the standpoint of really primarily identifying if ransomware or really any malware has infiltrated that backup or data protection environment, and if it’s becoming active and doing things, like encrypting data, as opposed to maybe identifying more at the point of breach. So that’s one thing to keep in mind here in terms of where this is going to play in.

Camberely Bates: In all of these situations, this is areas that the administrators can turn on. They’re not actively having to do anything other than look, then respond to the alerts. And hopefully, I know there’s been some discussion about how accurate this is because this could become pinging alerts all over the place. And so that ability to accurately identify something that’s a real problem, as opposed to something that you end up tuning out after a period of time, becomes really critical into the design of these systems.

Krista Macomber: Absolutely. Yeah. The alert fatigue is real, and I would say it plays in particular in the second category that we see, which is more what we call anomaly detection and really looking at behavioral patterns within, in this case, the data protection environment for unusual behavior that looks suspicious and could indicate that an attack has occurred. And that’s where, to your point, Camberely, we really want to make sure that these tools have a high degree of confidence and that maybe there is kind of some dashboarding for the administrator to be able to say, “Okay, only if it’s above a certain threshold do I want to actually receive the alert.” So in a lot of these tools, we’re seeing kind of a confidence score be built in as well, and really to date, it’s based primarily on machine learning and kind of the baseline that these tools are able to develop.

Camberely Bates: There it is, AI, machine learning.

Krista Macomber: Yeah, it’s going to get –

Steven Dickens: Got there eventually.

Camberely Bates: I think she went five minutes without saying it. You did well. Congratulations.

Krista Macomber: I was building the suspense, building the suspense.

Steven Dickens: You just tease us until the big mic drop for AI.

Krista Macomber: Yes. And I touched on that in the piece, so kind of the discussion of we’re seeing AI begin to be integrated as well, so kind of taking that step beyond machine learning. What’s very relevant there is the attackers are using AI, so they’re using AI to create more sophisticated attacks, smarter attacks, but also, I heard the word terminology, kind of these polymorphic attacks that can actually change over time, which is pretty scary. So that’s a little bit kind of more future-looking, but definitely something that we’re seeing the beginnings of development and something that in the piece I do mention as I beg everybody to keep an eye on.

Steven Dickens: We talk about the tools embracing AI, and we talk about the tools getting better, but that’s just the response to the hackers getting better, and they’ve got access to these tools. So you are always fighting against these guys.

Krista Macomber: It’s a whack-a-mole.

Steven Dickens: Yeah, it’s exactly that. It’s a game of whack-a-mole against the threats are increasing and changing, so you as an administrator and a sort of DevSecOps team has got to continue to improve. So it’s a fascinating space.

Camberely Bates: Well, and as I think we talked about a couple shows ago, the gall of the hacker suing the company.

Krista Macomber: Yes.

Camberely Bates: Filing with the company about SEC violations, because they didn’t expose the fact that the hacker, the ones that was reporting it, had stolen the data.

Krista Macomber: They’ll stop at nothing.

Camberely Bates: You got to go, “Okay, so are they going to show up in court?”

Krista Macomber: Right, in a mass or something? Yeah, yeah.

Camberely Bates: I guess that would be one way of calling their bluff is like, “Okay, so let’s go to court on this one,” and you can’t do this virtually or whatever. But it’s kind of like-

Steven Dickens: It’s a business, it is a business.

Camberely Bates: I read it a couple of times to make sure I’m reading it correctly that the hacker dudes are suing the whatever for not reporting the fact that they hacked and stole. Okay.

Krista Macomber: Yes. And I actually have a piece that we can link to. In all seriousness, I do have a piece that I wrote up about that. And yeah, again, in all seriousness, so if you can’t stay a step ahead of them because that’s becoming increasingly difficult, it becomes a matter of also how do you detect as quickly as possible, again, that an attack has occurred and respond as quickly as possible too, and that’s where AI is beginning to play a role as well. So doing things like being able to intelligently update the backup schedule, to be able to take a back up based on maybe some of these indicators of compromise or some of this anomalous activity or even based on business usage and kind of how usage patterns are changing and be able to maybe preemptively trigger even the beginnings of the recovery process to occur too. So it’s pretty wild, and I think, for as much as we’ve been talking about it, I think we’re kind of still at the beginning stages here. So it’s going to be interesting to see over the next year or two how this continues to develop. And the piece was fun to pull together.

Camberely Bates: And you’re headed out this week, or next week actually, you’re headed out next week to the HPE Security Conference. Is that what I heard?

Krista Macomber: HP, actually.

Camberely Bates: HP, yes. HP as in the PC people as opposed to HPE people.

Krista Macomber: Yes. Yeah. I think for the context of this podcast and really kind of our coverage for me personally, it’s going to be interesting to get a little bit more of I think that end user or that perspective as opposed to sitting just within the data center. But the next time we get together, I’m sure I’ll still have some interesting tidbits, because like you were mentioning, it all has to tie together at the end of the day. So it’s going to be great to get that other end of the spectrum, and I’m sure there’s going to be takeaways for IT ops and the infrastructure side of things too.

Camberely Bates: And last thing, I want to wrap it up with this one. There was an announcement that came out about Broadcom spinning off Carbon Black and their End-User Computing piece of it, which is kind of interesting that that’s going on. Carbon Black, as I’ve talked about before with folks, the guys that do the Carbon Black, RSA, et cetera, they’re really standalone businesses. They don’t integrate very well with the enterprise companies because of how the decisions are made for those offerings and their unique people, their unique group, they have their own world that they live in. So it’s interesting that they’re spinning that off, especially since they have Symantec, because they do have Symantec, and I kind of expected them to kind of bring those two together, but they’re not. Then the End-User Computing piece of it, which is what we know as VDI, the VDI offering, Horizon. They also said, in his quote, he said they’re looking for good solid providers that are going to take care of these assets of theirs that they’re spinning off. So it’s not like we don’t like you, it just doesn’t fit into the core of what they’re going to focus on, which is the VCF, VMware Cloud Foundation technology, and seeing that going on.

Steven Dickens: So in the spirit of more M&A activity going on, I’ve got one more before we wrap up here, and it’s somewhat analogous to what you’ve just said about the Broadcom and VMware piece. OpenText bought Micro Focus, that transaction closed in January, and the AMC business of OpenText, the mainframe modernization, COBOL sort of terminal emulation piece is being spun off, and Rocket is buying that.

Camberely Bates: Oh, that’s a good move, very good move.

Steven Dickens: That’s a very good move. So I had a chance to speak to Phil Buckellew on Monday about the strategic rationale for that. This one makes sense for both.

Camberely Bates: You did a podcast on that. I didn’t watch it, but okay, so that’s what that is all about. Now, I don’t have to watch it.

Steven Dickens: I’m mortified now, mortified that you didn’t watch it, but no, I think it makes sense for both.

Camberely Bates: You get all your news here, if you watch the news from here.

Steven Dickens: I get all the news free. I don’t have to watch. But no, I think it makes sense from an OpenText point of view. They took on a lot of debt to buy Micro Focus. This pays a lot of it down, gets them over half of it paid down in one transaction, and it allows OpenText to be more focused. And it’s great news for Rocket, because it gives their customers optionality. It almost doubles the size of their business from a scale perspective, so it’s great for them from a transaction point of view. I think this makes sense for both sides. It’s going to be interesting to see how it pans out.

Camberely Bates: So maybe we’ll see that with some of the other spin-offs that happen here, so we can-

Steven Dickens: Well, I think it’s similar to the Carbon Black kind of VMware/Broadcom piece. Get super focused, know want your business is. If the assets aren’t core, move them onto somebody else who can do more with them. It makes perfect sense.

Krista Macomber: Right.

Camberely Bates: All right, folks. Thank you very much for tuning in. Enjoyed it. And don’t forget to follow us wherever the button is on the bottom of the screen someplace. And guys, of course, Steve and Krista, it’s great to have you on and chat with you as we do. This is always fun.

Steven Dickens: Always a highlight.

Camberely Bates: Take care, everyone.

Krista Macomber: Alrighty. Have a good day, guys.

Other Insights from The Futurum Group:

Google TPU v5p and AI Hypercomputer: A New Era in AI Processing

Rocket Software Talks About Strategic Reasons Behind AMC Deal Announcement – Futurum Tech Webcast

VMware Acquisition Close: Q&A with Hock Tan, President and CEO, Broadcom – Six Five Insider