On this episode of the Six Five Webcast – Infrastructure Matters, hosts Camberley Bates and Keith Townsend dive into the latest announcements and developments in data infrastructure and AI markets, ahead of GTC. Their discussion delves into new advancements and community engagement shaping the technology landscape.
Their discussion covers:
- The importance of CIO chat and community engagement platforms like Blue Sky for encouraging high-level tech discussions.
- Updates on IBM Storage Scale and its enhanced role in supporting AI infrastructure, emphasizing its integration with vector databases.
- Developments in Pure Storage’s FlashBlade//EXA, highlighting the benefits of disaggregating metadata and data nodes for increased performance.
- Dell’s innovative “AI Factory” concept explains the landscape of AI markets ranging from hyperscalers to specialized AI factories and enterprise AI applications.
Watch the video below and be sure to subscribe to our YouTube channel so you never miss an episode.
Or listen to the audio here:
Disclaimer: Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
Keith Townsend: All right. You’re watching and listening to Episode 75 of Infrastructure Matters because it does. I’m the only beard on the stage today. Camberley, what’s going on with that?
Camberley Bates: Dion, I’m not sure. I mean, I text him, called him, Slacked him. We’re concerned about it. I’m hoping he’s not got a problem going.
Keith Townsend: We are. I know he did his famous CIO chats yesterday on the x.com, so I’m sure he’s fine. It’s Jesse’s scheduling snafu and he’ll be back next week to help us unload on all of the news coming out of GTC next week. I’m excited.
Camberley Bates: So what is the CIO chat? Let’s let everybody know what he’s doing.
Keith Townsend: Yeah, so he does this, he’s done and he’s done it for years, even going back into the X days where he basically hosts a virtual chat, a virtual discussion, this acentric, asymmetric conversation on x.com. And now even he does it on Blue Sky as well, where he basically poses these CIO level questions to audiences, #CIOChats. If you want to follow the hashtag, follow the historic conversation, search for that hashtag.
Camberley Bates: And so then he hosts a day that they’re chatting back and forth. So it’s a lot.
Keith Townsend: Yep. It’s a day, and I want to say it’s usually Thursdays. Thursdays every week, there is a back and forth chat with the larger CIO community. It’s a really great way to look into that community and see what the hot topics are and what their discussion.
Camberley Bates: Well, I’m actually going to have to follow them. I didn’t even know. And the other thing I want to ask about, because we had a internal meeting talking about some Six Five Media kind of things, which is part of what we’re doing here. I’m going to be doing some new podcasts as well, aren’t you? Didn’t I hear about…
Keith Townsend: Yeah, at least one new podcast is going to be Six Five on AI. Hopefully with me and Dave Nicholson. We’re going to have some of the world’s most, I guess I’m telling on myself because I’m saying some of the most smartest people. I don’t think that’s the correct way to say that, but we’ll have some of the world’s smartest people in AI, analysts, vendors, et cetera, interviewing them on the topics of Generative AI, Agentic AI and all things AI. I’m looking forward to the new podcast.
Camberley Bates: And I would love to connect you with a group that I was invited into called the American Society for AI, and it’s a beautifully cross disciplined organization. Everything from… We’ve got… There’s a Microsoft lawyer on there. There is a Lieutenant General, retired Lieutenant General on there. There’s Senators, House of Representative people that are part of it, and little old me, somehow or another I got invited to this thing because it’s by invitation only and it’s kind of like…
Keith Townsend: Yeah, I don’t know if little old you is… That’s a little humble, Camberley. With all due respect, I think if you ever decide to hang up your boots, it’s going to be a sad day in the industry for when and if that ever happens. Yeah.
Camberley Bates: Yeah. And we just submit a… For an RFI that the government had out for ideas, certain questions that they had put out as the organization. So consolidated this organization in terms of knowledge base. So I’m going to bring those people to you because there’s some really smart folks that are on that group.
Keith Townsend: I love actually smart folks, smart questions. I had Brian Lowe, who’s a principal engineer over at AWS. I had him on my CTO Advisor podcast and we talked about the art and science of using AI to do code development, and it was a really great preview to the conversations that we’ll have over on the Six Five channel. So we’re looking forward to that. So let’s dive into the news this week. Again, light news week, because we’re going to GTC and everyone is holding their cards. I think Dell has a pre-briefing at 10:00 at night on Monday the week of GTC. Everyone is really holding their cards closely. We have had some pre-briefings, but we’ll talk about what we can publicly. Your first story is data infrastructure announcements, IBM Scale.
Camberley Bates: Yes. So IBM Storage Scale, not to be confused with PowerScale from Dell or be confused with Spectrum Scale, which is what it used to be called. This is something that drives me crazy. The vendors do you get a new marketing person in and they rename everything. It’s like, don’t do that to me. Okay. So what this is GPFS, sorry guys. This is therapeutic. We’re going to back to that.
Keith Townsend: Therapeutic. PowerScale will always be the VNX to me. It just will. It will. But we digress. We’re showing our storage bias and the greatness in my beard is starting to come apparent of why it’s there. So let’s feel.
Camberley Bates: Just let me talk greatness in my hair. There we go. Okay. So IBM Storage Scale is the classic GPFS. GPFS has been the dominant parallel file system used by all the HPC market, not all of them, but a lot of the HPC people. Over the years, they have simplified the system, so it’s a whole lot easier than it used to be. It used to take a lot of probably a hundred students to put it up, and now according to my buddies that used to run this thing, it’s a whole lot easier to implement and install. So now what they’re going into, because this is their primary system for AI, as you can imagine, this thing scales well beyond most systems. This competes with Lustre in those kinds of areas. What they’re doing is they’re adding in connectivity to vector databases like we’re seeing many of the storage companies doing.
So a couple sessions ago we had a little bit of debate to say, is AI a layer on top of the current transaction area or systems? Is it a separate AI kind of system? The other debate is do you have vector databases integrated into your data storage system because you can smooth out your AI pipeline or is it separate? So what they’ve said, this is bringing out the vector database in there for that they have the ability to change it out. So just because it’s one vector database today, if there’s another one that’s more popular that comes out in the future, they’ll be able to change that over to a new system. So they’re jumping on this bandwagon, if you will, of where we see this piece going.
So now we have, in terms of this space, you’ve got NetApp that’s going to be rolling theirs out. They pre-announced their vector database capabilities. You have VAST, it’s already there. That’s got theirs out there. So we’re seeing, and Dell is doing an integration with another company that’s doing vector database, Starburst. They’re integrating with other companies. They haven’t announced their own vector database. So we’re seeing this. And then we’ve got some probably announcements coming from HPE I am sure because Discover is going to be coming up in a few months. So I’m sure that they’re going to be jumping down this space as well. So this is kind of like the pattern that’s going on.
Keith Townsend: So help me size or understand the market from a VAST data. We were on a call with Ren and what was that last week and he said, “If you want scale, there’s no other options other than VAST.” Obviously the HPC folks have been doing scale for a really long time. Is this a kind of pushback against the VAST story and saying, “Hey, if you want vector databases, you don’t have to go proprietary. You can go with a somewhat open system.”
Camberley Bates: So this is definitely a pushback to VAST. VAST has always claimed absolutes in terms of all their announcements. We are the most. We are the best. We are the whatever. We hear that often with announcements from companies claiming the top-notch space and to a certain extent because they’re not a pure parallel file system, they’re a PNFS implementation that is part of the Linux Kernel and that kind of thing. They can probably claim the best there. When you get to a pure parallel file system like you have with GPFS, Lustre, Panasas is probably another one, WEKA. Maybe there’s some differentiation there in terms of how they scale. One of the things that they’re talking about as well as others are saying that we’re going to exabytes in terms of these systems and we have to support exabytes. We have to support tens of thousands of GPUs. It’s getting going to be really big. I read an article, I look over and grab this piece on it, but was talking about where the financial institution companies are going and how big that data is going to be here in the next three, four years of what they’re expecting it to be. That all said is, can we afford it? I don’t know.
Keith Townsend: So keeping on the vein of storage companies or storage announcements, pure FlashBlade//EXA Pure is still at it with their all-flash systems. What’s the news there?
Camberley Bates: Well, the news with FlashBlade//EXA, just as I said, with VAST being a PNFS implementation, that’s what FlashBlade has done. So they are PNFS, parallel NFS file system implementation. Plus what they’ve done is disaggregated the Metadata nodes from the data nodes so this is going to super speed up their capabilities. They hyped up their experience with the hyperscalers, as I think we talked about last week. They’ve been doing a lot of work with Meta and most recently they announced that Meta is their client that is looking at them to OEM their direct flash modules. But in this system, they’re going with off-the-shelf technology for the data nodes. You have to buy their technology for the metadata nodes because of architecture, I’m sure, which appeals to your HPC and AI markets that are there. And then eventually they’ll roll it out with their own capabilities, their direct flash modules, which are right now coming out at 150 terabytes or I think they’re shipping 150 terabytes. I could be wrong with that. But yeah, so they’re right in the mix of this rollout with everybody else. We’ll see. Here’s some other ones probably this next week, but those are two of the big announcements that are coming through.
Keith Townsend: Earlier on in the fight of architectures, there was some debate on whether or not when compute was too expensive, if mixing or separating the data nodes from the metadata nodes had enough value. I think now it’s unquestionable with the advancements in indexing, et cetera, and the advantages of being able to distribute your metadata across large geographic regions or within a data center with tens of thousands of GPUs being able to identify what the data is, where that data’s at, and how to most quickly get to it within your data infrastructure. Milliseconds matter.
Camberley Bates: Yes, sir.
Keith Townsend: KIOXIA is 122 terabyte drives. We just mentioned 150 terabyte module, a 122 terabyte drives. We’ve heard these announcements from folks like Solidigm in the past. We’re still at the drive size race.
Camberley Bates: So we’re just going to continue on doing this. This is QLC so they’re following Solidigm. Solidigm has got the enterprise class drive that they brought out for the 122 KIOXIA is bringing out. The reason why I bring this up is because KIOXIA… KIOXIA or KIOXIA. Anyway, another one of those names.
Keith Townsend: I think they’re just happy that we’re talking about them.
Camberley Bates: Okay, there we go. Another one of those names. In January, they announced, I’m going to look at my notes here, they announced a AiSAQ software. I’m not sure how they’re pronouncing it. AiSAQ software that they open sourced and basically this is on the drive. And what they’re saying we can do is you can place the index. This software will speed up the access to the index data that you might have on DRAM, but now you can put it on the drive. It’s going to be as fast or close to as fast on the SSDs, which can have some considerable savings. And I want to see the data on this and the performance information. What they’re looking at is saying, “Okay, so that index data that’s on the DRAM there, that’s there. That’s really for the RAG technology where I’m using the RAG technology to update the data, et cetera. So if I can do that and separate that between those two devices to DRAM versus the SSD, it’s a whole lot cheaper.”
Keith Townsend: Yeah, we’ve seen this in the consumer space. We’ve seen really big SSDs, but with limited DRAM. If they have a limited DRAM, then the performance just suffers. It’s horrible as a matter of fact. You get a lot of storage, and this has been the knock on TLC, right? A lot of size, but not a lot of performance. So you get other technologies like TLC, et cetera, but it seems like companies like Solidigm has solved it. But that’s with using just enough DRAM to do caching and improve overall indexing performance. So I’m a little curious about this announcement story. Indexing on QLC absolutely brings down the cost of the drive, but performers-wise, I have to imagine there has to be a hit somewhere.
Camberley Bates: Yeah. And so I’m going to dig into it. I mean, I just kind of looked at saying, “Okay, so this is really curious. They’re catching up with whatever.” So the battlegrounds are going to be there, especially as we go into exabyte space, but I want to see this performance and capability. So maybe they’ll call our signal Six Five guys and say, “Hey.”
Keith Townsend: Yeah. Can you help validate these pretty aggressive claims? I love the competition in the space. This shows how you can take a single spec. We know that we’re at 122, we see the roadmap to 256. We see the roadmap. We know that the ability from a wafer perspective to get to 256 and beyond is there. But where do these companies differentiate and innovate and find different ways to eke out just enough performance, just enough price, et cetera? Where do they compete? And I’m happy that KIOXIA.
Camberley Bates: KIOXIA. KIOXIA. KIOXIA.
Keith Townsend: That they’re innovating. I think other than Micron, I’m not sure why all of these drive companies have such interesting names, Solidigm, KIOXIA, that they just don’t roll off the western tongue, that’s for sure. All right. Moving on to… Is it CISA or CISA red team?
Camberley Bates: CISA.
Keith Townsend: CISA red team.
Camberley Bates: So CISA is the cybersecurity agency in the government and it’s Homeland Security. And there was a big noise about them letting go of a contract that dropped a hundred people that were considered part of the red team, good guys that are doing penetration analysis. There’s a lot of noise that I picked up on multiple trends that can say, this is what the government is doing. They’re firing all these people. We don’t have it. I’ve been involved with CISA now, I don’t know, six years or whatever. There’s a local CISA here. They host. In fact, I’ll be down with them in Denver in a couple of weeks where they host a multi-dimensional group that comes in and shares information, everything from technology educators, police, local police, national FBI, et cetera, and share information.
But the CISA guys are that really the cybersecurity. When you get hit, especially like a bank, they’re going to be involved probably in some way. So they’re connected to all this Homeland Security kind of work, et cetera. So when they said they fired the red team, it’s kind of like, “Whoa, whoa, whoa, whoa. What?” And the answer is no. And I want to kind of get out there that that’s not what happened. What they did is they canceled one of the contracts that is a contractor that is doing some work for the CISA red team. The red team still stays in existence. They’re a big organization in terms of how they are working through all the penetration.
If you want to know more about it, just Google CISA government and you can see all kinds of stuff. So they have lots of education that goes both for the police and private sector in terms of cybersecurity, et cetera. They do focus a lot on consumer initiatives. A lot of the information there is a lot of consumer because that’s when you think about the local police, that’s who they’re really working with as opposed to the FBIs and those kind of guys or the NSA are working on the international problems that come across. So really interesting group, but I just want to bring that out if you don’t know they exist.
Keith Townsend: All right, so we haven’t talked a ton about AI other than the vector databases, but the market is complex and you have the show notes to talk about this complex relationship across all of the different AI markets. I want to highlight or zero in on the term that I’ve heard over and over again, the AI factory. Talk to us about the overall AI market.
Camberley Bates: So I know you’ve got opinions on this. Last summer, Dell coined a term called AI Factory and Jeff Clarke got up and talked about how every company is going to be building this AI factory, and that’s why they’re hammering down on what this looks like. And that term has now been picked up and it’s now ubiquitous. We’re going to hear a lot about it at GTC this next week. So then we’re talking about, okay, so what is this AI factory or what can it be? And in some of the presentations that we’ve had from these companies, the data companies as well as the server companies, they’re talking about this market. And what we’re seeing now is there’s kind of like these three major markets. There is the super hyperscalers, like an OpenAI kind of guy. Google’s doing their thing, et cetera, that are building the original LLMs.
And then if you go down from there, the next layer down is probably this company that is doing, or firm organization that’s doing like an AI factory. They’re specializing or they’re investing heavily in developing either this sub-foundation model, let’s say a DNA model. Let’s say it’s an energy model. Let’s say it’s an architecture model or a weather model. They’re building these huge environments that are going to be the guys that are absorbing in the exabytes, the big huge exabytes. And there may be specialized clouds. They may be a research center like our local CU here that I have in Boulder doing some sort of specialization. And that’s where the big numbers is. And so for next week at GTC, this is the market they’re probably focused on. This very, very big market that they’re saying, this is where the billions and billions or of dollars is going to be exploited because they’re building these huge environments out.
And then the next layer down is the enterprise AI, and they’re the ones that are going to be doing the RAGs. They’re going to be ones that are doing the streaming transactional data, respond, give you the customer service capabilities you want, the retail capabilities, the development capabilities, et cetera that we talk about is the implementation of what we’re seeing and the results of this. So those are the three markets. And when we start looking at these markets, you got to look and saying, “Okay, so what technology… From Infrastructure Matters, what technology do you need for an infrastructure, an enterprise organization, the state and local kind of companies? What technology do you need if you are this big AI factory that you’re building? And then what technology do the hyperscalers build? So that’s kind of where I’m going to start watching this market going to and saying, “Okay, so who are you talking to and how are you talking to and what are the demands of these people?”
Keith Townsend: Yeah. So this reminds me, Monday next week, next week of this recording. I’m headed to GTC obviously, but there’s kind of a pre-GTC Unconference hosted by AMD. It looks like it’s hosted by AMD. The idea is beyond CUDA, this idea that you must challenge the notion that A&T and NVIDIA is the solution for all of AI. It’s very difficult at this point to say any one solution is the answer to all of AI for any of these markets you just mentioned. So for example, at the most basic layer of compute, we’ve talked about some of the friends or customers of Dell, the Nature Fresh Farms of the world that’s doing some amazing stuff around growing tomato plants inside of greenhouses in which they collect data off each and every plant, analyze that data and try and prove their yields by one or 2% each individual crop.
All of that has been done on CPUs, just good old-fashioned VxRail nodes with no GPUs in there and not really using much NVIDIA technology. Whereas when you’re talking about that AI factory story, especially with Dell, and that’s very much an NVIDIA heavy conversation. And one piece of news that I did miss this week that it’s relevant to your highlight is Meta announced that they are testing and they’re phasing in their own GPU, I’m sorry, XPUs, which are accelerators but not full GPUs, and they are developing their own custom layer of software to be able to ingest all of this data and do all of the training. So it is an amazing, I think, technical landscape across all three of the markets that you’ve highlighted.
Camberley Bates: And I think as I talked about earlier this last year, we’re going to see within every industry there’s going to be this sub or multiple sub foundation models. It’s kind of like going back when I talked to the CIO at Swift, what they were building, the Swift is the transaction people. This is well over a year ago they had built the LLM that’s looking at wire fraud and it’s taking and ingesting all of that information, and if they can just drop the wire fraud a couple percentages, you’re talking trillions of dollars here. So it’s just a fascinating market of what is getting built out. And then in my thinking process about, okay, so if Swift can build that for the international transactions and everything goes through them, then if you share that, can that data then be shared with the other banks to implement on top of their web services capability that trains their data, et cetera but they don’t have to build the big huge model. So that was one of the things I was looking at is saying, “Okay, so this is where we go with this.”
Keith Townsend: Yeah. What happens when they can create a signature almost that allow and be able to push those down to other bank’s AI models, so they can now adjust the sensitivity based on their risk profile, their client base, et cetera. But what happens when you get the strength of a Swift to be able to do what we talked about at the onset of this, we’re talking about GPFS and vectorizing the data and absorbing all of these exabytes of data. What happens when you have a provider in the middle of that that can take that heavy lift and then go down to that third RAG layer that as you described it, and able to implement off the shelf technologies that help them to actually implement some of these techniques? Fascinating time and it’s absolutely why infrastructure matters. But before they get to that, my one piece of news is Kubernetes solving the problem, is FinOps solving the problem that we are hoping to solve in their enterprises. Over at CIO Dive, they’re challenging this notion that overprovisioning is done. If you don’t know what a container is, you’re probably listening to the wrong podcast. So there’s that. But there’s the idea that if you have this high level orchestration layer, you can better not only control the scaling of your applications, but the resulting costs. But developers are developers and do developers care about cost?
It is a invisible thing to them, this whole movement around DevOps and this idea that developers are the single source of both solving the business challenge of writing the code and controlling the infrastructure was always a bit of a pipe dream. And when you adopt technologies like Kubernetes, this is not a fix all. It is not going to solve every problem in your enterprise. You still need domain experts to really help tune in, negotiate between physical groups of people, analog. The analog is still the challenge. You still need to help educate developers, control costs, put systems into place that help to meet the end goal of scaling applications at a reasonable price. It doesn’t matter if these applications are on premises or in the public cloud, the issue is still the same.
Camberley Bates: So what you were saying is that they’re over provisioning server compute, storage probably.
Keith Townsend: It’s storage.
Camberley Bates: They always have, and they’re overprovisioning in the cloud despite them having a lot of tools that would enable them to keep it skinny.
Keith Townsend: Yeah. They’re just looking to solve the core business logic challenge. How do I process this many orders in this amount of time? And the cheat and enterprise IT and IT in general has always been throw more compute at a problem. It is much easier to throw more compute at a problem than it is to optimize code. Again, when we’re talking about AI generated code, your prompt is probably not including optimized for the amount of storage RAM and compute that I use to solve this problem. Your lens is typically help me write code that solves this problem. It doesn’t have to be efficient and help me write a YAML file that scales to the enough compute to accommodate this code.
Camberley Bates: So are you thinking that we’re going to use AI to help do that a little bit better?
Keith Townsend: I think we are going to use AI. We’re starting to see AI creep into conversations around network design, network configuration, FinOps. Obviously, the folks over on the security side and the SOCs have been way ahead of almost everybody using LLMs to help triage incidents and really get ahead of this thing. I think AI is the way, right? I can just take all of my logs. In theory, I can take my log files from my Kubernetes clusters and my costing data from AWS, Azure, and GCP, upload that to a multi expert model and say, “Hey, optimize for cost.” I might be oversimplifying it, but that’s the overall idea.
Camberley Bates: Yeah. Okay. Cool. So before we close out, I have a really important question.
Keith Townsend: Okay.
Camberley Bates: What’s the logo on your shirt?
Keith Townsend: Oh.
Camberley Bates: Got a show I’m doing.
Keith Townsend: It is the DePaul University Blue Demons. It is where a lot of my personal money and my son’s money and then my money again for my son to go to school. Last night of this recording in the Big East Tournament, they gave Credent a run for their money. They ended up losing in double overtime by four points. It’s heartbreaking. It’s heartbreaking.
Camberley Bates: Just a round ball.
Keith Townsend: That’s the game where the ball is round, you throw it into a basket. That is the game. It’s the only game that’s at DePaul. We don’t have any other sports ball games that anybody else pays attention to. It is…
Camberley Bates: No hockey?
Keith Townsend: Not that I know of. I’m sure there’s a tennis team. I’m sure there’s a golf team, but we have no football team. So basketball team is the one sport in Chicago that we cannot get good at again, which is a crime because we’re in Chicago. But I digress. You took me there.
Camberley Bates: I took you there. I’ve been thinking it for the last half an hour. What’s that logo on your shirt that
Keith Townsend: What is that? That is the great University of DePaul. You know what? The computer science program, it’s much better than the basketball program. I’ll just put it. I’ll just leave it. I’ll just leave it at that. All right. That’s it for this episode of Infrastructure Matters. Make sure to stay tuned. Next week, we’ll have the output from GTC. It’ll be more news than we can digest in a half an hour, but it’ll be worthwhile in preparation. Again, I suggest you put this onto a thumb drive, throw it onto the street and see how many people pick it up and put it into their computer and listen to it. Talk to you next episode of Infrastructure Matters.
Camberley Bates: Okay.
Author Information
Camberley brings over 25 years of executive experience leading sales and marketing teams at Fortune 500 firms. Before joining The Futurum Group, she led the Evaluator Group, an information technology analyst firm as Managing Director.
Her career has spanned all elements of sales and marketing including a 360-degree view of addressing challenges and delivering solutions was achieved from crossing the boundary of sales and channel engagement with large enterprise vendors and her own 100-person IT services firm.
Camberley has provided Global 250 startups with go-to-market strategies, creating a new market category “MAID” as Vice President of Marketing at COPAN and led a worldwide marketing team including channels as a VP at VERITAS. At GE Access, a $2B distribution company, she served as VP of a new division and succeeded in growing the company from $14 to $500 million and built a successful 100-person IT services firm. Camberley began her career at IBM in sales and management.
She holds a Bachelor of Science in International Business from California State University – Long Beach and executive certificates from Wellesley and Wharton School of Business.
Keith Townsend is a technology management consultant with more than 20 years of related experience in designing, implementing, and managing data center technologies. His areas of expertise include virtualization, networking, and storage solutions for Fortune 500 organizations. He holds a BA in computing and an MS in information technology from DePaul University. He is the President of the CTO Advisor, part of The Futurum Group.
