Get ready for an eye-opening journey into the future of work! 💼
On this episode of AI & Us, host David Nicholson is joined by Marc Hammons, Senior Distinguished Engineer, Office of the CTO, at Dell Technologies and Professor Song Han, Department of Electrical Engineering and Computer Science, at MIT for a conversation on the seismic shifts transforming the modern workplace, with a special focus on the truly revolutionary impact of AI-powered PCs. Discover how these intelligent machines are set to redefine efficiency, creativity, and collaboration for organizations worldwide.
Highlights include:
🔹Defining the Modern Workplace: Explore the core characteristics that define today’s evolving work environments, including the essential elements driving this transformation. Hear real-world examples of organizations successfully embodying these principles to enhance efficiency and spark innovation.
🔹Unleashing the Power of AI-Powered PCs: Explore how AI-powered PCs are fundamentally different from traditional consumer computers. Learn how their superior functionality and adaptability facilitate smarter decision-making, boost creativity, and foster seamless collaboration across teams.
🔹Practical Insights & Use Cases for Integrating AI PCs: Gain valuable insights into successfully integrating AI PCs into existing workplace environments. Discover specific, compelling use cases that vividly showcase their potential to redefine operational excellence and significantly boost employee engagement.
🔹The Future Workplace & Navigating AI Challenges: Get a glimpse into the trajectory of the Modern Workplace over the next decade. Understand the anticipated challenges in adopting advanced AI technologies and receive strategic guidance for CIOs and CTOs embarking on this critical transformational journey.
Learn more at Dell Technologies and MIT Media Lab.
Watch the full video at Six Five Media, and be sure to subscribe to our YouTube channel, so you never miss an episode.
Or listen to the audio here:
Disclaimer: Six Five Media is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Transcript:
David Nicholson: Welcome to AI and Us, a series where we explore our future together in the age of artificial intelligence This conversation will explore how AI is shaping the modern workplace, how the AI PC is a key enabler, and the actionable steps decision makers can make to create an environment that embraces the future of work. I’m Dave Nicholson with The Futurum Group and I’m joined by two visionaries, one from Dell Technologies and one from MIT, both who stand at the leading edge of AI. First, Dr. Son Han is an associate professor with MIT’s Department of Electrical Engineering and Computer Science. He’s credited as a pioneer of efficient AI computing with his deep compression technique which allows powerful AI programs to run more efficiently on low power mobile devices. He was named to the 35 Innovators Under 35 list by MIT’s Technology Review. And we’re also joined by Marc Hammons. Marc is a distinguished engineer and a member of Dell’s End User Computing Experience Technology team within the office of the CTO at Dell. His focus at Dell is to bring together solutions and capabilities such as AI, IoT or Internet of Things software infrastructure, edge analytics and augmented reality to enable Dell’s customers to have a unique and differentiated experience. Welcome to both of you.
Marc Hammons: Thanks Dave.
Dr. Song Han: Thank you.
David Nicholson: Good to see both of you. I want to start. Dr. Han, we use the phrase modern workplace. How would you define the modern workplace vis a vis AI moving forward?
Dr. Song Han: Yeah, that’s a good question. I think the modern workplace should have three aspects. Physical interaction, digitalization and also artificial intelligence. So we want to enable this kind of workspace that enables in person collaboration and team interaction like with smart meeting rooms, with AI powered cameras and also bridge this physical and virtual interactions so people can work from anywhere. And digitalization, we want to enable those paperless workflows. Everything is digitalized. You can have mobile and remote access to work systems from anywhere. And once it’s digitalized, you can apply artificial intelligence to automate many tasks like email summarization, data entry, content creation, decision making, etc. An AI Assistant will help with generating the insights, writing, planning, making business decisions and a lot of stuff. So physical, digital and AI.
David Nicholson: It’s interesting you mentioned digitalization. I think those of us who are close to this subject just assume that everything is already digitalized. Didn’t we create data lakes in 1962 or something like that? Dr. Han, where, where are we along this road towards the modern workplace in your mind?
Dr. Song Han: I think we are in the middle between all three aspects. Right from physical interaction that’s pretty much well set up. You have remote meetings and nowadays people work from home very easily from anywhere in the world. So physical and virtual, that’s almost, we’re almost there. But digitalization is hardly. Not everything is digitalized as you can imagine. There’s a huge amount of opportunity first of all to get rid of those physical receipts and OCR, scan everything into a computer. The technology is there. Technology is having very good accuracy using these visual language models these days. The adoption, I think we are getting there. And once we are getting there for digitalization, we can enable AI since everything can be automated unless tedious work is needed for day to day processing.
David Nicholson: Yeah. So Marc from your perspective, once these things are digitalized and once we are using our tools, the subject of AI PCs comes up. It’s at the forefront of a lot of these discussions, especially sort of out there at the edge with end users leveraging models. What’s your perspective on where we are in this journey towards the modern workplace? And how does the AI PC fit in? We’re going to double click on that.
Marc Hammons: I think it’s a bit early on the AI PC front. It is kind of the new kid on the block. We’re not newcomers to AI. It’s been with us for a while. Most people’s experience with AI today is cloud based. But we’re starting to see the emergence of technology that can come down onto the PC and actually work with you kind of as a companion as you go throughout your day. You may know CPUs and GPUs, they’ve been there for a while. AI is kind of common on the GPU. Everyone knows that. That’s where typically AI workload will run today. But now there’s this new kid on the block, the NPU, the neural processing unit. And that’s a special piece of silicon that’s made its emergence and arrival on AI PCs that’s enabling these AI workloads to be resident on your laptop and your desktop. So what was formerly reserved and only available in the cloud, now it’s available, starting to make its way down onto the laptop, onto your desktop so that it can work with you.
David Nicholson: So there’s actual hardware involved?
Marc Hammons: That’s right. Combination of hardware and software. So we’re seeing a confluence of both. We’re seeing some of the silicon becoming much more performant, much more power efficient, which really matters in a mobility world. And at the same time some of these algorithms. And it’s kind of incredible to me these days models make news. You know, we’re talking like trillion parameter models, 2 trillion parameter models with some of the latest developments. But it’s also going in the opposite direction. We’re starting to see these small language models start to emerge and make their appearance down on a PC and those things can run very efficiently on the device for things like image generation, text generation, any kind of collateral that might be assistive in helping you create some kind of work.
David Nicholson: So I think you could look at this from two different angles. The question of performance at the edge and what an AI PC does for you. But Dr. Han, are there other reasons why someone might want to be doing this computation in their own data center versus in the cloud? What are your thoughts about that?
Dr. Song Han: People want a local deployment either on the edge device or their PC or their on prem server because people want to protect their data, data sovereignty. Sometimes we don’t want to share the sensitive corporate data which might have user information to the cloud. So having them local would ensure such data sovereignty and privacy and also good latency and fast. With the modern AI chips, we can process lots of the small language models as fast on edge devices. So no matter where you are, like field engineers, they might be traveling to different parts of the workspace. They can also have access to them locally. They are available 24-7.
David Nicholson: Yeah, Marc, I’ve had a little bit of experience with some of the latest gen AI PCs and I can tell you I didn’t know what I was missing until I experienced what I was missing. It’s hard for people to understand until they experience it. There’s the hardware, of course, but what’s Dell’s view on how organizations should get started? So let’s say I just bought 10,000 new Dell AI PCs and I’m handing them out now. What?
Marc Hammons: Yeah, great question. And it’s important, just, just getting started is really important because a lot of people haven’t really thought through things like what the impact of this new technology is going to be and how it’s going to assist them throughout their day. We see a lot of things on apps that we use every day, apps like we’re communicating on right now that are allowing us this remote collaboration and connectivity. And we’ve experienced in the cloud how things like transcription, translation, note taking, meeting minute collection, action registers, all those things can be done through the cloud. But now the capability exists to do those things locally. And the great thing about that is there could be a personal twist to what’s produced. There might be some context that’s really rich, that’s fusing some of the other information that’s there on your PC, maybe some of the documents you’ve authored recently, PowerPoints you may have created for that presentation, and it can begin to weave in and fuse some of those things together as it works there locally on your device. So I think bringing the technology down and just experimenting with it, figuring out how it will fit in, because it’s not going to be the same for everybody.
David Nicholson: Dr. Han, I understand why folks like us get excited and interested in where the work is happening and whether it’s a GPU or a CPU or an NPU. And Marc especially talking about AI PCs and where they play, it’s really important. Ultimately the end user, I would argue, doesn’t care where any of this stuff is happening. They just care that it’s available and they figure out how to use it. What does the future look like with this in terms of our interaction with these tools? Whether I’m having my request fulfilled locally on an AI enabled PC or from someplace in the cloud, does that look like, as we move forward, do these AI entities take over? Is it a collaboration? What are your thoughts on that?
Dr. Song Han: I think the implication is that in the future there will be increased productivity and efficiency in day to day work. For example, lots of AI PCs will automate those repetitive tasks so that we can focus on those high value work such as creating, problem solving, innovative, focus more on the innovative part of the work and do more in less time. And secondly there will be more and more real time and on device intelligence with those dedicated AI chips. And thirdly, I think there will be a lot of hybrid and hybrid work and collaboration due to the capability of virtual meetings, auto focus, note taking, speaker tracking, lots of the live captions, and real time translation to make people work anywhere in the world. So of course there will also be new workflows and new jobs created in the future. So for universities and educators like us, we should be aware of this trend and make a lot of AI roles like prompt engineers, AI trainers, AI model optimizers, those a lot of new jobs will be created in the future.
David Nicholson: Marc, Dr. Han makes an interesting point. I think we, I think it’s fair to assume that hardware and software in this space is going to get better, cheaper, faster, stronger as we move forward. Everywhere, you know, at the core, in data centers, in the cloud, at the edge it’s all going to, it’s all going to get better over time. How does someone go about making the decision about where their next AI dollar is going to be spent today?
Marc Hammons: Yeah, great question. I think there’s several things you’ve got to consider certainly like what’s the complement of your organization, what are you trying to achieve? You really got to step back and understand what your goals are. Outlining a plan, a master level roadmap that’s going to drive your understanding of AI, maybe establishment of an AI center of excellence. Those types of things are paramount to start with. And then once you understand that roadmap, you can make better decisions about, hey, is it cloud, is it client, is it both? Because just like Dr. Han said, it’s a hybrid story. Long term, the distribution of intelligence is going to be really interesting to watch the evolution of over the next five to 10 years. But I think it’s a little bit of both. It depends on the needs of your organization. There’s a lot of things to consider, but again, it’s all going to be use case driven and really you really should pin it to a long term strategy and plan.
David Nicholson: Dr. Han, what are some of the sensible guardrails and reasonable concerns that you might have about the future of AI and how we best rein it in?
Dr. Song Han: I think we should define boundaries and also human oversight, for example, human in the loop design to ensure AI systems require human validation for critical decision making, especially in domains like healthcare. For example, if a task requires 10 steps in total, the AI agent can do nine steps. And we require humans to validate the last step, make sure it’s making the good decision for us.
Marc Hammons: Yeah, I think there’s a few things that are really key. We talked a lot about data sovereignty, privacy, security. Those things are paramount. I know they’re on the minds of just about every customer that we meet with in this domain. And so I think ensuring that you have a good plan around your data, where it is, who has access to it, keeping tabs on that will be really important. Another great thing to think about is identity. If there’s an agent, what is the identity of that agent? What do they have access to? What rights do they have? Who do they represent? How do we realize that in a modern world amidst what could be an ocean of agents that are working together at any given point? Point. It’s a really, really interesting thing to think about and something that I think the industry is working hard.
David Nicholson: If I get an AI PC from you, how big is the model? How much space is that taking up? And then how much memory does it need?
Marc Hammons: Yeah, sure, sure. A rough estimate is typically about a gig per billion parameters. So these things are mini gigs. Right? If you start with like a 4 billion parameter model that’s typically going to be around 4 gigs which is almost as big as the OS size for most PCs that are maybe running Windows. And so they’re not insignificant but thanks to Dr. Han and some of the work that they’ve been doing in research and he really, I can’t under express like some of the phenomenal things that he and his team have been producing in terms of optimizations around even KV cache. The context window, the context link. Super important because as that thing grows, your resource consumption varies quadratically with that. So it gets really, really exponentially difficult to really make use of resources as you get a larger context window. As he’s been leaning in and actually optimizing these things, we see a tremendous reduction in the number of, in the amount of space that’s required in terms of memory footprint on disk as well. So we’ve seen some really phenomenal improvements lately. But you’re right, space matters, memory matters, memory bandwidth matters. The ability to move data is paramount. And that’s why these quantization techniques, the different data formats, they’re all extremely important. And it’s a multifaceted problem when it comes to the optimization performance of these models.
David Nicholson: Dr. Han, how would you characterize the sort of state of the art when it comes to this balance of precision versus performance? So the efficient use of resources. If you have a model that’s in the simplest of terms only consuming 1 gigabyte or 2 gigabytes of storage and only has a limited amount of memory resource, what are we giving up today instead of precision? And then how long do you predict it will be before this won’t matter where the question of well wait a minute, can you run this model that previously could only be run in a data center on a laptop? We’ve all heard the stories, people can hold up their mobile device and say a thousand times the compute power required to go to the moon, you know. And so as we look towards say 2030, which isn’t as far away as we might think it is, what do you see in the future for PCs and at what point do we get to this question of compression and efficiency not being as important, at least in the sense that you’re making a sacrifice?
Dr. Song Han: So not to mention 2030, just nowadays we can easily compress a model from 8 bit precision to 4 bit precision, especially using an NVF P4 Nvidia’s floating point 4 bit format without losing, pretty much without losing accuracy. We can also use big model and small model work together to do self speculation, use a small model to speculatively guess the answer and use a bigger model to verify it from time to time. If the guess is correct, you can gain a lot of speed up and efficiency improvement. So those are some of the examples. We can drastically accelerate the AI models these days given a fixed AI model. And there’s even more opportunity to design those efficient models to begin with given the insights I just shared. So I see lots of opportunities already. So like you mentioned, by 2030 there will be a lot of boom in this industry. A lot of cool applications will come to everyday life.
David Nicholson: If you had to predict one significant advancement over the next five years. Go ahead and be crazy. What do you think? Gelatinous bio storage? Reverse engineered alien technology? What can you share with us?
Marc Hammons: I would love to see all of those. So if you’re aware of them, let’s talk more. I think to be futuristic and keep it grounded as well, leaning into some of the things Dr. Han’s talked about, the advancements in some of this technology is really going to be phenomenal and come at a very rapid pace. We’re already, you’ve heard of agents. So we’re already seeing kind of this agentic rise where you’re talking about autonomous, almost digital twin-like intelligence that’s able to write software, compose articles, create works of art, those types of things, and autonomously and collaboratively among a group of agents. I think we’ll continue to see that trend come to assist us in our daily lives, not just work, but at home as well. And I think that leaning into maybe that hybrid nature of things, disaggregated compute, we’ll call it ambient computing, where there’s compute around me that I can tap into and maybe models and model pipelines, workloads, agents are distributed across that computing infrastructure is something that I would see as a big trend over the next five years that will become a reality sooner than we think.
David Nicholson: Well, we hope that you found this conversation to be thought provoking. I certainly have. It’s always great to explore the intersection between AI and US, the human beings in the mix. I’d like to thank Dr. Han for the work that he does in this area. And you also, Marc, over at Dell. I hope the rest of you can tune in to future episodes of AI and Us, because this is truly about AI and US. I’m Dave Nicholson for The Futurum Group.
Author Information
David Nicholson is Chief Research Officer at The Futurum Group, a host and contributor for Six Five Media, and an Instructor and Success Coach at Wharton’s CTO and Digital Transformation academies, out of the University of Pennsylvania’s Wharton School of Business’s Arresty Institute for Executive Education.
David interprets the world of Information Technology from the perspective of a Chief Technology Officer mindset, answering the question, “How is the latest technology best leveraged in service of an organization’s mission?” This is the subject of much of his advisory work with clients, as well as his academic focus.
Prior to joining The Futurum Group, David held technical leadership positions at EMC, Oracle, and Dell. He is also the founder of DNA Consulting, providing actionable insights to a wide variety of clients seeking to better understand the intersection of technology and business.
