Disclaimer: The Six Five webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.

Transcript:

Patrick Moorhead: The Six Five is on the road at AWS re:Invent 2023 in Las Vegas. We are in the Future Frequency Container. Yes, a shipping container doing this. This is very apropos for Amazon.

Daniel Newman: Yeah, it’s hi, I’m in a container.

Patrick Moorhead: Exactly.

Daniel Newman: It feels like there’s a joke there, the tech show.

Patrick Moorhead: Dan, the conversations we’ve had this week have been incredible. They’re so diverse, and this really has become, and even people not affiliated with AWS, this is the B2B show. This is where it’s happening. We have talked compute, we’ve talked server-less, we’ve talked generative AI. What we haven’t talked yet is about app integration. And, quite frankly, when it comes to the reality of what powers businesses, it’s the app and you’ve got kind of … You have legacy monolithic, you have modern. There’s always new and modern ways to pull applications together and also develop new applications.

Daniel Newman: Yeah, look, every company is in this whole technical debt to modernization sprint. And generative AI has just accelerated it. We’ve seen the power of how it’s been able to enable companies to do everything faster.

Patrick Moorhead: Right.

Daniel Newman: And of course the whole gen AI is going to take over all the coding is totally not accurate. But having said that, we do see that every digital transformation shift is going to be in shorter and shorter and shorter timelines. And being able to modernize your apps to keep with, A, what the customer needs, the way a customer interacts, and then, of course, internal processes is going to be really important, Pat. So, this is a big thing. The event-driven app architectures are going to be a big topic for the next few years.

Patrick Moorhead: Well, hey, let’s introduce our guest, stop talking to ourselves.

Daniel Newman: Oh, ouch.

Patrick Moorhead: Emily and Nick, welcome to the show. First time in The Six Five. Thanks for coming on.

Emily Shea: Thanks for having us.

Nick Smit: Absolutely.

Daniel Newman: So, Nick, let’s start off. We’ll talk about kind of what is it? You heard me right at the end there, I said, “What is an event-driven architecture?” Talk a little bit about that and let’s just make sure the audience is on a level playing field.

Nick Smit: Yeah, well, let’s start with what is an event? And really an event is a statement of fact that something has changed in a system or business. So, we say a statement of fact and we say they’re immutable, right? It means that they’ve happened, they cannot change. So, I think a great example of this is just when you place an order in the system, right? That order has been placed. It has happened. You cannot change that. But events really represent these changes in our business. And so an event-driven application or an event-driven architecture is an application that reacts to changes in events. So, a lot of folks that we speak to are familiar with kind of integrating their applications with synchronous request response API calls, right? My order is placed in the system, I’m going to generate an invoice. So, I call my invoicing system to say, “Create an invoice,” right? But what happens over time is you add more and more applications to your overall architecture and you end up having to do all of these point-to-point API integrations between your different systems, right? That slows your teams down. It means your applications are really tightly coupled.

And so in event-driven, what we do instead is we say, “Each application just emits an event.”
It says, “Hey, that order was created.” Anything else that is interested in that event can simply subscribe to it, right? So, say I want to generate that invoice. My application just subscribes to my event stream, it sees the order is created event, and it takes an action based on that. And so what we have is this ability to then decouple all applications, to spread them out, to say, “We don’t have to do these point-to-point integrations between them.” And we see a lot of benefits from that, that decoupling allows our engineering teams to act more independently, to not be kind of having to tap each other on the shoulder, “Hey, can you send me an API request for this change?” It allows us to develop features faster, so when I add some functionality that does forecasting, I simply listen to that event and writing code that reacts to it. And it also allows our architectures to be more resilient, right? Because we have this decoupled integration between our applications, our order service could get a really big spike in events, but that doesn’t mean that some other service has to scale up immediately with the order service, right? It can sit there processing them asynchronously. So, those are some of the benefits we see around event-driven and why folks are moving to it.

Emily Shea: Yeah, and I would add on that one of the ways that I typically describe event-driven architectures to customers is that it’s where you got both your systems, and maybe your microservices or your applications, communicating by events and also your development teams. And so with the systems, now they’re able to scale independently. With the development teams, they’re also able to build independently and move super fast. And I think the other really cool thing that I find about event-driven architecture is that it really resonates with people that have been building software for a long time because this is not a new concept. People have been building with enterprise service buses and all these kind of asynchronous messaging technologies for a long time. But the really cool thing about it that we have now is we’ve got all of these cloud services that emit events by default. And so all of that is the events are already flowing through the cloud, and you just need to latch on to them.

Patrick Moorhead: So, we live in a world of heterogeneous applications and some that are 50 years old, 20, 30, 10, 5 and built this year. In an event-driven architecture world in the cloud, how do you manage the messages and the APIs? How does that work? Or is it even possible? Are we just talking about new applications here?

Emily Shea: Yeah, so we have a ton of customers obviously that are coming from a place where they’re needing to do a large migration, maybe for on prem or some legacy applications. And a lot of them are actually using event-driven architectures as a migration strategy for that. Because what they’ll do is they’ll build maybe their new integration solution with AWS native services and they’ll start to pull events and messages from that existing system. So, they’ve still got some continuity knowing that they’re probably not going to do a big bang migration over to the cloud. But they’re able to get started building those new applications, and so it can kind of bridge the old and the new as they move more towards the new.

Patrick Moorhead: So, the alternative, right, we see this a ton, you would have to do just a complete rewrite, right? You have multiple things and there are so many different ways you can do it. You can ETL data and have people hitting on the same thing, but what I see a lot in the SaaS world is a complete rewrite, right? Of the entire stack. That is one of the alternatives, right?

Emily Shea: Mm-hmm.

Patrick Moorhead: Send that to you, Nick.

Nick Smit: Definitely. But I think what we really like about event-driven architectures is it allows you to do this piece by piece. And we find that actually often that allows you to move a lot quicker. You’re not doing this massive architecture, architecture with a huge amount of risk in it. Instead you can say, “Let’s start emitting events from our legacy application.” There’s some small changes we make to do that. And then we start to build the applications in the cloud that react to those events. So, we start to replace some of our functionality that was running on premise or running on our legacy system with modern equivalence running in the cloud.

Daniel Newman: So, what I hear here, there’s some there there and I hear here. And we’ve done this ourselves as we’ve grown our organization and we’re tying ERPs and CRM systems together, you got your supply chains, is this kind of dedicated iPaaS. And that’s been kind of part of the solution, at least for a long time if you have many, many different applications running you build an iPaaS so that all the integrations happen. In fact, I’m going to have to actually go back to my team and talk to them about this, because I’m like, “Hm, are we doing this right?” It’s right now. We’re doing what we think is right right now, but what I’m hearing here is that this is sort of maybe a modernized approach to this dedicated iPaaS, which is still a very typical way that companies are connecting these different systems, different applications, different data layers. Talk a little bit about the maybe kind of similarities, differences, and appreciating that iPaaS is still common. Is it modernizing on that? Could this replace that? Kind of where is this heading?

Emily Shea: I would say it is modernizing on that. I think I do talk to a lot of customers that have maybe existing iPaaS or existing enterprise service bus or that type of model that they’re working with. I think that it’s one of the reasons that they’re moving to AWS and building that capability with AWS services is just the flexibility of AWS services. Because we know that our applications, and especially the integration needs, are going to evolve massively over time. And so I think that the flexibility of being able to use all of these purpose-built AWS services and combine those into an integration solution is definitely something that we’re helping some folks move off of, maybe a more traditional solution onto one on AWS.

Nick Smit: I think the other thing I’ll mention here is we see customers using more and more SaaS applications, purpose-built applications, whether it’s for their CRM, for their sales. And so these applications now are called part of their production systems. And so customers want to treat the connections between them like they would any other production system. They want their developers to use familiar tooling, a lot of their deployment processes, testing. So they don’t want totally separate proprietary systems managed by a business user. They want something that is hardened that they can rely on that is scalable.

Daniel Newman: You make such a great point though because you think about all these sort of apps within apps. Even when you look at a Salesforce, which you guys had a pretty significant set of announcements with this week. A lot of what they’ve built has been bolted on. It’s inorganic. These are not actually fully integrated systems or they are working on it in building these event-driven architectures. You got your RevOps cloud now and then you’ve got your project cloud now and then you’ve got your HR hiring cloud now. And all these systems have little … I always say it’s veins and it’s arteries, capillaries, you’ve got the blood moving between it pumping it in and out of the heart. And that data has to flow and you’ve got to make sure that the right systems are talking and things are getting updated. And it gets mad really quickly if you don’t have these integrations.

Nick Smit: Totally. And I think the key other part of this and why we see event-driven architectures being such an important part is that the realtime nature of this is really important too, right? So, you have these changes that happen in Salesforce in your different clouds. You need them to propagate immediately to a call center agent who’s taking in a call or a salesperson, right? And so the old approach of batch jobs that ran once a day or once an hour, that just doesn’t cut it, right? It has to happen instantly. And so that’s why we see event-driven being such a great fit with iPaaS. So, EventBridge, the product that I work on, has this native integration with Salesforce. We’ve seen some fantastic applications being built around that realtime integration that we have there.

Daniel Newman: Could be customers.

Patrick Moorhead: Yeah, Emily, I guess we didn’t have a green room per say, but in our chit chat running up we talked about integration first. And it’s something that AWS is recommending. It might sound really straightforward and maybe it is straightforward, but I want to make it … Why is that important in modernizing applications? I want to make sure that’s crystal clear here.

Emily Shea: Yes. So, I talk to a lot of customers that they’re working on maybe a large migration project or a large modernization project. And they recognize that integration is both central to the way that their applications interact, but also to the way that their teams interact. Because integration, it’s funny, you very quickly move from a technology conversation to an organization and a culture conversation just because the way that those teams are interacting and building together is going to be very fundamental to where you go.

Patrick Moorhead: Yeah, so apps teams, just doing what classic IT has done forever which, unless they’re motivated, operating in silos. I have this app, you have this app and this app. Even though there could be a tremendous amount of value from having some integration together. Is it like that?

Emily Shea: Yes, and I think that the cool thing that we’ve seen some customers be able to do with event-driven architectures is they’re able to give teams the amount of space that they need. So, if they’re building a new feature, they don’t need to go coordinate with the team next door to figure out how they’re going to break theirs. So, they’re able to build independently, but then they still have this core structure around essential event bus or a stream or some type of integration solution that allows them to have this place where they can go and know that all the events that they’re flowing throughout the company that they can pull from and build new applications on top of.

Patrick Moorhead: So, keep the speed, sense of ownership, adopting event-driven architecture.

Emily Shea: Absolutely.

Patrick Moorhead: Hey, how about that?

Emily Shea: You got it.

Daniel Newman: Okay, we’re done. All right, I’m going to bring out the white board, I want you to sketch this for me. No, one of the things that we love on the show is we always do kind of we call it ground source truth. And when we’re talking to CEOs it might be talking about their earnings and investor days. When we talk to technical or people in front of customers, we like to talk about customers. And I won’t put you put on the spot to actually give me customer examples, although if you will give me a customer example that would be great. But kind of walk us through the real life example within the enterprise, kind of how this event-driven application would flow.

Nick Smit: Yeah, so I think a wonderful example here is PostNL, they’re one of the largest logistics companies in the Netherlands, and they use event-driven architectures in their sorting facility. So, you can imagine thousands of packages coming in every minute, right? And they have these machines that decide does this package need to go left, right. Does it need to go into some overflow queue? And so there’s hundreds of decisions that impact that, lots of different systems, right? That will look at weather, that will look at what their capacity is. And so an event-driven architecture here allows them to have an event emit when a logistics package comes in, and then to have hundreds of different systems be able to consume those events. And they will have ideas for how they can improve that sorting, very easily they can add that as another consumer on that sorting event to be able to influence the direction that this package is going. So, I think for me that’s one of the really great examples of logistics. This is something that we all interact with and know about. And PostNL have been really successful at using that in that particular context.

Emily Shea: The other thing that’s cool about the PostNL use case is also the seasonality of the business. So, you’ve got a logistics company, obviously December is going to be when they’re hit the hardest, and the cool thing that they’ve talked about-

Patrick Moorhead: By my family primarily buying stuff.

Nick Smit: Right.

Emily Shea: And the cool they talked about is the fact that all of their AWS native services that have this surplus operating model, they scale up and down without them needing to really touch it. And that is able to flow throughout the year as the highs and lows of logistics.

Patrick Moorhead: That just sounds like magic. But it’s like, no, it’s reality. It shows the benefit because they guarantee that the combination of legacy applications and new ones are required to pull that together and have to be integrated. So, we’re at re:Invent and I think 250 things were announced here. What did all of you announce here with event-driven architectures? I heard you talk about one product and a service.

Nick Smit: Yeah, so Amazon EventBridge, the one that I work on, we’ve recently announced an integration with Adobe and Stripe. So, these are third party partners. Stripe does payment processing. Adobe has the Creative Cloud and a number of other kind of products. And so whenever you’re using these applications, things are changing. In Stripe a subscription might fail or a charge might succeed. You want to take some action in your application running on AWS for this. In the past, you typically had to do a whole lot of work to stand up HTTP endpoints so that they can send you web hooks or potentially write code that pulls. And so we’ve launched an integration with these third party providers where you simply provide them with your AWS account ID and your region and they will be able to push events directly to your event bus. So, none of that kind of integration code that we often have to write, and a very simple, scalable integration with these third party providers. Do you want to talk about the gen AI side of things, the step functions?

Emily Shea: Yeah.

Daniel Newman: I was worried we weren’t going to talk about that.

Patrick Moorhead: It took us 16 minutes to get here, so.

Emily Shea: We got there.

Patrick Moorhead: Yeah.

Emily Shea: So, yes, the other one, really cool one, that’s in some of the services that we work with is step functions, is Workflow Orchestrator, where you can pull in all kinds of different AWS services and build workflows around those. Really cool integration that we launched was native integration with Bedrock. So, our large language model provider and now you can pull that into your workflows, interact with that without writing any code. And we’ve always seen people do a lot of kind of AI ML use cases within step functions. So they maybe they have when a file gets uploaded they send that file off to maybe Textract or one of these other services that runs some AI ML and pulls data out of that and maybe sends that on to another location. And now they can pull Bedrock into those workflows without writing any code around it.

Nick Smit: I think just an interesting point on that, when you mention step functions, we see frequently that when customers build event-driven architectures, they need to think a lot about orchestration. When an event happens, it’s typically not just one thing that you’re doing, right? You need to say, “Well, let’s take insurance claims, for example. You bash your car, that’s the event, you need to submit a claim.” But now you need to do things like upload your driver’s license. Once you’ve done that you need to take a photo of the car. And so this is what we call orchestration. It’s things where one thing has to happen only after another thing has happened. And so we see this great combination of event-driven architectures with orchestration through something like step functions.

Daniel Newman: It’s really interesting too. We were talking to one of your leaders on the Bedrock team, but this even more so appropriate, is you really are seeing this rapid onset pivot from RPA. And what I mean by that is look at how event-driven, it’s kind of how much it’s going to pivot a lot of what that hard coding embedded at the core and give companies super flexibility to do these multi-step processes, and eventually … Look, I can invent here in the container because I can come up with ideas and things that I can’t built but you could. This should be almost natural language. Let me just walk the process through. Let me just talk about it. Here’s what we need to have happen.

Nick Smit: Totally.

Daniel Newman: Maybe someone could come up with a solution called Q for instance, and you could talk to Q and you could say, “Here is what the multi-step function looks like,” and then it could actually build the hooks.

Nick Smit: Yeah.

Daniel Newman: And it could start creating events. And this is where I think, like I said, you’ll always need programmers to do the most sophisticated things, and it’ll just keep evolving and the mundane stuff gets … Those people up-level and up-level. But when I’m listening to you guys, I’m like, “Wow, things that used to take years of hard code and screen scraping all this stuff that was done in our kind of critical workloads are now getting redesigned to be able to done with generative capabilities and natural ” Cool.

Nick Smit: And I think the point that you make there is that this focuses us on the business process, the business outcome that we’re trying to achieve rather than the really specific coding challenge that we have to implement that.

Patrick Moorhead: So, I have a question from a developer point of view. So, you look back in history of it seems simple at the time, right? You would build one application, it would sit on one server, it was on prem, you had no APIs, and then we moved this IDE environment, where it’s like, “Hey, we’re not going to need programmers anymore.” We never have had enough, by the way, for three years. And then we keep moving to these new abstractional layers. And I’m curious how do you respond to things like how responsiveness the app is and, hey, if I’ve got 14 different APIs that I’m pulling in, does observability and measurement come along with your offering as well? To know what happened when and maybe to go in and autotune and autofix this?

Nick Smit: Right. Well, I’ll say I think one of the big challenges we’ve seen with folks adopting this architectural pattern is that a lot of them are moving asynchronous world, request-response, where it’s very easy to observe what happens when you make a call and you get something back. And the event-driven world is asynchronous, right? So, things are happening without you necessarily being able to observe what is happening in this other system. And so that definitely takes a mind shift change for folks that are thinking about these systems.

Patrick Moorhead: It’s the big question that I get on this. Yeah.

Nick Smit: So, I think that for sure to being with we haven’t done as good enough jobs as we could have in terms of thinking about observability. But that’s really where the space is moving, is to be able to say, “Okay, we are in this asynchronous world.” And I think Werner, last year in his keynote, did this wonderful skit on asynchronous and how the world around us is asynchronous. And it would be crazy if we were to treat the world synchronously. But in programming that takes a mental shift for sure. And so I think a lot of our observability tools are catching up to that mind shift change. And so we’re starting to see some really fantastic developments there. In EventBridge specifically we’ve added a lot of logging, a lot of metrics that help folks understand, despite this thing not happening in my system, I still am able to observe it and see what’s happening there. So, I think that’s an area where it was more difficult in the past but is getting better and better.

Patrick Moorhead: By the way, apologize for my technical inaccuracy. I said API versus event, so just noted for the audience.

Emily Shea: It’s all integration.

Patrick Moorhead: Okay.

Nick Smit: You’ve got to have API.

Daniel Newman: So, I want to just say thank you both, Emily, Nick, it’s been a lot of fun having this conversation. I’ve invented at least one new product, what have you done? Since we’ve been here. But no, it’s actually very interesting to learn. I did hear a little hint that we may be hearing more about observability from AWS soon. And I can’t imagine how you’re going to keep building this without that. So, it’s very exciting. We’d love to have you all come back hopefully soon to talk to us more. And meanwhile, I hope you have a really good rest of your AWS re:Invent.

Nick Smit: Thanks so much for having us.

Emily Shea: Thank you. It’s been fun.

Daniel Newman: All right everybody, hit that subscribe button. Join Patrick and I here for The Six Five On The Road at AWS re:Invent in Las Vegas. We are in the Future Frequency container, where we are containerless, serverless, but we are human and we are excited to have you in our community. Got to say to goodbye now. See you all later.