Presentation: Dynamically Re-Configurable Event-Driven Systems

Danny: I'm CTO of The Glue, so I'm heading up our architecture, design, and engineering teams. We're building a next generation engagement platform for banks. My main focus is to make sure that it has all the functionality and features the banks are expecting. As part of it, there are some complex issues—that I will present in my talk—that need to be solved. I also focus on facilitating people to come up with pragmatic solutions for the problems we face using a variety of new technologies.

Danny: The biggest challenge is guaranteeing exactly once processing of the events. This is extremely, extremely important in the financial world—once you have acknowledged an event, we take on the responsibility to make sure that whatever the action is linked with, it will be executed. No matter what happens, we must assure that it gets executed. So it’s absolutely the biggest challenge in this type of system. Another challenge is scalability to cope with load peaks, you cannot just move the data around, or you can move it around it has a cost, so it's this balancing of scalability for processing versus having heavy data—which is heavy to replicate, is also an issue for which a solution will be found. And that's exactly what we've tried to solve in our system.

Danny: I'm going to pose challenges to the audience and try to give a small example for scalability. Typically, in a financial application, you know quite well how the load will evolve, with the exception of the things happening in the stock market. If it's about payments for instance you know that on the first day of the sales or on the weekend before Christmas there will be a high increase in electronic payments. So you can plan your scalability in advance, but you still must be able to do this scaling. So I'll be introducing a problem, with some small example and then say how we in our system we solve that.

Danny: I think solution architects will get a lot from this talk, as well as CTO’s who are interesting in learning about the challenges we faced, and overcome.

Danny: The two messages I think they should take away with it is, firstly, if you have an application that must be stateful—especially with shared state—if you want use an event-driven way, there's quite some impact. The second thing is how to build a system such that people who are not developer rockstar's can still use it successfully.

Danny: If it's about efficient development it's definitely investing in a fully automated DevOps chain, continuous integration. This is not new, but absolutely necessary. And the second thing is, I would absolutely look at everything that has to do with being able to have your systems deployed on a cloud-based system, anything which requires dedicated hardware or which can only run on a local systems is doomed.