QCon New York June 15-19, 2020 | | Modeling the Real World With Elixir/OTP

Location: Empire Complex, 7th fl.

Duration: 2:55pm - 3:45pm

Day of week:

Level: Intermediate

Persona: Developer

This presentation is now available to view on InfoQ.com

Watch video

What You’ll Learn

Learn how actors are an easy way to model real world concurrency problems.
Hear PagerDuty’s use case for leveraging Elixir to build a highly scalable architecture.
Understand an approach to building concurrent systems that do not use shared memory and instead use a message passing paradigm.

Abstract

Building software that interacts with the real world is not as trivial as it sounds. When you build software that interacts with the real world, you have model your program to represent the real world. The traditional approach to modeling this is to model real-world events sequentially, one after the other. This falls in line with the von Neumann architecture of a computer where a CPU fetches one instruction at a time. In reality, this does not fit the real world. In the real world, an unimaginable number of events occur simultaneously. In fact, most programming languages used today, model the world in a sequential way, where one event happens at a time and failures and errors are an exception, not the norm. Elixir, a language built on top of the Erlang BEAM VM provides not only built in concurrency primitives but effective error handling that makes modeling the real world a natural exercise.

In this talk, I would provide the audience with a walkthrough of concurrency support and error handling in Elixir. This will be followed by a quick run-through of modeling a simple system and running a large number of concurrent BEAM processes along with graceful error handling behaviour.

Question:

What is the focus of your work today?

Answer:

I work on event-based systems that leverage Elixir/OTP and quite a bit of Apache Kafka. My team is building a platform for enriching and processing high volumes of data in real time.

We’re working on what we call the event intelligence platform. Essentially, customers sent us all sorts of monitoring data including things like email integrations, legacy monitoring tools, and new monitoring tools. That can be several hundred thousand or even millions of events per second. Our systems are built and designed with this sort of scale in mind.

Question:

What’s the motivation for this talk?

Answer:

The motivation is twofold. First, the actor model despite having been around for a while does not have as much adoption as shared-memory based traditional concurrency models. Therefore one of the motivating factors behind this talk to help the audience model a “real world” problem with Elixir’s actors (BEAM processes and OTP abstractions).

The second is to briefly touch on how Elixr and OTP provide clean abstractions around event-based modeling, briefly touching on event-sourcing. I hope to tie this back into the fact real-world problems are generally event based.

Question:

The talk is called Modeling the Real World With Elixir/OTP. Is it language talk or will you be talking things like event sourcing architectures with Elixir?

Answer:

This is mostly a language talk. It’s really about how choosing a language helped us gain a better architecture. While I will be briefly touching on concepts like event sourcing, I’m really assuming the audience is familiar with it already. It’s really about how you can use OTP and Elixir (essentially anything built on top of Beam) to build concurrent systems.

Question:

How do you describe the persona and level of the target audience?

Answer:

I would describe the target persona to be engineers familiar with the concepts of event sourcing and actor-model but not aware how Elixir or OTP provides such abstractions around it. Ideally, people from other senior backgrounds such as JVM based languages will be a good target.

Question:

What do you want this persona to walk away from your talk with?

Answer:

The target persona should walk away with the feeling that actors are an easy way to model real-world concurrency problems and additionally feeling that Elixir makes a good fit for such problems.

Question:

What do you feel is the most important trend in software right now?

Answer:

With the rise in log and event-based systems such as Apache Kafka, I feel that there is an industry-wide shift in modeling state as event sequences. I feel that this is a very important shift from the traditional way of persisting state.

Speaker: Aish Dahal

Engineer @pagerduty

Aish works as an Engineer at PagerDuty in San Francisco. He currently works in building PagerDuty’s event intelligence platform often dealing with fallacies of distributed computing. His recent focus has been on Elixir/OTP and building event driven microservices using Kafka and Elixir. In the past he has worked as an early stage employee at HackerRank as well as a programmer at Goldman Sachs.