Presentation: Programming for Hostile Environments

At Packet, we deploy bare metal infrastructure. We deploy it globally across 16 locations.

When you're dealing with programs that have to operate on top of physical infrastructure, there are unfortunate realities that are not found in something like a virtual machine or a JVM runtime, and as a result we’ve needed to adjust how we develop our programs to account for these practical realities.

This talk will describe of the best practices and how  real-world situations that we've had to deal with internally at Packet.

I would say both.

We deploy bare metal on demand, so if you're a user of Packet you would expect a server to show up in between four and six minutes, and you would expect it to be successful. Almost as importantly, you expect to be able to do that 24 hours, a day seven days a week. Actually delivering on that promise is challenging because we're dealing with bare metal.

Bare metal systems are unreliable and may not show up when we expect them to show up. So I'll go through practical things that we needed to do to combat that reality as well as what that means for best practices around Golang.

I'll also discuss some of the choices that we made along the way as we transitioned from a Ruby on Rails based application into Go-lang microservices architecture, and how those services communicate with each other globally.

This is probably more of an anti-pattern. Imagine a pretty innocuous block of code where you're doing power and boot control on a system.

At Packet, we have front and center a block of code that says if the header that's being passed in indicates to us that there is a certain device manufacturer we're interacting with, perform certain logic based off of it. We've chosen this approach rather than exposing an interface and providing an object-oriented method of delivering the implementation because for us that hid from our engineers the reality that the code is going to be operating in an environment where you need to think about these things.

So we chose not to hide the code in an implementation of an interface. Of course, we do have interfaces and we do implement them for actual business logic in other places. But in this case, the choose helps keep where the code is operating forefront in the developer's mind.