QCon New York June 15-19, 2020 | | BLESS: Better Security and Ops for SSH Access

Track: Building Security Infrastructure

Location: Plymouth - Royale, 6th fl.

Day of week:

Slides: Download SlidesNEW!

Level: Intermediate

Persona: Architect, CTO/CIO/Leadership, Developer, DevOps Engineer, General Software, Security Professional

What You’ll Learn

Learn an effective way to manage SSH keys for access to servers and to protect the related infrastructure.
Improve understanding and thinking about the key management problem and suggest some approaches that have been used at Netflix.
Discuss risks associated and their impact with real world attacks around key management.

Abstract

How can using SSH certificates improve security and simplify operations for instance access at Netflix-scale? How can you smoothly transition existing infrastructure to use SSH Certificates? Netflix created and uses BLESS, an SSH Certificate Authority that runs as an AWS Lambda function and is used to sign SSH public keys. In this talk, you will start by learning about BLESS in general: what it is, how it works, and how you can start using it. Next, we will explore the Netflix BLESS production architecture and how other companies have used BLESS in different ways.

From there, we will dig deeper together to discuss Netflix’s deployment and operational details, leveraging BLESS for security insight, and future plans for authorization improvements. The entire talk will be interactive with demos along the way.

Question:

QCon: What is your focus at Netflix today?

Answer:

Bryan: I lead some of our product security teams. We basically do two big buckets of things.

One is we build security services to make it easy for other engineerings at Netflix to get security right. These are things like key management, secret protection, and TLS enablement.

Second thing we do is work really closely with other engineers to make sure that the code that they're producing has the right quality bar from a security standpoint. These are things like making sure that Internet facing services aren't vulnerable to attack.

Question:

QCon: What is BLESS from your title?

Answer:

Bryan: BLESS is an open source project we put together at Netflix a couple of years ago. It's a way to handle SSH access to your instances in the cloud. A lot of people will set up a bastion that you log in to before accessing your production instances, but, these approaches (while great), raise some interesting questions around key management. From a risk perspective, this is particularly important when you have a lot of instances in the cloud and a lot of developers accessing those instances.

What BLESS allows us to do is actually shift that around so that we use an AWS Lambda function to work as an SSH certificate authority. This Lambda function has access to a private key that can sign a certificate that gives you access to SSH into a cloud instance for five minutes. So it works like this, you prove you're the right person and have access to the resource. Then BLESS provides your well scoped credential. If that credential is ever lost you know you have a very limited scope that works for a limited time period.

Question:

QCon: So what's the focus of the talk?

Answer:

Bryan: Here we are a year later after releasing BLESS as an open source project. We're going to talk about the ecosystem that we've created around it, and how it fits into Netflix. We will be talking about some of the operational lessons learned. The hope is to make it easy for an engineer to think through these ideas and see how they could fit into their world, and they could deploy BLESS or a similar type solution if they wanted to.

Speaker: Bryan Payne

Leads Product & Application Security @Netflix

Dr. Bryan D. Payne has dedicated his career to the complex field of computer security. He currently leads the Product and Application Security team at Netflix where they provide the security expertise and create systems that protect Netflix’s large cloud footprint. Over the years he has worked on both offensive and defensive security projects for government, academia, and industry. As a result, Dr. Payne brings a unique perspective to modern security issues.