Presentation: More Reliable Delivery with Monte Carlo & Mapping

Track: High Velocity Dev Teams

Location: Majestic Complex, 6th fl.

Day of week:

Slides: Download Slides

Level: Intermediate - Advanced

Persona: CTO/CIO/Leadership, General Software

What You’ll Learn

  • Learn how Monte Carlo Simulations can improve your ability to forecast.
  • Understand the techniques used implement project forecasting with Monte Carlo Simulations.
  • Hear how the combination of Story Mapping provides the a key piece that’s important for implementing Monte Carlo Simulations.

Abstract

Most teams are familiar with estimates using relative sizing, or story points based on the Fibonacci series, or t-shirt sizes. Most teams either agonize over the details of every estimate, hoping to be as accurate as possible, or rush through planning knowing the result will be off no matter the effort.

There's a better way, though; one that doesn't involve marathon meetings or half-hearted attempts; a way that can provide accurate plans automatically. Monte Carlo simulations are based on previous performance and sound statistical techniques. Teams no longer need to estimate individual stories - simply building a backlog is enough to forecast delivery dates. In order to build a strong backlog, we'll need to come up with an accurate number of stories up front. Using user story mapping, we can be confident we have the entire picture.

In this talk, we'll walk through how to use Monte Carlo simulations to remove the guesswork from planning, story mapping to discover the whole story up front, and review an example of how to automate a useful forecast.

Plans are worthless, but planning is everything.

Question: 

QCon: What is the focus of your work today?

Answer: 

Conal: I’m a product manager at Handshake, a B2B commerce platform that facilitates sales between manufacturers/distributors and their customers. I’m trying to make sure that we’re delivering as much value to our customers as we can. Like everywhere, our resources are limited, so mostly I’m focused on what outcomes can we influence the most, and what solutions can we design to help our customers.

Question: 

QCon: What’s the motivation for your talk?

Answer: 

Conal: I’ve gotten a lot out of the New York City lean/agile community, especially from people like Jonathan Hansen. I’m excited to have the opportunity to give back, and help other practitioners improve the way they work.

I’ve also rarely talked to someone that’s happy with their planning process, but everyone seems to try the same things when there is a more useful, less painful approach that relies on sound statistical methodologies and can be fairly automated. This seems like it’s the perfect topic for the QCon audience.

Question: 

QCon: How you you describe the persona of the target audience of this talk?

Answer: 

Conal: I’m aiming this to experienced developers who have tried a lot of different planning strategies, and aren’t quite thrilled with anything. They’re more senior, and should be expected to influence the processes at their company.

Question: 

QCon: What is a Monte Carlo Simulation and how does it apply to estimation?

Answer: 

Conal: A Monte Carlo Simulation is essentially just looking at past data to predict future results.

There's a couple of different statistical techniques that go into it. Essentially, you're going to get a range of all the cycle times (or the takt times for stories you've delivered before). You're going to then extrapolate those to a very large sample size.

We're talking about usually 10,000 samples of cycle times (or takt times). Then you're going to input the size of your project to figure out roughly how long it's going to take you to complete the stories based on a historical random sample.

The way this works is that by looking at the past data and then extracting it, we're doing something called bootstrapping. Where we're taking something that might have very high variability and by extrapolating that into a larger sample size, we're going to key in on what the actual true values are. Then we're going to take that and distribute that into our project to make sure that we have a good understanding about when the work's going to be delivered later on.

It's a little bit different from traditional project management. With Monte Carlo Simulation what comes out of it is a range of probabilistic outcomes. So rather than saying that we're going to be done with this on July 4th. We're now going to say we have a 50 percent chance of finishing this by June 30th, and we have a 75 percent chance by July 4th.

Question: 

QCon: QCon targets advanced architects and sr development leads, what do you feel will be the actionable that type of persona will walk away from your talk with?

Answer: 

Conal: I have two main goals: first, I want people to buy in to the fact that this is the right way to build a backlog and forecast your projects; second, I want to provide enough information and references to enable them to implement Monte Carlo simulations with story mapping effectively. This is a deep topic, but if I can pique interest enough for attendees to try to experiment with this forecasting on their own, I’ll call it a success.

Question: 

QCon: What do you feel is the most important thing/practice/tech/technique for a developer/leader in your space to be focused on today?

Answer: 

Conal: As a product manager, I’m always striving for real continuous improvement in how we deliver value to our customers. We have to be willing to experiment, measure, and learn. With Agile being so well-established, it’s easy to fall into the same patterns that are common in many organizations, even if they’re not productive. To differentiate your product, company, and culture, you have to be willing to try new things and find what works specifically for your team.

Speaker: Conal Scanlon

Product Management, Lean/Agile Software Development @FlatironHealth

Conal is passionate about product development and how things get built in general. Currently, he is a product manager at Flatiron Health, building tools to help people learn from the experience of every cancer patient. He's worked as a developer, analyst, and Scrum Master, and loves talking about all things lean, agile, or product.

Find Conal Scanlon at

Similar Talks