One of the recurring tasks as Head of Product is coming up with quarterly roadmaps. Now that the fabulous product management team at commercetools and I have done this together six times, I feel like we’re getting somewhere good. Good enough to be worth sharing anyway. Why roadmap anyways (and how to do it well) I’ll admit […]
One of the recurring tasks as Head of Product is coming up with quarterly roadmaps. Now that the fabulous product management team at commercetools and I have done this together six times, I feel like we’re getting somewhere good. Good enough to be worth sharing anyway.
I’ll admit it — when I started as a product manager, I was not a fan of building roadmaps. Hated it. Because they looked like this:
More of a project plan, listing all the outputs in a specific order (with mostly improvised box-sizes and dates), than a document that highlighted what we were planning as the next steps to deliver on the product vision and bring value to customers and company alike.
So, why have a roadmap? At commercetools, we use them for three main reasons:
1. Make sure we don’t stray from our vision: We roadmap quarterly, which gives us a built-in quarterly checkpoint to remind the product teams where we want to go, and what the next steps for getting there are.
2. Help customers make wait vs. build decisions: One of our product’s strengths is that it can be extended and customized, allowing our customers to master all sorts of commerce-related challenges with it. It’s great. But it is even greater for our customers if their challenges are solved out of the box. By being transparent about our plans and progress, customers can focus and plan their investments better.
3. Start a dialog between product and customers: We do have a lot of brilliant, innovative, and enthusiastic customers. When we show them what we plan to work on, they do share their thoughts, ideas, and pains with us — and we listen and learn.
In short, we love our roadmaps. And make them without fidgety boxes and pretend accuracy.
Prioritization methodologies or frameworks (Weighted-Shorted-Job-First, RICE, MoSCoW Analysis, and a dozen more) all aim to bring work items into a specific order: What do we start working on? And then? And then? And then?
They do this by looking at value and risk.
Value = good. Risk = bad.
Some go to great lengths to define how to calculate both, others are more like: Hey, how about starting with the most valuable one? Which isn’t exactly helpful advice.
Summarised, high values in the “good” section will put items to the top of your list, high values in the “bad” section move them to the bottom.
Find a way to quantify value and risk that is not susceptible to biases that do harm (some biases are sort of okay, and can even be good), and is not impossible to use for your team.
Some influencing factors to consider:
Sooo… looking at this, we can rule out all frameworks that depend on effort or complexity. It cannot be too heavy on statistical data but has to take customer input as well as growth plans into account. As I work with people that are too smart to just believe what they are told, the methodology needs to be research-driven, data-informed, and as opinion-free as possible.
Another challenge we faced is that our system was close to the limits it could handle — both organisationally, and technically. That meant we needed to invest in costly and urgent growth initiatives. “Costly” meaning a couple of developer years, “urgent” meaning that we now wish we would have done it earlier. Sort of like when you wish you would have gone to the dentist before the toothache started.
Very relevant side note: When reading that we postponed getting our metaphorical teeth checked, you might have thought: “Well, that wasn’t smart.” But, and this is what this article is about, it was smart. When previous roadmaps have been built, a prioritization method that favored technical innovation, fast progress, and winning new customers absolutely made sense. It all depends on the situation. Also, premature optimization smells. We have since made the investment to set our system up for a higher magnitude of load and scale.
Just looking at the value aspects of prioritization methodologies, we need something that works equally well for innovation, improving user experiences, organization together with architectural setup, and scaling up our platform to work with x times the load from last year.
Weighted scores are pretty perfect for that, as they do not prescribe what matters to you, and allow you to rank the influence each of the dimensions has.
Weighted scores can be a bit hard to wrap one’s head around. Here’s a food-based example to illustrate it: Deciding what’s for lunch.
Dimensions that matter:
Each day, those dimensions can have different importance (weight) for you. Monday, your mean calendar only gives you a 30-minute lunch break, so you weigh Speed at 70%, No-Food-coma at 25%, and Yumminess at a has-to-be-edible-5%. Which results in a “whatever is in the fridge” sandwich. On Saturdays you have plenty of time and want to indulge yourself and go for 20% Health, 80% Yummy — and I leave the rest to your imagination.
When deciding on what to eat, you assign a value to each of those dimensions to every single food item you have on your shortlist, sort by rank, and pick the top one — or for a roadmap, the top n ones. This is what we do in our quarterly plannings. This is how it looks like:
We review these dimensions once per year and adjust the weights with each quarterly planning. The main data point we use to inform the weight for “enjoyable experiences” and “scaling” is the number and impact of incidents, measured by SLOs and error budgets. Another important source is customer feedback and questions our support team gets.
To rank an individual item in the “innovation” and “enjoyable experiences” dimensions, the most important data point we use is the user impact score. It is basically the first two letters of RICE — Reach (“how many customers are having this problem?”) and Impact (“how important is it to them?”). We diligently collect every piece of customer feedback throughout the year and use that to determine the user impact score.
This is a challenge I struggled with for years. Having been a software engineer for about half my career, I know the difficulties of estimating efforts or complexity for pretty much any programming work that couldn’t also be done by a program. That is especially true for topics around scaling and optimizations, which by nature require a lot of experimentation and testing; or those nasty Heisenbugs. Additionally and as mentioned earlier, using efforts or complexity to measure risk is fundamentally biased against tackling tooth-ache preventing work.
But ignoring delivery risk completely doesn’t seem wise either — reliable roadmaps are helpful, internally and for customers.
Curtains up for: “The Managing Complex Change Model“ (Mary Lippitt, 1987), also referred to as the Knoster model for managing change. The model identifies five key elements for successfully implementing organizational changes — and it applies well for all sorts of change, including changes to a product. In a nutshell, if one of those elements is missing, you won’t be able to deliver what you set out to do.
Using this, we can reduce the risk part to a set of yes / no questions, leading to a simple yes/no on whether or not a team can deliver the product change. If the answer is yes — brilliant, put it on the roadmap! If the answer is no — identify what is missing, and put these items on the roadmap. It is as easy as it sounds, and leads to great insights to improve teams or the organization.
TL; Don’t want to read? Watch the video of my presentation at the OSN Speaker series!