How we Roadmap in 2021

One of the recurring tasks as Head of Product is coming up with quarterly roadmaps. Now that the fabulous product management team at commercetools and I have done this together six times, I feel like we’re getting somewhere good. Good enough to be worth sharing anyway. Why roadmap anyways (and how to do it well) I’ll admit […]

One of the recurring tasks as Head of Product is coming up with quarterly roadmaps. Now that the fabulous product management team at commercetools and I have done this together six times, I feel like we’re getting somewhere good. Good enough to be worth sharing anyway.

Why roadmap anyways (and how to do it well)

I’ll admit it — when I started as a product manager, I was not a fan of building roadmaps. Hated it. Because they looked like this:

A project plan displaying boxes in a sequence, each representing a piece of work. The boxes are sorted into a timeline from January to March.

More of a project plan, listing all the outputs in a specific order (with mostly improvised box-sizes and dates), than a document that highlighted what we were planning as the next steps to deliver on the product vision and bring value to customers and company alike.

So, why have a roadmap? At commercetools, we use them for three main reasons:
1. Make sure we don’t stray from our vision: We roadmap quarterly, which gives us a built-in quarterly checkpoint to remind the product teams where we want to go, and what the next steps for getting there are.
2. Help customers make wait vs. build decisions: One of our product’s strengths is that it can be extended and customized, allowing our customers to master all sorts of commerce-related challenges with it. It’s great. But it is even greater for our customers if their challenges are solved out of the box. By being transparent about our plans and progress, customers can focus and plan their investments better.
3. Start a dialog between product and customers: We do have a lot of brilliant, innovative, and enthusiastic customers. When we show them what we plan to work on, they do share their thoughts, ideas, and pains with us — and we listen and learn.

In short, we love our roadmaps. And make them without fidgety boxes and pretend accuracy.

A sixty-second summary of all prioritization methodologies I used or researched

Prioritization methodologies or frameworks (Weighted-Shorted-Job-First, RICE, MoSCoW Analysis, and a dozen more) all aim to bring work items into a specific order: What do we start working on? And then? And then? And then?

They do this by looking at value and risk.

Value = good. Risk = bad.

Some go to great lengths to define how to calculate both, others are more like: Hey, how about starting with the most valuable one? Which isn’t exactly helpful advice.

Summarised, high values in the “good” section will put items to the top of your list, high values in the “bad” section move them to the bottom.

Good: Value, Impact, Importance, Reach, Satisfaction, Increase in satisfaction, Contribution to “drivers” (which are dimensions you define as valuable), Urgency, Cost of Delay. Bad: Item complexity, Implementation efforts, Uncertainty about the scores given for value, Costumer’s Dislikes

The challenge

Find a way to quantify value and risk that is not susceptible to biases that do harm (some biases are sort of okay, and can even be good), and is not impossible to use for your team.

Some influencing factors to consider:

  • The scale of the operation: Are you roadmapping for the whole product org or one team? Within a feature or smaller, or from 10k feet?
  • The stage your company and product are in
  • Product quality and stability within the last quarter
  • How the teams are doing: Forming, norming, storming, or performing?
  • Company culture and people: If you have strong #noestimates advocates, stay away from using efforts or complexity to quantify. If you work with people that understand statistics, be careful with quote-unquote “statistics”.

Our situation

Sooo… looking at this, we can rule out all frameworks that depend on effort or complexity. It cannot be too heavy on statistical data but has to take customer input as well as growth plans into account. As I work with people that are too smart to just believe what they are told, the methodology needs to be research-driven, data-informed, and as opinion-free as possible.

Another challenge we faced is that our system was close to the limits it could handle — both organisationally, and technically. That meant we needed to invest in costly and urgent growth initiatives. “Costly” meaning a couple of developer years, “urgent” meaning that we now wish we would have done it earlier. Sort of like when you wish you would have gone to the dentist before the toothache started.

Very relevant side note: When reading that we postponed getting our metaphorical teeth checked, you might have thought: “Well, that wasn’t smart.” But, and this is what this article is about, it was smart. When previous roadmaps have been built, a prioritization method that favored technical innovation, fast progress, and winning new customers absolutely made sense. It all depends on the situation. Also, premature optimization smells. We have since made the investment to set our system up for a higher magnitude of load and scale.

How we roadmap in 2021

Just looking at the value aspects of prioritization methodologies, we need something that works equally well for innovation, improving user experiences, organization together with architectural setup, and scaling up our platform to work with x times the load from last year.

Weighted scores are pretty perfect for that, as they do not prescribe what matters to you, and allow you to rank the influence each of the dimensions has.

Weighted scores can be a bit hard to wrap one’s head around. Here’s a food-based example to illustrate it: Deciding what’s for lunch.

Dimensions that matter:

  • Healthiness
  • Yumminess
  • Speed of preparation and eating
  • Food-coma-likelihood

Each day, those dimensions can have different importance (weight) for you. Monday, your mean calendar only gives you a 30-minute lunch break, so you weigh Speed at 70%, No-Food-coma at 25%, and Yumminess at a has-to-be-edible-5%. Which results in a “whatever is in the fridge” sandwich. On Saturdays you have plenty of time and want to indulge yourself and go for 20% Health, 80% Yummy — and I leave the rest to your imagination.

When deciding on what to eat, you assign a value to each of those dimensions to every single food item you have on your shortlist, sort by rank, and pick the top one — or for a roadmap, the top n ones. This is what we do in our quarterly plannings. This is how it looks like:

Sorted list of features with their weights for the four drivers vision, UX, delivery speed, scale

Using data to inform weights and scores

We review these dimensions once per year and adjust the weights with each quarterly planning. The main data point we use to inform the weight for “enjoyable experiences” and “scaling” is the number and impact of incidents, measured by SLOs and error budgets. Another important source is customer feedback and questions our support team gets.

To rank an individual item in the “innovation” and “enjoyable experiences” dimensions, the most important data point we use is the user impact score. It is basically the first two letters of RICE — Reach (“how many customers are having this problem?”) and Impact (“how important is it to them?”). We diligently collect every piece of customer feedback throughout the year and use that to determine the user impact score.

User impact score for an example feature, displaying customer feedback with their importance

Evaluating risk

This is a challenge I struggled with for years. Having been a software engineer for about half my career, I know the difficulties of estimating efforts or complexity for pretty much any programming work that couldn’t also be done by a program. That is especially true for topics around scaling and optimizations, which by nature require a lot of experimentation and testing; or those nasty Heisenbugs. Additionally and as mentioned earlier, using efforts or complexity to measure risk is fundamentally biased against tackling tooth-ache preventing work.

But ignoring delivery risk completely doesn’t seem wise either — reliable roadmaps are helpful, internally and for customers.

Curtains up for: “The Managing Complex Change Model“ (Mary Lippitt, 1987), also referred to as the Knoster model for managing change. The model identifies five key elements for successfully implementing organizational changes — and it applies well for all sorts of change, including changes to a product. In a nutshell, if one of those elements is missing, you won’t be able to deliver what you set out to do.

To succeed, you need vision + skills + incentives + resources + action plan. No vision => confusion. No skills => anxiety. No incentives => slow progress. No resources => Frustration. No action plan => False starts.

Using this, we can reduce the risk part to a set of yes / no questions, leading to a simple yes/no on whether or not a team can deliver the product change. If the answer is yes — brilliant, put it on the roadmap! If the answer is no — identify what is missing, and put these items on the roadmap. It is as easy as it sounds, and leads to great insights to improve teams or the organization.

TL;DR and Related Thoughts

  • When prioritizing work, we look at everything that takes time holistically — product innovation, organization, and tech. We think this is a better alternative than budgeting for tech or org work and planning those separately.
  • We assembled different parts of different frameworks into something that is a good fit for our situation and context. Tip: Look at completely different areas, don’t only stick to product management. You’ll find that there is a treasure trove of inspiration and ideas to try!
  • We do collect customer insights throughout the year, and it is… another treasure trove! Tip: If you use data to prioritize, make sure it is complete and correct enough to be valuable.
  • Tools help! We use and love productboard (the screenshots you see in this post are taken from there), but there are others, too.
  • Lastly, but most importantly — Let your north star guide you. No roadmapping methodology can ever replace a solid vision.

TL; Don’t want to read? Watch the video of my presentation at the OSN Speaker series!


How we Roadmap in 2021 was originally published in commercetools tech on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: Commercetools