Tag Archives: risk


Feedback and Data in Scrum

Collecting feedback and data is a vital aspect of Scrum. It enables you to learn if the right product with the right features is created. Scrum employs a three-step process to achieve this: A product increment is created, which is then exposed to the users, the customers, and the other stakeholders. This generates feedback and data, which triggers product backlog changes, as the following picture shows.


To leverage this process, the techniques used to gather the data and validate the product are crucial. If you use the wrong method, you are likely to collect wrong or insufficient data and draw the wrong conclusions. While there is a range of techniques available, Scrum recognises only one: the product demo, which is performed in the sprint review meeting.

But this does not mean that you should or cannot employ another technique! The opposite is true: The sprint demo may or may nor be right for product. Additionally, using a single data collection method over an extended period of time is usually a mistake. Every technique has its strengths and limitations, and none is always appropriate. You should hence choose the one that is most helpful for your product and combine complementing techniques.

A Selection of Helpful Techniques

To help you select the right method, I have complied five techniques happily used in Scrum in the table below. I discuss each technique in the following sections.

Technique Description Strengths Weaknesses
Product demo Demo the latest increment. Listen to the feedback, and ask open questions. Test limited functionality: Helps people imagine what using the product would be like. Feedback is available immediately. Feedback is based on what users hear and see; danger of influencing users.
Usability test Observe how users interact with the prototype or product in a controlled environment such as a lab. Understand if users employ the product as anticipated. Possibly collect data using analytics software. Data is available immediately. Artificial environment: users do not use the product “in the wild”; observer effect.
Release Release the software to a group of target users and collect data using analytics. Find out how the product is used in its target environment. Reach a larger test group. Run A/B tests. Does not explain why users employ the product in a certain way. Can take time to collect enough data.
Observation Observe users employ the product preferably in its target environment. Understand how users interact with the product. Can be time consuming; danger of observer effect and bias.
Spike  Create an executable prototype to address an architecture risk. Understand if an architecture or technology choice is feasible. Can lead to an over-engineered solution.

Product Demo

As its name suggests, the product demo presents the latest product increment to the appropriate users, customers, and internal stakeholders. The presenter explains how the users would employ the product to get a job done. Product demos are particularly valuable in the early sprints, as they allow you to get immediate feedback on limited functionality and very small increments: By wrapping the increment in a story, people can imagine what it would be like to use the product.

This strength is also a major weakness: The feedback is based on what people see and hear, not on their actual experience of using the product. What’s more, the presenter can influence the users inappropriately by talking up the product or asking closed questions, and powerful individuals like a senior manager can influence the views of the group. My post “The Product Demo as an Agile Market Research Technique” shares more tips and tricks on employing product demoes effectively.

Usability Test

A usability test allows you to understand how users interact with your product. Usability testing takes typically place in a controlled environment such as a meeting room or lab. Target users are asked to perform a task using the latest product increment, which may be a paper prototype or executable software. You then observe and record how people employ the product, and you can end the test with asking the participants about their experiences and impressions. It is often possible to conduct the test in the sprint review meeting.

While a usability test quickly generate real user data, the artificial environment and the observation can cause the users to act differently compared to working with the product in “the wild”, that is, in the environment in which the product will be used, for instance, at home, at work, in the car, on the train. This is where the next technique comes in.


Releasing software means giving a group of target users access to an early version of the product and asking them to use it in its target environment. The usage data is then recorded by an analytics tool, for instance, Google Analytics. The product is now employed in its target environment, and a larger test group can be employed, which reduces the risk of collecting data that is not representative for the market segment targeted. With the right analytics software, you can also collect data such as who interacts with which product feature when and how often, and which variant of a feature people prefer (A/B test).

On the downside, the technique cannot explain why people use a certain feature, or why they don’t. Additionally, there is a delay: It takes people time to download, install, and start using the latest increment before enough data is available to draw the right conclusions. In a Scrum context, this means that you either have to postpone the next sprint or continue with a different feature to mitigate the risk of moving in the wrong direction.

If you primarily use releases then your sprint review meeting changes: It is now used to synch the internal stakeholders and review the project progress rather than to validate the product.


Observation means watching users carefully as they employ the product in its target environment. This helps you understand how people interact with the product and use its features. It also allows you to debrief with the user and to learn about the user’s experience whilst interacting with the product.

The main drawback of observations that they can be time consuming particularly if you want to observe more than a few users. Users may also act differently with somebody watching them (observer effect), and your own biases may interfere with you ability to see clearly.


Spikes are prototypes to test an architectural or technology-related assumption, for instance, that Enterprise JavaBeans will be able to deliver the desired performance. They are usually cheap to create, generate the relevant knowledge quickly, and allow you to see if you can meet critical nonfunctional requirements.

Spikes become problematic if you employ them too much, if you worry more about how to build the product than why and what to build. If this happens the result is in an over-engineered product and a solution-centric mindset rather than a user-centric one. I once met a team that had been building spikes for nearly two years without having to show any shippable code. Telling them to stop it and think about the users was quite shocking for them.

Choosing the Right Technique

All the techniques discussed have their strengths and weaknesses. While you should always choose the technique that helps you meet your sprint goal and validate the increment effectively, I find it a helpful rule of thumb to use product demos, usability tests and spikes in the early sprints, and releases and observation in the later sprints, as the following picture illustrates.


The diagram above tries to balance the strengths and weaknesses of the different techniques, and it corresponds to a risk-driven approach where the key risks and critical assumptions are tackled in the early sprints. (For more info assumptions, risks and learning in Scrum see my post “Get Your Focus Right: Learning and Execution in Scrum“.)

Whichever techniques you choose, don’t make the mistake to rely on a single technique for an extended period of time. Every technique has its benefits and drawbacks, and no single technique is perfect. Combine qualitative and quantitative techniques, for instance, releases and observation, to leverage their respective strength and mitigate their weaknesses. Don’t be shy to experiment with different techniques to find out what works best for your product, and use the sprint retrospective to review their effectiveness.

Learn more

You can learn more about applying the right technique to gather feedback and data and its impact on the sprint review meeting and the “grooming” process by attending my Certified Scrum Product Owner training course. I also deliver the course onsite. Please get in touch to discuss the details.


I find it helpful to consider three questions when choosing a sprint goal: Why do we carry out the sprint? How do we reach its goal? And how do we know that the goal has been met? My sprint goal template therefore consists of three main parts: the actual goal, the method employed to reach the goal, and the metrics to determine if the goal has been met. It additionally provides a header section that allows you to state to which product and sprint the goal belongs, as the picture below shows. You can download the template as a PDF from romanpichler.com/tools/sprint-goal-template/ or by clicking on the image below.


The template above has grown out of my experience of working with Scrum for more than ten years, and it is inspired by the scientific method and Lean Startup. Let’s have a look at the template sections in more detail.

The Goal Section

The goal section states why it is worthwhile to undertake the sprint. Examples are:

  • Test an assumption about the user interaction and learn what works best for the user, for instance: “Will users be willing to register before using the product features?”
  • Address a technical risk such as: “Does the architecture enable the desired performance?”
  • Release a feature, for instance: “Get the reporting feature for general release.”

The sprint goal hence differs from listing the user stories that should be implemented. It communicates the reason for carrying out the work, and it provides a motivation for running the sprint. The sprint goal should be shared: The product owner and the development team should believe that working towards the goal is the right thing to do.

To choose the right sprint goal I find it helpful to consider the amount of uncertainty present. In the early sprints, addressing risks and testing assumptions allows me to learn about what the product should look like and do and how it is built. Once the key risks and critical assumptions have been dealt with, I like to focus on completing and optimising features, as the following picture shows:


The Method Section

This section addresses the question of how the goal is met. The default Scrum answer is simple: Create a (potentially shippable) product increment using the high-priority product backlog items, and demo it to the stakeholders in the sprint review meeting. But writing software and employing a product demo are not always the best methods to achieve the goal! A paper prototype can be good enough to test a visual design idea or an assumption about the user interaction, for instance. What’s more, other methods such as carrying out a usability test or releasing software to run an A/B test may well be more effective than a product demo. You should therefore carefully choose the right method and state it in this section.

But don’t stop there. Determine the test group, the people who should provide feedback and data. Who these individuals are depends on the sprint goal: If you are validating an assumption about the visual design, the user interaction or the product functionality, then you probably want to collect feedback and data from the users. But if you are addressing a technical risk, then users may not be able to help you. Consider inviting a senior developer or architect from another team instead. Stating the test group clarifies who “the stakeholders” are, who is required to provide feedback so that the right product is developed.

The Metrics Section

The metrics section communicates how you determine if the goal has been met. Which metrics you use depends on the method chosen. For a product demo, you may state that at least two thirds of the stakeholders present should respond positively to the new feature, for instance; for a usability test, at least three of the five testers are complete the task successfully in less than a minute; and for the release of a new feature, you might say that at least 80% of the users use the new functionality at least once within five days after launching the feature.

Whichever metrics you choose, make sure that they allow you to understand if and to which extent you have met the goal.

The Header Section

The header section consists of the two subsections “Product” and “Sprint”. They simply allow you to state which product and which sprint the goal belongs to.

Customise this section according to your needs. If you work for an agencies or an IT solution provider, you could replace “Product” with “Project”, for instance.

User Stories and the Sprint Goal

You may be wondering how the template relates to the user stories. Let me first reiterate that your sprint goal should differ from your user stories. The goal explains the why it is a good idea to carry out the sprint an implement the stories. The user stories enable you to reach the goal. It’s a common mistake to confuse the two.

To connect the template and the stories you have two options: You can state the relevant user stories in the template’s method section, or you can list them separately on the sprint backlog, as the following picture illustrates.


In the picture above, the sprint goal is stated on the left to the sprint backlog, which lists the user stories and the tasks required to meet the goal in form of a task board.

Learn more

You can learn more about choosing effective sprint gaols and applying the sprint goal template by attending my Certified Scrum Product Owner training course. I have written in more detail about sprint planning in my book “Agile Product Management with Scrum”. Please contact me for onsite and virtual product owner training.


When I talk to product owners about capturing ideas and requirements for their products, the conversation often focuses on users stories. While stories are great to describe the product functionality, they are not enough to create a great user experience.

Creating a Great User Experience

I like to think of a product description as a four-piece jigsaw puzzle: One piece describes the functionality, another one the user journeys and interactions. There is a third piece that captures the visual design, and a final piece for the nonfunctional properties such as performance. All four pieces have to be present and fit together to create a product with a great user experience, as the following picture illustrates:


The four pieces are best covered by different techniques: User stories work well for the product functionality. Scenarios, workflows, and storyboards are great to describe the journeys. Sketches and mock-ups capture the design, and constraint stories describe the nonfunctional properties, as shown in the picture below. No single technique can handle all four pieces well.


You can employ further techniques, of course, including screenshots and photos, context and activity diagrams, value stream maps, story maps, and so fourth. You can also combine different techniques, for instance, providing a design sketch for each step in a scenario. Use the techniques that work best for your product, and that enable you to describe your ideas so that they are easy to understand and to validate.

(Note that I have placed storyboards between “User Journeys” and “Visual Design” in the picture above as the technique can cover both aspects.)

Identifying Assumptions and Risks

There is another reason why a holistic product description is beneficial especially for new products and new features: It helps you identify all relevant assumptions and risks. If you only write user stories, chances are that you are unaware of the interaction and design risks. This may result in late failure or a poor user experience.

For instance, every now and then when I shop online, I come across a website that asks me to select the payment type before allowing me to enter the payment details on the next page. Unfortunately, I typically forget to make the right choice in the first step. As a consequence, I have to go back to the initial page to correct my selection and then move on to the details page. As my payments details are lost, I have to re-enter them. While this experience may be good to test my patience, it certainly does not make me want to use the website again. Considering and validating the user interaction could have easily resulted in a better design and in a more pleasant user experience.

Visualising and Managing the Four Aspects

A handy tool to capture and manage the four aspects is the Product Canvas. The canvas provides three major sections: The first section is called “Target Group” and describes the users and customers with their needs in form of personas. The second section named “Big Picture” outlines the desired user experience and describes the important user journeys, product capabilities, design ideas, and nonfunctional properties. The third section called “Product Details” captures the goal or hypothesis of the next cycle together with actionable items, which may be written as ready stories.

The Big Picture section plays a key role in capturing the four pieces, as the following sample canvas shows:


The picture above is an extract of the Product Canvas I used to create the current version of my website, romanpichler.com. The Big Picture section in the middle contains epics to describe the product functionality, storyboards to capture the user interactions, design sketches to illustrate important design ideas, and a performance constraint. It provides a coarse-grained but holistic description of the product. For more information on the canvas, please visit the Product Canvas tool page, which also provides downloadable template.


When you create a new product, take a holistically holistic approach. Describe the relevant user journeys, the product functionality, the visual design, and the nonfunctional properties. Neglecting one of them is likely to result in a suboptimal user experience, which may well have a negative impact on the success of your product.

You can learn more about the approach and techniques described in this post by attending my Certified Scrum Product Owner and my Product Canvas training course.


The Minimum Viable Product

The minimum viable product (MVP), as defined by Eric Ries, is a learning vehicle. It allows you to test an idea by exposing an early version of your product to the target users and customers, to collect the relevant data, and to learn form it. For instance, to test the viability of using ads as the major revenue source, you could release an early product increment with fake ads, and measure if and how often people click on them.

As lack of knowledge, uncertainty, and risk are closely related, you can also view the MVP as a risk reduction tool. In the example above, the MVP addresses the risk of developing a product that is not economically viable.

Since the MVP is about learning, it’s no surprise that it plays a key part in Lean Startup’s build-measure-learn cycle, as the following picture shows:


The MVP is called minimum, as you should spend as little time and effort to create it. But this does not means that it has to be quick and dirty. How long it takes to create an MVP and how feature-rich it should be, depends on your product and market. But try to keep the feature set as small as possible to accelerate learning, and to avoid wasting time and money – your idea may turn out to be wrong!

While the MVP should facilitate validated learning, I find it perfectly OK to work with MVPs such as paper prototypes that do not generate quantitative data, as long as they help to test the idea and to acquire the relevant knowledge.

The Minimal Marketable Product

The minimal marketable product (MMP) is a different type of product. It is based on the idea that less is more. The MMP describes the product with the smallest possible feature set that addresses the user needs, creates the desired user experience, and can hence be marketed and sold successfully. The MMP is a tool to reduce time-to-market: It can be launched more quickly than a fat, feature-rich one.

Creating a product with just the right amount of features sounds like common sense. Why would we create more features than necessary? Sadly, I have seen many projects develop over-engineered products with lots of shiny features that provided little value to the users, but cluttered the product and increased the maintenance cost. And it’s not just the others: I am constantly tempted to add just another cool feature to a product, or to write a few extra lines in a blog post. Using the concept of an MMP helps me focus on what really matters, and remove unnecessary features (and lines).

A great example of an MMP is Apple’s original iPhone launched in 2007. I know that the first iPhone was a complex product, and that many people worked incredibly hard on it. But I find it amazing how many features the phone did not provide compared to its competitors: no copy-and-paste, no Outlook integration, and no voice recognition, to name just a few. Nevertheless the phone was still a staggering success. How come?

The key to creating a successful MMP is to “develop the product for the few, not the many,” as Steve Blank puts it, and to focus on those features that make a real difference to the users. To discover the right features, the MVP is a fantastic tool.

Combining the MVP and the MMP

To combine the two concepts, develop one or more MVPs to test your ideas and to acquire the relevant knowledge. Then use your new insights to create and launch the MMP – a product with just the right features and a great user experience, as the following picture shows:


Note that a minimal marketable product differs from a viable one: It is complete enough to be ready for general release, as indicated by the gift wrapping in the picture above. What’s more, launch preparation activities have to take place for an MMP, for instance, creating advertising campaigns, or gaining certification. Some of your MVPs are likely to be throwaway prototypes that only serve to acquire the necessary knowledge; others are reusable product increments that morph into a marketable product.

More Information

You can learn more about applying the two concepts by attending my Agile Product Planning training course. You can also find a more detailed discussion of the minimal marketable product in my book “Agile Product Management with Scrum“.


Discovering Lean Startup was inspiring for me: I felt I had found an approach that could complement Scrum nicely. Since then I have been combining the two approaches in my own new product development work as well as helping my clients to do so. This post shares my experiences and insights. It maps out a high-level process for creating new products within existing businesses focussing on product management practices and tools.

From Vision to Launch

Lean Startup encourages us to first investigate if there is a need worthwhile serving before we worry about the details of the new product. The former is called “Problem Validation”, the latter “Solution Validation”. While traditional approaches also suggest carrying out market research and analysis before we engage in product planning and definition, lean approaches increase the speed at which we operate. This allows us to fail and learn faster, to adapt our product strategy and tactics quickly, and to hopefully launch the right product with the right features sooner.

The following diagram illustrates the process I apply to create new products:

Problem Validation

The new product development process depicted above starts with a vision. It then uses a series of cycles or iterations to transform the vision into a product ready for launch. The focus of the early iterations is to understand what problem the product solves, who the customers and users are, and how the product benefits the company creating it.

I find it important that the product owner leads this effort and carries out the necessary work together with a cross-functional team. While the team composition depends on the product, having a marketer, sales representative, service representative, a UX designer, and a developer on board is usually helpful. Product owner and team capture their assumptions about the market by using a tool like the Vision Board, Business Model Canvas, or Lean Canvas.

Employing techniques such as direct observation, problem interviews, and competitor analysis helps them validate their assumptions and update the board or canvas, as the following pictures illustrates:

Additionally, the developers may create spikes (throwaway prototypes) to explore the feasibility of the product. Target users and customers participate in the process, for instance, by acting as interview partners.

Solution Validation

Once product owner and team have shown that there is a problem that’s worthwhile addressing, the focus changes to validating the solution and developing the actual product: The team needs to learn what the desired user experience is, what functionality the product should provide, and how it should be built.

The new focus requires adapting the team composition: The product owner, the UX designer and the developer stay, but the marketer, sales and service representatives leave the team. They continue to participate in the development effort as stakeholders. New developers, testers, and other roles required to create a great product join the team.

Product owner and team capture their assumptions about the product including the user interaction, the user interface design, the functionality, and the nonfunctional aspects using a tool like the Product Canvas and techniques such as scenarios, user stories, and design sketches. The assumptions are tested by collecting and analysing feedback on prototypes, mock-ups, and product increments/MVPs. Useful techniques to gather the feedback include product demos, users tests, and releases (to selected users).

As you may have noticed, the picture above suggests that “Stakeholders incl. users” participate in the process, and provide feedback or data on work results. While using target users and customers to validate ideas is generally helpful, it is not always appropriate. Imagine addressing a key technical risk in your first solution validation iteration: It probably makes little sense to invite users to the review meeting to understand if the approach chosen is viable. Similarly, if a disruptive product is developed it can be hard to find target users that are not too attached to their current solution.

Scaling and Launch Preparation

Once the Product Canvas and the architecture have started to stabilise, you can start adding more people to the project. I find it useful to break-up the original team so that at least one or two members are part of each new team, as this helps the new members get up to speed. I also suggest you grow your new product development project in a step-wise fashion: Scale from one to two teams, then from two to four, and so forth. This allows you to understand the impact on the people and the process including product ownership .

Be aware of the danger of premature scaling: Adding too quickly too many people. This tends to lead to a bloated, over-engineered product in my experience, and it prevents you form being able to experiment effectively. Therefore delay scaling until you have resolved the main risks – until you can focus on completing features and adding new ones.

Finally, as you make progress with your solution validation work, don’t forget to prepare the product launch. Having a marketer present at the sprint review meetings should help, but you may find that a dedicated marketing may be required to prepare and execute the launch well.


The following table summarises my recommendations for transforming an idea into a shippable product:

The process described in this post is based on work by Steve Blank, Ash Maurya, Eric Ries, and Ken Schwaber. The eagle-eyed process historians amongst you may have noticed that the idea of progressive scaling has its roots in the (Rational) Unified Process. Experimental, iterative product development has been around for quite some time of course, I believe at least since the late 19th century when Thomas Edison established the Menlo Park laboratory.

You can learn more about combining lean and agile techniques to create new products by attending my Agile Product Management training course. Please contact me for onsite workshops.

If you have any feedback, comments, or questions, then I’d love to hear from you!

Sample Storyboard

User stories are great at capturing product functionality. But they are less suited to describe the user interaction in more detail. This is where scenarios and storyboards come into play: Both are great tools to describe the interaction steps. In this post, I explain what scenarios and storyboards are, and how they can be used effectively in an agile context, and how the two techniques relate to user stories.

Scenarios in a Nutshell

Scenarios and storyboards are great to explore and describe how a user interacts with a product. When we started to work on the re-launch of our website, for instance, I wrote the following scenario:

  1. It’s Tuesday morning, and Mary is working on her computer. She wants to book Roger Smith on a public Certified Scrum Product Owner course taught by Roman.
  2. Mary visits romanpichler.com and chooses a public CSPO class.
  3. She enters the participant information including first name, last name, email address, special dietary requirements.
  4. She then chooses a payment option and enters the payment details.
  5. Mary accepts the terms and conditions, and confirms the booking.
  6. Mary sees that her booking has been successful. After a short while, Roger receives an email confirmation with the booking details.

The scenario above describes the steps Mary has to take to book a seat on one of our public training courses. Mary is a persona who represents a user of our website: an HR employee of a large company, and who’s main need it is to book employees on a training course.

Note that I have tried to make the scenario descriptive and engaging while focussing on the key aspects of the interaction.

Storyboards Summarised

Storyboards are similar to scenarios: They illustrate the interaction required to achieve a goal. But instead of using a list of steps, a storyboard visualises the interaction similar to a comic strip. Here is a sample board I created to explore another interaction for our new website:

The storyboard above describes how the persona Mary books several employees on the same training course. The board consists of a series of frames. Each frame shows sample data. Underneath it, I added a brief description of what Mary does at each step.

Note that I have done my best to describe the functional aspects of the interaction, and not to design the user interface: When I was working on the board, we did not have any design sketches and mock-ups available. I generally find it good practice to capture the product functionality necessary to meet the main user needs before designing the user interface.

What about User Stories?

User stories are another technique to describe the user interaction. A large story or epic allows us to summarise the interaction acting as placeholders for more detailed stories. I like to think of an epic is as a scenario rolled up into a brief narrative: it hides all the specifics of the user interaction. Detailed stories correspond to individual steps in a scenario, and describe a specific piece of product functionality.

The first thing I usually do when working on a new product is to write epics. To discover the right ones, I use the needs of the personas. Starting out with epics helps me quickly sketch the new product functionality, and it keeps the Product Canvas or product backlog concise and manageable.

But working exclusively with epics can be problematic, particularly when the epic carries risk: If we only have a coarse-grained description available, then it’s difficult to test our assumptions about how the users interact with the product. I therefore prefer to create a scenario or storyboard for risky epics, as the picture below shows:

Creating scenarios or storyboards for selected epics allows me to explore the user interaction in more detail, to describe the necessary steps and their relationship. This helps me test my assumptions, for instance, by creating a paper prototype that implements the scenario in order to carry out an early user test.

This is, of course, not the only way to combine scenarios and user stories. You can also derive stories from a scenario, and you can use a scenario to illustrate the relationship between different stories. The following diagram illustrates the three options:

Choose the options that are most helpful in your context, and combine them, as it makes sense. You can, for instance, start with the first option (as I did above), then derive new stories from your scenario or storyboard, and finally capture the relationship between the new stories in a new scenario.


Scenarios and storyboards are great tools to describe how users interact with a product. They also complement user stories nicely: Scenarios and boards help explore risky stories, discover new user stories, and capture the relationship between stories. Make sure you include scenarios and storyboards in your product owner toolbox.

You can learn more about working with scenarios and storyboards by attending my Certified Scrum Product Owner training course.