Tag Archives: user feedback


Feedback and Data in Scrum

Collecting feedback and data is a vital aspect of Scrum. It enables you to learn if the right product with the right features is created. Scrum employs a three-step process to achieve this: A product increment is created, which is then exposed to the users, the customers, and the other stakeholders. This generates feedback and data, which triggers product backlog changes, as the following picture shows.


To leverage this process, the techniques used to gather the data and validate the product are crucial. If you use the wrong method, you are likely to collect wrong or insufficient data and draw the wrong conclusions. While there is a range of techniques available, Scrum recognises only one: the product demo, which is performed in the sprint review meeting.

But this does not mean that you should or cannot employ another technique! The opposite is true: The sprint demo may or may nor be right for product. Additionally, using a single data collection method over an extended period of time is usually a mistake. Every technique has its strengths and limitations, and none is always appropriate. You should hence choose the one that is most helpful for your product and combine complementing techniques.

A Selection of Helpful Techniques

To help you select the right method, I have complied five techniques happily used in Scrum in the table below. I discuss each technique in the following sections.

Technique Description Strengths Weaknesses
Product demo Demo the latest increment. Listen to the feedback, and ask open questions. Test limited functionality: Helps people imagine what using the product would be like. Feedback is available immediately. Feedback is based on what users hear and see; danger of influencing users.
Usability test Observe how users interact with the prototype or product in a controlled environment such as a lab. Understand if users employ the product as anticipated. Possibly collect data using analytics software. Data is available immediately. Artificial environment: users do not use the product “in the wild”; observer effect.
Release Release the software to a group of target users and collect data using analytics. Find out how the product is used in its target environment. Reach a larger test group. Run A/B tests. Does not explain why users employ the product in a certain way. Can take time to collect enough data.
Observation Observe users employ the product preferably in its target environment. Understand how users interact with the product. Can be time consuming; danger of observer effect and bias.
Spike  Create an executable prototype to address an architecture risk. Understand if an architecture or technology choice is feasible. Can lead to an over-engineered solution.

Product Demo

As its name suggests, the product demo presents the latest product increment to the appropriate users, customers, and internal stakeholders. The presenter explains how the users would employ the product to get a job done. Product demos are particularly valuable in the early sprints, as they allow you to get immediate feedback on limited functionality and very small increments: By wrapping the increment in a story, people can imagine what it would be like to use the product.

This strength is also a major weakness: The feedback is based on what people see and hear, not on their actual experience of using the product. What’s more, the presenter can influence the users inappropriately by talking up the product or asking closed questions, and powerful individuals like a senior manager can influence the views of the group. My post “The Product Demo as an Agile Market Research Technique” shares more tips and tricks on employing product demoes effectively.

Usability Test

A usability test allows you to understand how users interact with your product. Usability testing takes typically place in a controlled environment such as a meeting room or lab. Target users are asked to perform a task using the latest product increment, which may be a paper prototype or executable software. You then observe and record how people employ the product, and you can end the test with asking the participants about their experiences and impressions. It is often possible to conduct the test in the sprint review meeting.

While a usability test quickly generate real user data, the artificial environment and the observation can cause the users to act differently compared to working with the product in “the wild”, that is, in the environment in which the product will be used, for instance, at home, at work, in the car, on the train. This is where the next technique comes in.


Releasing software means giving a group of target users access to an early version of the product and asking them to use it in its target environment. The usage data is then recorded by an analytics tool, for instance, Google Analytics. The product is now employed in its target environment, and a larger test group can be employed, which reduces the risk of collecting data that is not representative for the market segment targeted. With the right analytics software, you can also collect data such as who interacts with which product feature when and how often, and which variant of a feature people prefer (A/B test).

On the downside, the technique cannot explain why people use a certain feature, or why they don’t. Additionally, there is a delay: It takes people time to download, install, and start using the latest increment before enough data is available to draw the right conclusions. In a Scrum context, this means that you either have to postpone the next sprint or continue with a different feature to mitigate the risk of moving in the wrong direction.

If you primarily use releases then your sprint review meeting changes: It is now used to synch the internal stakeholders and review the project progress rather than to validate the product.


Observation means watching users carefully as they employ the product in its target environment. This helps you understand how people interact with the product and use its features. It also allows you to debrief with the user and to learn about the user’s experience whilst interacting with the product.

The main drawback of observations that they can be time consuming particularly if you want to observe more than a few users. Users may also act differently with somebody watching them (observer effect), and your own biases may interfere with you ability to see clearly.


Spikes are prototypes to test an architectural or technology-related assumption, for instance, that Enterprise JavaBeans will be able to deliver the desired performance. They are usually cheap to create, generate the relevant knowledge quickly, and allow you to see if you can meet critical nonfunctional requirements.

Spikes become problematic if you employ them too much, if you worry more about how to build the product than why and what to build. If this happens the result is in an over-engineered product and a solution-centric mindset rather than a user-centric one. I once met a team that had been building spikes for nearly two years without having to show any shippable code. Telling them to stop it and think about the users was quite shocking for them.

Choosing the Right Technique

All the techniques discussed have their strengths and weaknesses. While you should always choose the technique that helps you meet your sprint goal and validate the increment effectively, I find it a helpful rule of thumb to use product demos, usability tests and spikes in the early sprints, and releases and observation in the later sprints, as the following picture illustrates.


The diagram above tries to balance the strengths and weaknesses of the different techniques, and it corresponds to a risk-driven approach where the key risks and critical assumptions are tackled in the early sprints. (For more info assumptions, risks and learning in Scrum see my post “Get Your Focus Right: Learning and Execution in Scrum“.)

Whichever techniques you choose, don’t make the mistake to rely on a single technique for an extended period of time. Every technique has its benefits and drawbacks, and no single technique is perfect. Combine qualitative and quantitative techniques, for instance, releases and observation, to leverage their respective strength and mitigate their weaknesses. Don’t be shy to experiment with different techniques to find out what works best for your product, and use the sprint retrospective to review their effectiveness.

Learn more

You can learn more about applying the right technique to gather feedback and data and its impact on the sprint review meeting and the “grooming” process by attending my Certified Scrum Product Owner training course. I also deliver the course onsite. Please get in touch to discuss the details.


The Minimum Viable Product

The minimum viable product (MVP), as defined by Eric Ries, is a learning vehicle. It allows you to test an idea by exposing an early version of your product to the target users and customers, to collect the relevant data, and to learn form it. For instance, to test the viability of using ads as the major revenue source, you could release an early product increment with fake ads, and measure if and how often people click on them.

As lack of knowledge, uncertainty, and risk are closely related, you can also view the MVP as a risk reduction tool. In the example above, the MVP addresses the risk of developing a product that is not economically viable.

Since the MVP is about learning, it’s no surprise that it plays a key part in Lean Startup’s build-measure-learn cycle, as the following picture shows:


The MVP is called minimum, as you should spend as little time and effort to create it. But this does not means that it has to be quick and dirty. How long it takes to create an MVP and how feature-rich it should be, depends on your product and market. But try to keep the feature set as small as possible to accelerate learning, and to avoid wasting time and money – your idea may turn out to be wrong!

While the MVP should facilitate validated learning, I find it perfectly OK to work with MVPs such as paper prototypes that do not generate quantitative data, as long as they help to test the idea and to acquire the relevant knowledge.

The Minimal Marketable Product

The minimal marketable product (MMP) is a different type of product. It is based on the idea that less is more. The MMP describes the product with the smallest possible feature set that addresses the user needs, creates the desired user experience, and can hence be marketed and sold successfully. The MMP is a tool to reduce time-to-market: It can be launched more quickly than a fat, feature-rich one.

Creating a product with just the right amount of features sounds like common sense. Why would we create more features than necessary? Sadly, I have seen many projects develop over-engineered products with lots of shiny features that provided little value to the users, but cluttered the product and increased the maintenance cost. And it’s not just the others: I am constantly tempted to add just another cool feature to a product, or to write a few extra lines in a blog post. Using the concept of an MMP helps me focus on what really matters, and remove unnecessary features (and lines).

A great example of an MMP is Apple’s original iPhone launched in 2007. I know that the first iPhone was a complex product, and that many people worked incredibly hard on it. But I find it amazing how many features the phone did not provide compared to its competitors: no copy-and-paste, no Outlook integration, and no voice recognition, to name just a few. Nevertheless the phone was still a staggering success. How come?

The key to creating a successful MMP is to “develop the product for the few, not the many,” as Steve Blank puts it, and to focus on those features that make a real difference to the users. To discover the right features, the MVP is a fantastic tool.

Combining the MVP and the MMP

To combine the two concepts, develop one or more MVPs to test your ideas and to acquire the relevant knowledge. Then use your new insights to create and launch the MMP – a product with just the right features and a great user experience, as the following picture shows:


Note that a minimal marketable product differs from a viable one: It is complete enough to be ready for general release, as indicated by the gift wrapping in the picture above. What’s more, launch preparation activities have to take place for an MMP, for instance, creating advertising campaigns, or gaining certification. Some of your MVPs are likely to be throwaway prototypes that only serve to acquire the necessary knowledge; others are reusable product increments that morph into a marketable product.

More Information

You can learn more about applying the two concepts by attending my Agile Product Planning training course. You can also find a more detailed discussion of the minimal marketable product in my book “Agile Product Management with Scrum“.


As the product owner you should look outward to the market, and inward to the team and the stakeholders – similar to a cubistic Picasso portrait where the person’s eyes look in two different directions, as the following picture shows:


The picture above is based on Pablo Picasso’s portrait of Nusch Éluard. Picasso created the painting in 1938 using charcoal and pencil on canvas.

The Outward View: Users, Customers, and Competitors

As the product owner, you should be first and foremost concerned with understanding the needs of the users and customers, and figuring out how well your product is meeting them. Helpful techniques to test your ideas and acquire the necessary knowledge include observing users, conducting user interviews, carrying out product demos and users tests, and employing MVPs. You should hence “get out of the building,” as Steve Blanks puts it, and engage with users and customers. Don’t blindly trust the opinions of senior management or the sales group. Use data to validate your assumptions.

Don’t forget to pay some attention to the competition, and the overall market developments. Even a company like Apple cannot afford to ignore their competitors, as the latest iOS release shows: Some of its new features, such as killing an app by swiping upwards, had been available on other devices.

The Inward View: Team and Stakeholders

While the users and customers should be your number one priority, creating a great product requires the product owner to closely collaborate with the development team. This collaboration is essential: It provides direction to team, and it leverages the team’s creativity and knowledge. Helpful techniques include jointly carrying out the research/validation work such as product demos and users tests, and updating the Product Canvas or backlog together.

As important as it is, collaborating with the team is not enough – particularly in larger organisations. Take into account the interests and needs of the internal stakeholders including senior management, marketing, sales, service, operations, and other business groups. Involve their representatives early and regularly, for instance, by inviting them to review meetings. This allows you to align the stakeholders, and to benefit from their ideas and knowledge.

Make sure, though, that you own the product and have the final say about what gets done. Avoid the trap of becoming a feature broker – someone who negotiates compromises between the stakeholders. Great products are not created by determining the smallest common denominator, and a product that tries to please everyone is likely to please no one. Follow your vision instead, and test your ideas with actual users.


You certainly don’t have to become a world-renown painter to be a good product owner. But to create a successful product, you should keep a firm eye on the market while collaborating with the development team and the internal stakeholders. Balance the two aspects, and avoid neglecting any for a longer period of time. In doubt, focus on understanding the user and customer needs and how to best address them. Leverage the ideas of the team and the stakeholders. But follow your vision, and validate your ideas early and frequently.

You can learn more about becoming an effective product owner by attending my Certified Scrum Product Owner training course.


Discovering Lean Startup was inspiring for me: I felt I had found an approach that could complement Scrum nicely. Since then I have been combining the two approaches in my own new product development work as well as helping my clients to do so. This post shares my experiences and insights. It maps out a high-level process for creating new products within existing businesses focussing on product management practices and tools.

From Vision to Launch

Lean Startup encourages us to first investigate if there is a need worthwhile serving before we worry about the details of the new product. The former is called “Problem Validation”, the latter “Solution Validation”. While traditional approaches also suggest carrying out market research and analysis before we engage in product planning and definition, lean approaches increase the speed at which we operate. This allows us to fail and learn faster, to adapt our product strategy and tactics quickly, and to hopefully launch the right product with the right features sooner.

The following diagram illustrates the process I apply to create new products:

Problem Validation

The new product development process depicted above starts with a vision. It then uses a series of cycles or iterations to transform the vision into a product ready for launch. The focus of the early iterations is to understand what problem the product solves, who the customers and users are, and how the product benefits the company creating it.

I find it important that the product owner leads this effort and carries out the necessary work together with a cross-functional team. While the team composition depends on the product, having a marketer, sales representative, service representative, a UX designer, and a developer on board is usually helpful. Product owner and team capture their assumptions about the market by using a tool like the Vision Board, Business Model Canvas, or Lean Canvas.

Employing techniques such as direct observation, problem interviews, and competitor analysis helps them validate their assumptions and update the board or canvas, as the following pictures illustrates:

Additionally, the developers may create spikes (throwaway prototypes) to explore the feasibility of the product. Target users and customers participate in the process, for instance, by acting as interview partners.

Solution Validation

Once product owner and team have shown that there is a problem that’s worthwhile addressing, the focus changes to validating the solution and developing the actual product: The team needs to learn what the desired user experience is, what functionality the product should provide, and how it should be built.

The new focus requires adapting the team composition: The product owner, the UX designer and the developer stay, but the marketer, sales and service representatives leave the team. They continue to participate in the development effort as stakeholders. New developers, testers, and other roles required to create a great product join the team.

Product owner and team capture their assumptions about the product including the user interaction, the user interface design, the functionality, and the nonfunctional aspects using a tool like the Product Canvas and techniques such as scenarios, user stories, and design sketches. The assumptions are tested by collecting and analysing feedback on prototypes, mock-ups, and product increments/MVPs. Useful techniques to gather the feedback include product demos, users tests, and releases (to selected users).

As you may have noticed, the picture above suggests that “Stakeholders incl. users” participate in the process, and provide feedback or data on work results. While using target users and customers to validate ideas is generally helpful, it is not always appropriate. Imagine addressing a key technical risk in your first solution validation iteration: It probably makes little sense to invite users to the review meeting to understand if the approach chosen is viable. Similarly, if a disruptive product is developed it can be hard to find target users that are not too attached to their current solution.

Scaling and Launch Preparation

Once the Product Canvas and the architecture have started to stabilise, you can start adding more people to the project. I find it useful to break-up the original team so that at least one or two members are part of each new team, as this helps the new members get up to speed. I also suggest you grow your new product development project in a step-wise fashion: Scale from one to two teams, then from two to four, and so forth. This allows you to understand the impact on the people and the process including product ownership .

Be aware of the danger of premature scaling: Adding too quickly too many people. This tends to lead to a bloated, over-engineered product in my experience, and it prevents you form being able to experiment effectively. Therefore delay scaling until you have resolved the main risks – until you can focus on completing features and adding new ones.

Finally, as you make progress with your solution validation work, don’t forget to prepare the product launch. Having a marketer present at the sprint review meetings should help, but you may find that a dedicated marketing may be required to prepare and execute the launch well.


The following table summarises my recommendations for transforming an idea into a shippable product:

The process described in this post is based on work by Steve Blank, Ash Maurya, Eric Ries, and Ken Schwaber. The eagle-eyed process historians amongst you may have noticed that the idea of progressive scaling has its roots in the (Rational) Unified Process. Experimental, iterative product development has been around for quite some time of course, I believe at least since the late 19th century when Thomas Edison established the Menlo Park laboratory.

You can learn more about combining lean and agile techniques to create new products by attending my Agile Product Management training course. Please contact me for onsite workshops.

If you have any feedback, comments, or questions, then I’d love to hear from you!

Product Canvas Creation Workshop Overview

The Product Canvas is a simple, yet powerful tool that helps you create a product with a great user experience and the right features. This post explains how you can create your initial canvas using a collaborative workshop.

Note that many of the recommendations below also apply to other agile and lean product definition tools including a traditional product backlog.


The Product Canvas creation workshop wants to kick-start your solution validation and product definition activities. It helps you change the focus from problem validation – exploring if there is a need that the new product addresses – to building a product with the right features and the right user experience. The following picture summarises the workshop, and the rest of this post explains the details.


Everyone tasked with creating the product should attend the canvas creation workshop: the product owner, the team developing and testing the product, and the ScrumMaster or coach. This creates shared ownership, and it is likely to result in better decisions, as the entire team’s creativity and knowledge are leveraged.

I generally recommend that the stakeholders – the users, and customers as well as the internal stakeholders – do not attend the workshop, but share their ideas and feedback based on prototypes and product increments, for instance, in the sprint review meetings. This allows the product owner and the team to be creative before the stakeholders provide input.


Before you start the workshop, you should be able to confidently answer the following questions:

  1. Who are the product’s users and who are the customers?
  2. What problem does the product solve? What benefits does it generate for its users? What is the product’s value proposition?
  3. What business benefits does the product creates? Why should the company invest in it?
  4. What kind of product is it? What are the three to five features that make it stand out?

I like to capture the insights above using the Vision Board. But you can also employ other tools including Ash Maurya’s Lean Canvas and Alexander Osterwalder’s Business Model Canvas, as the following picture shows:

Being able to answer the questions above means that some problem validation has taken place prior to the workshop, for instance, by carrying out user observations and problem interviews. Note that the strength of the Product Canvas is solution validation – building the right product – and not discovering if the product should be built in the first place!

To ensure a smooth workshop, use a facilitator, for instance, the ScrumMaster. Organise an appropriate room with lots of wall space. Have the necessary materials available including paper sheets, paper cards, adhesive notes, masking tape, markers, and pencils. Starting out with paper and pencil is effective and fun in my experience, even if you intend to use a digital canvas.

Steps to Create the Canvas

To create your initial Product Canvas take the following three steps: Create personas; outline the user experience and the features; determine what to do in the first sprint, as the picture below shows:

The first step creates personas based on the insights gained in your problem validation work. The personas allow you to connect with the target users and customers. Their needs enable you to discover the right product features. I also recommend using a primary persona, as it creates focus and facilitates decision-making. You can read more about personas in the post  “A Template for Writing Great Personas”.

The second step describes the product comprehensively but at a rough, coarse-grained level. Helpful techniques to capture the user experience and the product functionality include scenarios, storyboards, epics, constraint stories, and design sketches / mock-ups. Make sure that the product features you identify address the need of a persona, or support the business model.

The third step determines what should be done in the first sprint. As you are about to start building the first product increment, you should address the greatest risk or the biggest uncertainty. This could be a lack of knowledge surrounding the user interaction, the user interface design, a product feature, or the architecture and technology. See my post “Effective Sprint Goals” for more information on choosing the right goal or hypothesis. Finally, determine what needs to be done to reach the goal, or to test the hypothesis, for instance, creating a scenario and a paper prototype to learn more about the user interaction.

The three steps above form a breadth-first approach: The product is sketched holistically, but the details are determined on a sprint-by-sprint basis. This keeps your canvas concise, and allows you to make changes quickly and effectively.


At the end of the workshop you should have a Product Canvas that is good enough to start sprinting: to start building product increments/MVPs, gathering feedback, and integrating the insights gained, as the following picture shows:

Make sure you create a good-enough canvas, but not a perfect one. Your Product Canvas will change and evolve anyway based on the feedback you receive!


Four to eight hours should be enough time to create your initial canvas. If you require more time, then this may be a sign that you haven’t got enough information available and may have to carry out more problem validation and market research.


Employing a collaborative workshop, and following the process described above creates an initial Product Canvas that allows you to focus on solution validation – building the product with the right user experience and features. Make sure you understand the product’s value proposition before you enter the workshop, describe the product holistically at a coarse-grained level, and don’t worry too much about the product details: These will emerge incrementally based on the feedback you gather.

You can learn more about the Product Canvas creation workshop by attending my Certified Scrum Product Owner training course. If you would like me to help you run a Product Canvas workshop, then please contact me.



I don’t know about you, but every time I read the word “nonfunctional”, I feel a yawn coming on, and I start to look for something else to do. While nonfunctional requirements or quality attributes, as they are also called, may not be sexy, they are still important: They impact the user experience, and they influence architecture and design decisions.

Say you are looking for a new car, and you’ve found one that looks great. The car also has a powerful engine, lots of safety features, and a sat nav. But during a test drive, you discover that it’s noisy inside the cabin, the seats aren’t comfy, and the actual fuel consumption is surprisingly high. While the car has all the right features, the driving experience is not great – and that’s due to its poor nonfunctional properties.

Another example is the training part of my website. If the response time is sluggish, and it takes too long to receive a confirmation once a seat has been booked, then users are unlikely to have an enjoyable experience. Other common nonfunctional requirements include robustness, interoperability, usability, and compliance with a regulation or a standard.

Discovering Nonfunctionals

As important as they are, nonfunctional properties are sometimes overlooked, particularly in an agile context where we spend less time with upfront research and analysis. This can be particularly painful for those attributes that apply to the entire product. If, for instance, you forget to capture that your mobile app should be available on iOS and on Android, then this may require correcting fundamental architecture and technology choices, which is likely to delay the project or make it more expensive.

If you are unsure which nonfunctional requirements are important for your product, then I suggest you engage with the users and the development team. Try observing users, conducting problem interviews, and carrying out user tests using early product increments / prototypes. The tests allow you to discover new attributes and to understand if, for instance, the performance is acceptable.

The development team is a great partner for finding relevant nonfunctional requirements, particularly if the team members have worked on a similar product before or deal with support and production issues. If that’s not the case, then you should consider inviting members of the operations and customer services group to listen to their views on qualities such as robustness and availability.

Describing Nonfunctionals

I like to capture nonfunctional requirements as constraint stories. Here is an example:

Constraint Story

I call the story above a constraint, as it constraints other user stories. Just like a regular story, the constraint has two parts: a narrative and a list of acceptance criteria. The narrative describes the nonfunctional requirement from the perspective of the persona Mary. The criteria clarify the interaction and describe the environment. Both are required to validate the constraint. (Tom Gilb’s Planguage, which is an alternative approach to describing nonfunctional requirements, would call the first condition “scale” by the way. The narrative contains the “target” and the “gist”.)

Whatever format you choose to capture your nonfunctionals, ensure that the description precise and testable. If I say, for instance, that booking a training course on our website should be quick, then that’s a first step towards describing the attribute. But it would be too vague to characterise the desired user experience, to help the development team make the right architecture choices, and to validate the constraint. I will hence have to iterate over it, which is best done together with the development team.


Explore nonfunctional requirements that apply to the entire product or to important features early on. This helps you create a great user experience, and make the right architecture and technology decisions. Capture the requirements precisely to ensure testability. And don’t rush to test-drive a Ford Shelby Mustang GT500 depicted at the top of this post. It’s not the greatest sports car on the planet – at least according to Top Gear’s Jeremy Clarkson.

You can find out more about working with nonfunctional requirements by attending my Certified Scrum Product Owner training course. Unfortunately, I won’t be able to tell you more about Mustang cars: I’ve only driven one many years ago and only for a few miles. All I remember is: It was fast.