The sprint review is maybe the most important Scrum meeting for product people. Applied correctly, it increases the chances of creating a successful product. But I find that the meeting is not always used effectively. My article addresses this issue and shares practical tips for getting the most out of the sprint review.
Use the Meeting for Product Discovery, Not Only Delivery
The sprint review meeting serves two main purposes:
- Product discovery: Validate the latest product increment/prototype and discover opportunities to offer new and enhanced features. Ensure that the product provides the right functionality and user experience. One way to achieve this is to invite (selected) users and customers, as well as stakeholders, run a demo, and ask the attendees for feedback.
- Product delivery: Determine what is done and track the development progress. Maximise the chances that the product goal (outcome) can be achieved on time and on budget, and that the product will be delivered as expected.
Many sprint reviews I have attended emphasised the second objective and neglected the first one. Don’t make this mistake. If a product does not offer the right features, delivering it on time and on budget is of little value.
Involve the Right People
Collecting feedback from the right people is crucial to make effective product decisions: If you invite the wrong individuals or if key people are missing, you are unlikely to receive the feedback you need. You should therefore make sure that you invite the right individuals.
As a rule of thumb, ask those people to attend whose input you need to validate the latest product increment and move the product forward. These are usually users and customers, as well as the key stakeholders. The latter may include people from marketing, sales, service and support, and other business units, depending on your product and organisation.
Ask the Scrum Master to facilitate the meeting and establish ground rules. Alternatively, see if another product manager/owner can act as a facilitator. This frees you from having to moderate the meeting and allows you to focus on gathering feedback and determining the development progress.
Split the Meeting When Needed
Sometimes it’s helpful to split the sprint review meeting into two parts. In the first part, you and the development team get together. The team demos the product increment to you. You then give feedback to the team members and determine which items are done and how much progress has been made using a tool like the release burndown chart (which I discuss in more detail below).
In the second part, the stakeholders and (selected) users and customers join the meeting. I find that as the person in charge of the product, you are often best suited to present the product increment: You are likely to better understand how a user would interact with the product and use the new functionality compared to a development team member. Additionally, you probably know better how to collect feedback that helps you learn if the right user experience and functionality are offered.
Splitting the meeting allows you to have a focused conversation with the development team, determine the development progress, and address potential issues before users, customers, and stakeholders join you. This is particularly helpful when you didn’t have the chance to interact with the team during the sprint. Make sure, though, that the development team members attend the entire meeting and are present in the second part. Hearing feedback directly from users, customers, and stakeholders is invaluable.
Encourage Feedback but Don’t Say Yes to Every Idea
More than one sprint review meeting I’ve attended was quickly over: A development team member demoed the functionality to confused-looking attendees with the product owner in the background. Afterwards, the Scrum Master asked if there were any questions or feedback, but the individuals just looked at each other, a few said “nice job” and “looks good”, and left. The valuable feedback gathered in this meeting was zero.
To avoid such a meeting, encourage people to actively participate and share their views, ideas, and concerns. Use open-ended questions, for example, “What do you think about the improvements we made to the registration feature?” Try to understand why somebody likes or dislikes something. Receiving feedback such as “looks great” may feel good, but it does not offer any new insights. Why does the person like the feature? And is there anything that could be improved further?
Additionally, do your best to attentively listen to the attendees and appreciate their views. But don’t be afraid to say no to ideas and requests if they are not helpful and realistic. Use the current product goal together with the product strategy to decide if you should take on a request. If you are unsure, use the next sprint to test if the idea or request would be beneficial for the users.
Don’t Rush Important Decisions
Sometimes, it’s possible to make the right decisions in the sprint review meeting and even adjust the product backlog. But often—particularly if the feedback has a bigger impact—you need more time to analyse it, draw the right conclusions, and make the right backlog changes. Additionally, you may not have the relevant data available in the sprint review meeting if you decide to collect user and stakeholder feedback separately, as discussed above.
You should therefore consider separating collecting feedback and data from analysing and actioning it. You may choose, for instance, to have a short, focused product backlog workshop prior to the next sprint planning meeting that offers you the opportunity to objectively evaluate the feedback and update the backlog together with the development team; or you may want to schedule a product backlog session in the next sprint once you have collected enough user data with the help of an analytics tool, as I explain in my article When should Product Backlog Refinement Take Place?
Visualise the Development Progress
Imagine that all feedback suggests that people are going to love your product. But if you are way behind schedule and over budget, your product may not become a success. It is therefore important that you regularly determine the progress made.
The sprint review meeting is a great opportunity to do this, as you should now be able to tell which items were completed using the definition of done. This set of criteria states when a product backlog item is considered to be finished. For example, it has been unit, integration, and system tested, and has achieved a test coverage target; the code complies with the agreed coding standards, and its complexity is sufficiently low; the appropriate user and development documentation is available.
As the person in charge of the product, it’s up to you to determine which items are done and which ones aren’t. Any item that does not fulfil all the done criteria is deemed not done and put back into the product backlog. Offer clear and constructive feedback to the development team, and never accept partially finished or defective software. If necessary, use the sprint retrospective to identify improvement measures and help the team complete all items that are worked on in a sprint.
Once understand how many items are done, you can use a tool like the release burndown chart to determine how far you have come, and if you are on track to meet the product goal as expected. This information is also valuable for the key stakeholders who attend the meeting so that they can adjust their work, if required. What’s more, discussing the progress puts the current sprint into context and connects it with the previous ones.
The burndown chart, a standard artefact in Scrum for tracking the development progress and anticipating how the development effort is likely to unfold. The chart shows how the effort in the product backlog develops from sprint to sprint—the idea being that the effort is reduced or “burned down” over time, as shown by the solid line in Figure 1.
The dotted line in Figure 1 represents the forecasted development progress. Fortunately, everything is on track based on the sprints carried out so far.
No matter which tool you use, make sure that you understand how fast you are progressing and what adjustments you need to make—be it adding a UX designer to the team, partially meeting the product goal, and possibly delivering fewer features than planned, or pushing out the release date.
Consider Collecting User and Stakeholder Feedback Separately
The original idea of the sprint review meeting is to bring the right people together and collect feedback from everyone at the same time. If that works for you, then that’s great. Often, though, I find that it is more helpful to gather feedback from users and internal stakeholders separately. Why? Both groups tend to have different perspectives and needs.
Testing product increments with users allows you to understand if the product is likely to do a great job for its target group, if it offers the right user experience and the right features. Discussing increments with stakeholders helps you understand if the product can be effectively offered, if it can be operated, marketed, sold, and supported by your organisation (depending on its type).
What’s more, it’s often best to collect feedback from users and stakeholders using different techniques: Demoing the product increment to end users only makes sense when very little functionality has been implemented. Otherwise, it tends to be more helpful to observe or measure how people actually use the product, employing, for example, usability tests and early releases, as I explain in more detail in my article How to Choose the Right Product Validation Technique in Scrum.
As these techniques usually require more time than the sprint review meeting offers—it may take several days to collect the relevant data after you’ve released a product increment—this naturally separates collecting the user data from gathering stakeholder feedback. Consequently, you would invite stakeholders to the review meeting, but not necessarily users and customers.
Bear in mind that users trump stakeholders: If all your stakeholders are happy, but the product doesn’t do a good job for the users, it’s unlikely to become a success.
Post a Comment or Ask a Question
6 Comments
Hi Roman, I have faced a scenario where a few ‘difficult’ characters in the Scrum team want to use the sprint review meeting as means of ‘confirming’ whether a particular user story, spikes etc are indeed done or not. This has now meant that we do not close the sprint until after the sprint review has taken place. Also, all the various user stories, spikes, enablers etc all have acceptance criteria documented and understood, yet we have now resorted to ‘agreeing’ whether an item can indeed be marked as done during the sprint review meeting. I have pushed back but to no avail and wanted to get your perspective on this please.
Thanks for sharing your question Javed. In Scrum, the product owner determines if an item is done using the definition of done and, if user stories are used, the stories’ acceptance criteria. If other Scrum team members repeatedly disagree with the product owner, then investigate in the next retrospective why this is. Is the definition of done not clear? Is it too weak or too ambitious? Or is the role of the product owner not properly understood? Your Scrum Master should be able to help you with this issue. Good luck!
Many thanks for coming back Roman. The only clarification I’m looking for is it good practice to hold the sprint review and use that forum to move stories to done. My experience tells me to use DoD and ACs to close stories and then demo what has been completed in the sprint review; not to use it as a forum to close stories and only end the sprint after the sprint review meeting.
You’re welcome Javed. The sprint review meeting provides a good opportunity to determine what is done and how much progress has been achieved–as well as to collect feedback from (selected) users and customers and the stakeholders. The product owner may review backlog items just in time as they get done in the sprint if this works for the Scrum team. The development team should demo only items that the members believe to be done. Hope this helps!
One project is on a two week sprint cycle. At the end of the sprint, they demo to the product owner. 4 days later they demo to a set of stakeholders the same thing… in the meantime the scrum team have started on the next sprint without adding the new requests to the product backlog / reprioritizing it. The stakeholders then have additional requirements that may or may not find their way into the product backlog until the 3rd sprint assuming the product backlog has now been groomed.
Question: What the team is doing – demo to PO and then 4 days later demo to other stakeholders while the Scrum team is on its second sprint – is this a good practice and should it continue? And how does one account for the lag in documenting the user stories.
Thanks for sharing your question, Manisha. How important it is to immediately validate a product increment and the related product decisions, depends on the amount of innovation and uncertainty present. When you build a brand-new product or create a major update of an existing one, having a lag between finishing the product increment and receiving feedback/validation is undesirable, at least in the first few sprints of the development effort, as this might result in the dev team heading down a wrong path. But once you have addressed the major risks in your product backlog, such a delay is usually not a big issue. (I discuss different options to gather feedback and update the backlog in the article “When Should Product Backlog Grooming Take Place“.)
Having said that, I would be interested to understand what it would take to have the stakeholder feedback in the sprint review meeting. It would be helpful for the dev team members to witness the stakeholders’ reaction to the new increment and directly hear their feedback. Additionally, don’t forget to regularly collect user feedback and data. The users of your product ultimately determine its success, not the stakeholders.
Hope this helps!