Scrum, day 22: a small improvement experiment

Today's objective: describe an improvement as a testable expectation, including its scope, evidence, cost, and review point.

Read alongside: Mastering Professional Scrum, chapter 1, focusing on the continuous improvement process. Read with one change from your own work in mind. Allow roughly five minutes for this companion, plus the assigned book section.

Give the proposed change a fair test

The fictional ShiftSwap team has selected a problem: consequential differences between recent and older request records are sometimes discovered after implementation has begun. Its proposed improvement is to discuss a few representative examples earlier.

That sounds reasonable. It is also incomplete. If the team merely adds “discuss more examples” to its habits, nobody will know when the change has been tried, whether it helped, or when to reconsider it. An improvement can become another permanent meeting without ever receiving a fair evaluation.

The team needs a small agreement about what it expects to learn. The purpose is to make the change inspectable while keeping the effort proportionate to the problem.

State the expected mechanism

The Developers propose this expectation:

For the next two changes involving existing requests, discussing a recent record, an older record, and an exceptional case before implementation should reveal important behavior differences while the plan is still easy to adjust.

The sentence says more than “better refinement will improve delivery.” It explains how the change might help: an early conversation could expose differences before they become expensive assumptions. The examples are a means of creating that conversation.

It also limits where the experiment applies. A purely visual correction to an unrelated page would not provide a useful test of this particular expectation. The team should choose relevant opportunities rather than count any completed task as evidence that the experiment worked.

Capture a modest starting point

Nobody needs to spend a week reconstructing perfect historical data. For the two recent changes that prompted concern, the team records the important question that surfaced late, roughly when it surfaced, and the decisions revisited as a result.

For the decline explanation, the question was whether older records held text appropriate for employees. Interface and service decisions had already assumed that they did. This concrete account is more useful than a retrospective estimate that the item was “thirty percent inefficient.”

The starting point is incomplete, and the team labels it accordingly. Its purpose is to prevent memory from changing to fit the desired conclusion. If future comparisons involve very different work, that difference should remain visible too.

Decide what to notice before seeing the result

For each trial change, someone keeps a few short notes: who joined the example discussion, which differences it revealed, which decisions changed, and what consequential questions still emerged later. The team also records the time spent preparing and discussing the examples.

Useful evidence might be a previously overlooked cancellation state identified before implementation. Evidence against the proposed mechanism might be that all three examples were discussed but the people making the subsequent decision never saw the conclusions.

That second result would not mean “examples never work.” It would suggest that information transfer remains a problem. An experiment should help distinguish explanations, including explanations that require revising the original proposal.

Make room for the cost

Early discussion takes attention. A colleague may need to locate suitable records, remove personal details, or ask a manager about an unfamiliar state. The team should not describe the practice as free merely because the work happens before implementation.

For this trial, it agrees to start with a short conversation using existing, safe examples. If preparing them requires substantial investigation, the Developers will make that work visible and discuss its effect on their plan. They will avoid expanding the exercise into an exhaustive catalogue of every possible record.

The useful question is whether the information changes decisions enough to justify the effort. A practice that prevents one small correction while delaying several other items may need a narrower application.

Inspect mixed evidence honestly

Imagine the first trial uncovers a cancelled request with an outdated message. The Developers adjust their approach before writing the relevant behavior. The second trial reveals nothing new, takes longer to prepare, and still encounters a permission issue later.

These invented results do not support a sweeping claim about improved productivity. They suggest that the examples helped with one kind of uncertainty and did not address another. Two opportunities provide limited evidence, especially when the changes differ in complexity.

The team might keep the conversation for changes to request state, shorten preparation, and explicitly include access questions when they matter. It might also continue observing before making a firm decision. “Promising, with a narrower use” is a legitimate result.

Choose a review point and possible decisions

ShiftSwap will inspect the trial after the two relevant changes, using the next Retrospective if the timing fits. If the practice obstructs work or exposes a serious concern sooner, the team will discuss it immediately rather than wait for a ceremonial end date.

Before starting, it identifies three possible decisions: keep the practice, modify it, or stop it. Keeping it means the benefit appears worth its cost in the observed circumstances. Modifying it means the evidence suggests a better version. Stopping it means the expectation was unsupported or a different problem deserves attention.

Radu can help the conversation remain focused, but the experiment belongs to the people changing their work. It should not become a private Scrum Master project that everyone else performs to satisfy a report.

Keep the change small enough to interpret

Suppose the team simultaneously changes its backlog tool, Sprint length, meeting schedule, and example practice. A smoother Sprint afterward would be encouraging, but attributing the improvement to the example conversation would be difficult.

Sometimes several changes are necessary together. In that case, acknowledge that the experiment concerns the combination. Here, the team can keep the trial narrower and record other changes that may influence the result. It is trying to make a sensible local decision, without pretending to conduct a perfectly controlled laboratory study.

Draft your own experiment

Write five sentences: the problem observed, the change proposed, why it might help, what evidence you will collect, and when you will decide what to do next. Add one possible unwanted effect and one observation that would challenge your expectation.

Keep the record short enough that someone can use it during ordinary work. If collecting the evidence becomes more complicated than the improvement itself, simplify the question or the observation method.

Reflection: would you be willing to stop this practice if the result disappointed you? If the answer is no, identify whether you are testing a change or seeking reassurance for a decision already made.

Next, day 23 examines the shared purpose that makes local improvements worth pursuing.