Blog

In several recent field notes, I have explored how quality begins before testing. Requirements may form part of the reference model against which a product is assessed. Acceptance criteria can make expectations explicit. Test cases often reveal details that were missing from the original product definition. Definition of Ready and Definition of Done can connect intention, implementation and assessment. Product Ownership has a much more important quality role than simply maintaining a backlog.

Together, these ideas suggest a stronger quality sequence:

1. The Product Owner defines the intention.
2. The story makes it concrete.
3. The team implements it.
4. Testing provides evidence.
5. The Definition of Done confirms completion.

That would already be a significant improvement over treating quality as something testers add near the end. But I am beginning to think it is still not enough.

A well-delivered story is not necessarily a good product

A story can be clearly formulated. Its acceptance criteria can be precise. The implementation can be technically sound. The tests can pass. Every condition in the Definition of Done can be satisfied. And the product can still be poor.

The story may solve the wrong problem. It may work in isolation but create inconsistency elsewhere. It may support an obvious scenario while failing across a complete user journey. It may introduce complexity that weakens maintainability. It may function at current scale but not at the scale required by future customers.

This does not necessarily mean that the story, testing or Definition of Done failed. It may mean that we asked them to carry more of the quality model than they can support.

Stories are delivery instruments

Stories help us divide work into manageable pieces. They support conversation, prioritization, implementation and feedback. But users do not experience a collection of stories. They experience a complete product.

Usability emerges across journeys. Performance appears under realistic load. Resilience becomes visible when dependencies fail. Maintainability is affected by accumulated design decisions. Security and data integrity cut across features and organisational boundaries. These qualities cannot always be assessed meaningfully within a single story.

This creates an important distinction:

Story-level acceptance is not the same as product-level assessment.

The first asks whether an increment meets its expectations. The second asks whether the product as a whole is good enough for its purpose, stakeholders, risks and context. We need both.

Definition of Done is not a definition of quality

The Definition of Done is an important mechanism. It can establish shared expectations for code review, testing, documentation, deployment and other necessary activities. But it may be given more meaning than it can carry.

A Definition of Done usually tells us whether the team has completed the expected work correctly. It does not necessarily tell us whether:

• the product solves the right problem;
• the complete user experience is coherent;
• the architecture remains sustainable;
• operational risks are acceptable;
• the intended outcome is being achieved.

A strong Definition of Done contributes evidence. It is not, by itself, a complete reference model for product quality.

Quality extends from intent to outcome

Perhaps the deeper mistake is that we have confused a delivery process with a quality model. The delivery process may look like this:

Product Owner → story → implementation → testing → done

A broader quality perspective looks more like this:

Intent → product definition → implementation → integrated product → operation → outcomes → learning

This wider view introduces questions at several levels:

• Did we understand the problem and intended outcome?
• Did we translate that understanding into clear and assessable expectations?
• Did we build the solution correctly and sustainably?
• Does the integrated product exhibit the quality characteristics required in realistic use?
• Does it create the intended value after release?
• What have we learned that should change the product—or our understanding of quality?

Testing contributes to this system, but it does not contain the whole system. Product Ownership contributes to it, but one role cannot represent every relevant quality perspective. Stories contribute to it, but they cannot describe every emergent property of the complete product. The Definition of Done contributes to it, but process completion does not prove fitness for purpose.

This is a leadership question

Teams can improve stories, acceptance criteria, refinement, testing and their Definition of Done. But a holistic quality approach extends beyond the authority of a single team.

It may require alignment between strategy, product management, engineering, architecture, operations, security, support and other stakeholders. It requires explicit choices about quality characteristics, risks, evidence and decision-making.

Only leadership can create the conditions in which these perspectives form one coherent quality system. The leadership question is therefore not only:

Are teams following the expected quality practices?

It is also:

Does the organisation have a sufficiently complete model of quality in the first place?

A broader working hypothesis

Quality begins before testing. But it also extends beyond testing, beyond individual stories and beyond the Definition of Done.

The familiar delivery sequence remains valuable. It helps teams convert product intentions into tested increments. We should not abandon it. We should stop mistaking it for the complete quality model. A collection of accepted stories is not automatically an acceptable product.

A collection of accepted stories is not automatically an acceptable product. Product quality cannot be secured at the end of delivery.

It must be governed from intent to outcome.

The Product Owner role was intended to be much more than backlog management. Product Owners are expected to think about the future of the product, align stakeholders, maximize value and make difficult choices about direction, priorities and trade-offs.

Yet in many organizations, the role seems to have drifted downstream. Product Owners spend much of their time managing backlogs, writing user stories, refining requirements and preparing sprint planning sessions. These activities matter, but they can consume so much attention that the strategic purpose of the role begins to disappear.

Recently, while exploring the relationship between assessment models, reference models and requirements, I started to wonder whether we have pulled the Product Owner too far downstream. Requirements do not emerge in isolation.

Before requirements come business goals, product vision, intended outcomes, stakeholder expectations, success criteria, constraints and quality expectations. Someone has to bring these perspectives together and create a shared understanding of what success looks like. That responsibility feels remarkably close to the real purpose of Product Ownership.

Viewed this way, user stories and backlog items are not the primary responsibility of the Product Owner. They are outputs.

The deeper responsibility lies upstream: clarifying purpose, aligning stakeholders, making priorities explicit, defining intended outcomes and shaping the criteria by which the product will eventually be judged.

This also sheds new light on acceptance. The Product Owner should not act as a final tester or as a ceremonial gate through which completed work must pass. An Increment is not Done because the Product Owner approves it. It is Done because it meets the team’s Definition of Done.

But Done is not the same as fit for release, rollout or wider use.

A technically complete Increment may still leave important product questions unresolved:

• Does it produce the intended outcome?
• Is the remaining risk acceptable?
• Is the quality adequate for the next stage of use?
• Are important stakeholder concerns sufficiently addressed?
• Does the available evidence justify release or further investment?

These are product judgments. If the Product Owner helps define what success means, they should also play a central role in judging whether the accumulated evidence supports those decisions.

That does not mean making every judgment alone. Developers, testers, architects, operations and other specialists contribute evidence and expertise. The Product Owner’s responsibility is to bring that evidence back to purpose, value, priorities and business risk.

Perhaps many organizations have not weakened the role intentionally. Perhaps the Product Owner has simply been absorbed by the growing volume of downstream work. When that happens, the backlog remains managed, but the product may lose the person who holds together its direction, stakeholder expectations and definition of success.

Improving Product Ownership may therefore require more than better refinement or better-written user stories. It may require returning the Product Owner to where the role creates the most value: upstream, closer to purpose, outcomes and the decisions that determine what good should mean.

Most agile teams have heard of Definition of Ready and Definition of Done. Yet in many organizations, both have become little more than checklists. Teams debate which items should be included, whether they are mandatory and how strictly they should be applied. Surprisingly little attention is given to why these mechanisms exist in the first place.

Recently, while exploring testing as a form of assessment, I started to view DoR and DoD differently. An assessment needs two things: a reference model that defines what good looks like, and evidence that allows us to judge whether those expectations have been met. Viewed through this lens, DoR and DoD appear to serve complementary purposes.

Definition of Ready is not merely a planning tool. It helps determine whether the reference model is sufficiently mature for work to begin responsibly. Are the requirements clear enough? Are the intended outcomes understood? Have important business rules, assumptions, constraints and quality expectations been made visible? Is there enough shared understanding to make implementation a reasonable next step?

This does not mean that everything must be known in advance. A rigid DoR can become a gate that delays learning and encourages teams to over-specify work before they have had a chance to explore it. Readiness should not mean certainty. It means that the team understands what good looks like well enough to begin, while recognizing what still needs to be learned.

Definition of Done serves a related but different purpose. It describes the state that work must reach before it can be considered complete. That may include conditions the product must satisfy, such as updated documentation or the absence of critical defects. It may also include practices that strengthen confidence in the conclusion, such as code review, automated testing, security analysis or demonstration.

These are not merely procedural tasks. They either contribute to the required state of the product or generate evidence that the required state has been achieved.

DoD therefore helps answer two questions:

• Has the work reached the expected level of quality?
• And is there sufficient evidence to support that conclusion?

This creates an interesting symmetry:

DoR protects the quality of the criteria. DoD protects the quality of the conclusion.

Without a meaningful DoR, teams risk building against an incomplete or poorly shared understanding of success.

Without a meaningful DoD, teams risk declaring success without reaching the required state or producing enough evidence to justify the claim.

Perhaps the real purpose of DoR and DoD is therefore not process compliance. They safeguard the two foundations of assessment: a sufficiently trustworthy reference model and a sufficiently trustworthy conclusion.

Seen this way, they are not administrative artifacts. They are quality mechanisms.

During quality assessments, I have often found myself making recommendations that seem to sit outside the testing domain. Instead of focusing only on test automation, test processes or testing skills, I sometimes recommend strengthening Product Ownership, business analysis and requirements capabilities.

At first glance, this may appear indirect. If the goal is better testing, should we not improve testing? Recently, while exploring testing as a form of assessment, I started to see a clearer explanation.

An assessment depends on a reference model: a structured description of what the assessed object is expected to be.

In product development, an important part of that reference model is expressed through requirements, acceptance criteria, business rules, quality expectations and other descriptions of what success looks like.

Requirements are not the complete reference model for system quality. Important expectations may also come from architecture, operational needs, regulation, user research, risk analysis and the intended business context.

But requirements work is one of the main ways in which those expectations are made explicit and shared. When that part of the reference model is incomplete, testing becomes harder.

Testers spend time clarifying expectations, identifying missing scenarios, discovering hidden assumptions and resolving ambiguities. Test design becomes partly an exercise in completing the model against which the product will later be assessed.

This creates an important distinction. Some testing problems originate within testing. The team may lack appropriate skills, techniques, environments, tools or automation.

Other testing problems are inherited from upstream. The team cannot assess the product efficiently because the expected outcome was never made sufficiently clear.

Improving the test process alone will not remove that second kind of weakness. This is why a quality assessment may need to look beyond the formal boundaries of testing.

The Product Owner and business analysts do more than produce user stories. They help clarify the purpose of a change, align stakeholder expectations, make business rules visible and determine what success should mean in a particular context.

They also help ensure that quality concerns are considered before implementation begins:

• What level of performance will be sufficient?
• Which failures would be unacceptable?
• Which users and operating conditions must be supported?
• Which assumptions still need to be validated?
• What trade-offs are acceptable?
• What evidence will be needed before the product can be judged ready?

These questions strengthen the reference model on which developers and testers both depend.

The effect is broader than better testability. Developers gain a clearer basis for implementation. Testers gain a clearer basis for evidence collection. Product Owners gain a better basis for prioritisation and acceptance. Stakeholders gain a more explicit understanding of the risks and trade-offs involved.

Seen this way, recommending stronger Product Ownership or requirements capabilities is not an excursion beyond the scope of a quality assessment. It may be a response to the actual cause of the observed testing weakness.

An assessment should not be constrained by the label attached to the problem. It should follow the chain of dependency far enough to identify what needs to improve. Sometimes that leads to test practices. Sometimes it leads upstream.

Perhaps some of the most effective ways to improve testing are not found within the testing discipline itself. They are found in the capability to define, align and communicate what success looks like before the assessment begins.