Field Notes from Quality & Delivery Transformations

Blog

  • Field Note: Who Owns the Product Reference Model?

    The idea of a product reference model creates an immediate question:

    Who owns it?

    If the model describes what the product is intended to achieve, the conditions under which it must operate, the quality expected of it and the evidence needed to judge it, ownership cannot remain implicit. Many people may contribute to the model. Someone must still own whether it forms a coherent whole. That responsibility naturally belongs to Product Ownership.

    Ownership Does Not Mean Writing Everything

    A Product Owner cannot personally define every part of the product reference model.

    Architects contribute technical constraints. Security specialists contribute controls and risks. Developers contribute feasibility and implementation knowledge. Testers contribute evidence needs and ambiguity detection. Operations contributes knowledge of production reality. Customers and users contribute expectations and desired outcomes.

    Leadership also contributes by setting strategic direction, investment boundaries, customer commitments and acceptable levels of risk. All of these inputs may be valid. But a collection of valid inputs is not yet a coherent product model. Someone must ensure that they fit together.

    Leadership Sets the Boundaries

    Leadership defines the space in which the product is expected to operate. It determines strategic direction, business outcomes, investment limits, policy constraints and acceptable risk.

    These decisions shape the product, but they do not automatically translate into something that can guide development and assessment. That translation belongs closer to the product. Leadership defines the boundaries. Product Ownership determines what the product must become within those boundaries.

    Leadership also has to provide the conditions needed to meet its expectations. A product cannot be expected to be secure, resilient, scalable and compliant without sufficient time, competence, authority and investment. Constraints without enabling conditions become governance theatre.

    Product Ownership Owns Integrity

    The Product Owner does not need to be the greatest expert in every dimension of the product. The Product Owner does need to ensure that the different dimensions form one coherent whole.

    That means connecting business objectives to product outcomes, making constraints visible, resolving important trade-offs, recognizing risks and ensuring that quality expectations and evidence needs are explicit. The central responsibility is therefore not authorship. It is integrity. The Product Owner owns the integrity of the product reference model.

    Specialists remain responsible for the validity of their contributions. The architect must provide credible architectural guidance. The security specialist must provide credible security expertise. The tester must identify meaningful evidence. The developer must explain technical implications.

    But those contributions should not remain separate professional perspectives. They must still describe one intended product.

    The Product Owner as Custodian

    The product reference model is not finished when the initial requirements are written. It changes as the organization learns. Operational incidents may reveal false assumptions. Larger customers may expose scalability limitations. Regulation may change. User behavior may challenge expectations. Business priorities may shift.

    Someone must ensure that the model changes with the product and its context. This is why the Product Owner is best understood as the custodian of the product reference model. A custodian does not create every element. A custodian ensures that the whole remains coherent, current and useful.

    Leadership defines direction, constraints and acceptable risk. Specialists ensure that their contributions are credible. Product Ownership ensures that it all still describes one product.

  • Field Note: A Product Reference Model Must Feed Development

    A reference model can easily become something we use after the fact. We define what good looks like, compare reality against it and identify gaps. That is useful for assessment.

    But if the model only becomes relevant when the product is being evaluated, it arrives too late. The same understanding that supports judgement should also support creation.

    A product reference model must not only help us assess the product. It must also help us build it.

    The danger of an assessment-only model

    Suppose the product reference model says that the product must be scalable, auditable, secure, configurable and easy to integrate. If those expectations are only checked near release, they function as acceptance criteria for work that has already been designed and implemented.

    By then, the architecture may be difficult to change. The wrong assumptions may already be embedded in the solution. The team may discover that important evidence cannot be produced because observability, testability or traceability were never designed in.

    A late reference model can reveal gaps. It cannot prevent them.

    The model should influence earlier decisions

    If the reference model expresses the intended product, then it should influence:

    • product strategy;
    • roadmap decisions;
    • architecture;
    • requirements and stories;
    • acceptance criteria;
    • quality scenarios;
    • test design;
    • observability;
    • release preparation;
    • operational feedback.

    It should help people ask better questions before implementation begins.

    Which users and stakeholders are affected?

    Which qualities are important in this context?

    Which assumptions are we making?

    What could make the product unsuitable even if the feature works?

    What evidence will we need later to judge whether the intended outcome has been achieved?

    These are not only assessment questions. They are development questions.

    Requirements are translations of the model

    Requirements, stories and acceptance criteria remain necessary. But they should be understood as selected translations of the wider product reference model into engineering work. They do not contain the complete product understanding. A story may describe a behaviour.

    The reference model provides the reason, context, quality expectations, constraints, risks and intended outcome behind it. That wider understanding helps engineering make better decisions when the requirement is incomplete or when several technically valid solutions exist. It also reduces the risk that testers have to reconstruct missing product intent after implementation.

    Evidence should be designed in

    If the organisation knows what it will need to assess, it can design the necessary evidence into the product and delivery system. Performance expectations can shape load testing and monitoring. Reliability expectations can shape resilience engineering and incident measures. Usability expectations can shape research and usage analytics. Business outcomes can shape product instrumentation.

    The reference model therefore influences not only what is built, but how the organisation will learn whether it works.

    A shared object of reasoning

    The model should not belong exclusively to Product Ownership, architecture, QA or testing. It should become a shared object through which business and engineering reason about the same product.

    Product Ownership helps maintain the connection to intent and outcomes. Engineering uses the model to guide implementation and technical trade-offs. Testing and assessment use it to determine what evidence is needed. Operations and support use real-world experience to challenge and update it.

    The model moves in both directions: intent guides development
    and operational evidence changes future intent.

    More than a ruler

    A ruler tells us whether something meets a standard after it has been produced. A useful product reference model should do more. It should guide decisions while the product is being shaped, help generate the evidence needed for assessment and evolve when reality challenges earlier assumptions.

    A product reference model that only measures the product is incomplete. Its real value appears when it helps the organisation create, assess and evolve the right product.

  • Field Note: What Belongs in a Product Reference Model?

    If quality is assessed, something must provide the basis for judgement. We need to know what the product is intended to achieve, who it is intended to serve and what “good enough” means in its context.

    Requirements provide part of that reference. But they rarely provide the whole of it. Requirements tend to describe selected behaviour or changes that engineering should implement. User stories, acceptance criteria and specifications help translate product decisions into development work.

    A product reference model would need to preserve a wider perspective. It should help answer not only:

    Did we build what was specified?

    But also:

    Are we building the right product, with the right qualities, for the right purpose?

    More than requirements

    The model would probably begin with the product’s purpose. Why does the product exist? What problem should it solve? What business or customer outcome is expected?

    It would also need to identify the customers, users and other stakeholders whose needs and constraints matter.

    From there, it should describe the capabilities the product must provide and the quality characteristics that are important in its context. For one product, performance and scalability may be decisive. For another, usability, security, auditability, accessibility, data quality or ease of integration may matter more.

    The model should also include what is uncertain. Product development is based on assumptions:

    • that a customer need exists;
    • that a proposed capability will address it;
    • that users will adopt the solution;
    • that the product can operate under expected conditions;
    • that the investment will create value.

    These assumptions are part of the product understanding and should not disappear when work enters the backlog.

    A possible structure

    A product reference model might therefore contain:

    • business intent and strategic context;
    • customers, users and stakeholders;
    • intended outcomes and value;
    • required product capabilities;
    • relevant quality characteristics;
    • constraints and obligations;
    • assumptions, risks and uncertainties;
    • dependencies and relationships with the wider product or ecosystem;
    • operational conditions;
    • evidence required to judge whether expectations have been met;
    • learning from actual use.

    This should not necessarily become one large document. Much of it may already exist across roadmaps, business cases, customer research, architecture, requirements, quality scenarios and operational objectives.

    The problem is often not that the information is completely absent. It is that it is fragmented, implicit or disconnected.

    A living model

    The product reference model cannot remain fixed. Business priorities change. New customer groups are targeted. Products scale. Regulation develops. Operational evidence challenges earlier assumptions.

    The model must therefore evolve as the product and its context evolve. It should guide development before release and support judgement after release.

    That makes it more than an assessment artefact. It becomes a shared model for creating, evaluating and learning about the product.

    We do not yet know exactly what form such a model should take. But the need is becoming clearer:

    Requirements describe parts of what should be built. A product reference model preserves the wider understanding needed to build, assess and evolve the right product.

    The product reference model is not another source alongside the existing sources. It is the structure that makes those sources function as one coherent model of the product.

  • Field Note: What Is the Business Value of Quality?

    The business case for quality is often reduced to defects. Better quality means fewer defects. Fewer defects mean less rework, fewer incidents and lower support costs.

    That is true, but incomplete. A product can contain few defects and still solve the wrong problem. It can function correctly while being difficult to use, expensive to operate or impossible to scale. It can be delivered quickly and reliably while creating little customer or business value. If quality is broader than testing and defect prevention, its business value must also be broader.

    DORA measures an important part of the story

    DORA metrics help organisations understand how effectively engineering delivers software changes. They provide evidence about speed, stability, recovery and rework within the delivery system. This matters. An organisation that cannot change its product quickly and safely will struggle to respond to customers, risks and opportunities.

    But delivery performance is not the same as product quality. DORA can show how well we deliver changes. It does not tell us whether we selected the right changes, whether the product is good enough for its context or whether it achieves the intended outcome.

    The broader question is:

    How effectively can we turn business intent into a valuable, fit-for-purpose product—and continue improving it sustainably?

    Quality reduces the cost of being wrong

    When product intent, stakeholder needs and assumptions remain unclear, engineering may deliver exactly what was requested without creating meaningful value. The result may be:

    • rarely used functionality;
    • solutions to misunderstood problems;
    • unnecessary complexity;
    • rework caused by late clarification;
    • delayed learning about customers.

    A broader quality approach makes product reasoning explicit and challenges assumptions earlier. Its value is not only fewer implementation defects. It is less wasted product investment.

    Quality reduces the cost of failure and operation

    Some weaknesses become incidents, security exposure, data errors or customer complaints. Others become permanent operating costs:

    • recurring support contacts;
    • manual workarounds;
    • unstable services;
    • repeated defect correction;
    • difficult recovery;
    • complicated maintenance.

    These costs are often distributed across engineering, operations, support and the business. They may never appear together as the cost of poor product quality. A lifecycle perspective connects them to the product decisions that created them. Its value includes lower risk, lower cost to serve and fewer expensive surprises.

    Quality protects the ability to change

    Quality also affects how easily the product can be improved tomorrow. Weak maintainability, poor testability, unclear product intent and accumulated technical debt make every future change slower and riskier. The organisation then spends more effort preserving existing behaviour and less effort creating new value.

    Quality engineering protects the ability to change through:

    • shorter lead times;
    • safer releases;
    • less rework;
    • lower development cost;
    • less dependence on individuals;
    • greater responsiveness to markets and customers.

    This is where DORA becomes part of the wider quality story. It helps show whether engineering can change the product quickly and sustainably.

    Quality increases the likelihood of useful outcomes

    Quality is not only about reducing cost and risk. A reliable, understandable, responsive and context-appropriate product is more likely to achieve:

    • adoption;
    • customer satisfaction;
    • trust;
    • retention;
    • productivity improvements;
    • business benefits;
    • access to more demanding markets.

    The causal chain is rarely simple, but it can be made visible:

    better product understanding

    → better engineering decisions

    → stronger product behaviour

    → better customer experience

    → improved business outcomes

    The aim is not to claim perfect attribution. It is to make the relationship explicit enough to support better decisions.

    Quality also has a cost

    A broader quality approach requires investment in:

    • understanding stakeholders and outcomes;
    • defining relevant quality expectations;
    • improving testability and observability;
    • collecting engineering and operational evidence;
    • interpreting that evidence;
    • involving specialists;
    • improving products and ways of working.

    There is also a risk of bureaucracy. If quality modelling becomes a documentation exercise, it can slow delivery without improving judgement. The depth of assessment should therefore reflect the product’s value, risk, complexity and cost of failure. The goal is not maximum quality activity. It is sufficient quality understanding to make responsible business decisions.

    A broader business case

    The business value of quality can be understood through four effects:

    • reducing the cost of being wrong;
    • reducing the cost of failure and operation;
    • reducing the cost of future change;
    • increasing the likelihood of valuable outcomes.

    The cost lies in the modelling, measurement, assessment and improvement needed to achieve those effects.

    This should not be compressed into one universal quality ROI figure.

    Each product needs an explicit quality argument:

    Which business risks and outcomes matter?

    Which product and engineering qualities influence them?

    What evidence shows whether they are improving?

    Is the expected value worth the required investment?

    DORA provides an important view of engineering delivery.

    A broader product-quality model connects that view to customer value, operational cost, business risk and product outcomes. That may be the real business case for quality:

    Quality reduces the cost of being wrong, the cost of failure and the cost of future change—while increasing the likelihood that the product achieves its intended purpose.

  • Field Note: An Assessment Is a Snapshot. Improvement Requires Reassessment.

    I have usually thought of an assessment as a one-time intervention. We define a reference model, collect evidence, judge the current state and identify improvements. The result is a snapshot. That is useful for diagnosis.

    But it is not enough when the purpose is improvement. Once change begins, the subject of the assessment starts to move. Practices change. Responsibilities shift. New evidence appears. Some improvements work, others do not. New weaknesses become visible. The original assessment gradually becomes a description of a state that no longer exists.

    An assessment may be a snapshot. Assessment should be a recurring capability.

    The first assessment establishes a baseline

    A thorough initial assessment remains valuable. It clarifies:

    • what is being assessed;
    • what good looks like;
    • the current state;
    • important strengths, weaknesses and risks;
    • priorities for improvement.

    Without that baseline, improvement can become a collection of disconnected actions. But the baseline is only the beginning.

    Improvement creates new evidence

    Every improvement action is also a hypothesis. We expect that changing a process, role, tool, structure or behaviour will lead to a better result.

    That expectation should be tested. Did the change actually happen? Did behaviour improve, or only documentation? Did the intended result appear? Did the improvement create new problems elsewhere?

    Without reassessment, these questions are often answered through impressions or completion reports.

    But completing an improvement is not the same as achieving improvement. Reassessment brings the organisation back to evidence.

    Reassessment does not mean repeating everything

    Continuous assessment does not require a full formal assessment every sprint or every month. That would quickly become bureaucratic. Reassessment can happen at different depths. A lightweight review may examine recent evidence and a few critical indicators. A periodic deeper assessment may revisit the complete reference model. A major change, failure or shift in context may trigger an earlier reassessment.

    The principle is simple:

    Reassess often enough to learn whether improvement is actually happening.

    The reference model may also change

    Reassessment is not merely updating a score against a fixed model. As understanding improves, the definition of good may also evolve. Some expectations may prove irrelevant. New capabilities may become important. The organisation may discover that its original target state was incomplete or misguided.

    Reassessment should therefore ask two questions:

    Are we moving closer to what we defined as good?

    And:

    Is our definition of good still the right one?

    From report to improvement loop

    A static assessment often ends with findings and recommendations.

    A continuous assessment process closes the loop:

    reference model

    → evidence

    → judgement

    → decision

    → improvement

    → new evidence

    → reassessment

    The assessment should therefore identify not only what needs to improve, but also:

    • what evidence will show progress;
    • when reassessment will occur;
    • what should trigger an earlier review;
    • which assumptions remain uncertain.

    Assessment then becomes part of the improvement system rather than an event before improvement begins.

    The real purpose of reassessment

    The purpose is not to prove that the plan was followed. It is to discover whether the plan was right. Reassessment helps the organisation detect ineffective improvements early, adapt priorities, expose unintended consequences and remain focused on outcomes rather than activity.

    An assessment tells us where we are. Reassessment tells us whether we are actually moving—and whether we are still moving in the right direction.