Marcel van Oosterwijk

Category: Assessment

Quality, testing, assessment, evidence, judgment, reference models, and the development and evaluation of quality and capability.

  • Practice Note: Product Requirements for Operations

    We are used to translating product intent into requirements for development.

    We want to achieve something, so we determine what the product needs to do. Development implements it. Testing gives us evidence that it works. Eventually, we decide that the product is good enough to release.

    But development is only a means to an end.

    We do not build $shiny_new_function because the world needs another successfully implemented function. We build it because we expect something to happen once people start using it.

    Suppose $shiny_new_function has been implemented correctly. It produces the right results, performs well, is secure, handles the expected load and passes all its tests. Development has done an excellent job.

    Now suppose that, according to the original intent, success also meant that the function would be used at least 500 times per day.

    Three months after release, it is being used 17 times per day.

    Is the product good? That is a surprisingly different question from whether it was implemented correctly.

    Before release, we could test whether the function can handle 500 uses per day. We could simulate the load and collect evidence that the implementation is capable of doing what we expect.

    But no test can establish that real users will actually use it 500 times per day. For that, we need reality.

    This suggests that when we translate intent into requirements, we may be doing only half the job.

    We are quite accustomed to asking:

    What do we need to build to achieve this intent?

    Perhaps we should simultaneously ask:

    What would we need to observe in operation to know whether we actually achieved it?

    The first question gives Development something to realize.

    The second gives Operations something to observe.

    If the intent is to reduce customer effort through a new self-service capability, we will probably derive requirements describing what that capability must do.

    But if reduced customer effort is really what matters, we should also know what we expect to see once the capability is operating. Are customers actually using it? Are they completing the process? Where are they abandoning it? Are fewer customers contacting support? Has customer effort actually decreased?

    Those aren’t merely interesting metrics to add to an operational dashboard. They are evidence about whether the product is doing what we created it to do. And that means we may need to think about them before the product reaches production.

    If we want to know how $shiny_new_function is being used, we need to make that usage observable. If we want to understand where users abandon a process, we need to capture the relevant events. If we want to know whether self-service reduces support demand, we need some way of connecting those pieces of information.

    Otherwise, we may arrive in production and discover that we cannot answer one of the most important questions about the product:

    Is it actually working as intended?

    Not whether the software is running. Not whether the implementation satisfies its requirements. Whether the thing we built is actually producing the effect for which we built it.

    This is where production becomes particularly interesting for quality.

    Before release, much of our evidence necessarily concerns what we expect to happen. Requirements, reviews, tests and simulations can give us confidence that our realization is capable of fulfilling the intent.

    After release, we get something we did not have before. We get to see what actually happens. And reality may disagree with us.

    If $shiny_new_function works perfectly but receives only 17 of the expected 500 uses per day, that does not immediately tell us what is wrong. Perhaps users cannot find it. Perhaps they do not trust it. Perhaps another way of doing the same thing is easier. Perhaps we misunderstood their needs. Perhaps the assumption behind 500 uses per day was wrong. Perhaps 17 uses are actually enough to produce the business outcome we wanted, and we chose the wrong success criterion.

    Some of those answers would lead us back to Development. Others would lead us much further back, to our assumptions and even to the original intent.

    That is why a production observation should not merely become another defect or improvement ticket. It should become learning.

    Quality work has traditionally put enormous effort into determining whether something is good enough to enter production. That remains important. We need evidence that the realization is sufficiently sound before exposing it to reality.

    But perhaps we have concentrated too much on the means and not enough on the end.

    A product does not become successful because it passed its release assessment. That only means we had sufficient confidence to let it start doing the job for which it exists. The more important assessment begins when it actually does.

    So perhaps product intent should point in two directions from the beginning. It should tell Development what needs to be realized. And it should tell Operations what needs to be observed.

    One gives us evidence that the product can do what we intended. The other gives us evidence about whether it does.

    And the difference between those two is where some of our most valuable learning may be found.

  • Practice Note: From Assessment Model to Assessment Approach

    Earlier Practice Notes use the term Assessment Model to describe the way an assessment is structured and performed.

    I have come to think that Assessment Approach is a better term.

    The distinction matters because a Reference Model and an Assessment Approach are not the same kind of thing.

    A Reference Model describes what we assess against. It makes explicit our understanding of the subject: its purpose, expected qualities, characteristics, relationships, constraints, or other aspects relevant to judging it.

    An Assessment Approach describes how we perform the assessment. It determines how we investigate the subject, what evidence we seek, how we collect and interpret that evidence, and how we arrive at justified judgments.

    Put simply:

    Reference Model = what we assess against.

    Assessment Approach = how we assess.

    Calling both of these things models creates an unnecessary symmetry between them. The Reference Model represents an understanding of the subject. The Assessment Approach is the means by which we investigate reality in relation to that reference.

    This Practice Note therefore introduces a terminology change.

    From this point onward, I will use Assessment Approach rather than Assessment Model.

    Earlier Practice Notes will remain unchanged. Where they use Assessment Model in this sense, it should generally be understood as what I would now call an Assessment Approach.

    This is not a change to the underlying ideas in those notes. It is a refinement of the language used to express them.

    And perhaps there is a broader principle here as well.

    A developing body of knowledge does not need to rewrite its past every time its language improves. Earlier thinking can remain visible while later notes clarify, refine, or supersede it.

    The history of the terminology then becomes part of the history of the understanding.

  • Practice Note: Agile Capability Development?

    How do we approach capability development in an Agile product development organization? I first started thinking about this question during an assessment.

    Agile product development teams are deliberately given a high degree of autonomy. They are expected to organize their own work, use their professional expertise and continuously improve how they work. Different teams operate in different contexts and may therefore develop different practices.

    That creates an interesting problem. Suppose an assessment shows that an organizational capability needs to improve. Perhaps teams need to become better at identifying product-quality risks. Perhaps requirements need to become clearer before implementation. Perhaps testing needs to provide useful evidence earlier. Perhaps teams need to become better at learning from production.

    How should the organization make that happen?

    We could define a better process and ask every team to implement it. But then we are no longer developing a capability. We are prescribing its realization.

    At the other extreme, we could simply tell autonomous teams to improve. But if the capability matters at organizational level, leaving its development entirely to independent local initiatives provides little direction or continuity.

    There seems to be a tension between deliberate capability development and team autonomy. Perhaps Agile product development itself provides a way through it.

    What If We Developed Capabilities Like Products?

    Agile teams already work within a similar tension.

    A Product Owner or Product Manager can determine that something about the product needs to change without prescribing exactly how developers should implement that change.

    A Product Backlog makes those required changes visible and allows them to be prioritized.

    The development team then determines how to realize them.

    What if capability development worked in much the same way?

    A Capability Owner could maintain a Capability Backlog.

    A Capability Backlog Item could describe a required change in organizational ability.

    For example:

    Teams can identify significant product-quality risks early enough to influence development.

    The Capability Owner can decide that this ability matters and prioritize its development.

    But the backlog item does not say:

    Introduce mandatory product-risk workshops.

    Nor:

    Train every team in method X.

    Nor:

    Implement tool Y.

    Those are possible ways of realizing the required capability.

    They are interventions.

    The Capability Backlog Item describes what the organization needs to become capable of doing. The people developing the capability determine how to make that ability real.

    Perhaps capabilities themselves could be developed using Agile principles.

    From Implementation to Intervention

    The parallel with product development is surprisingly direct.

    A Product Backlog Item describes a required product change. Developers determine how to implement it.

    A Capability Backlog Item describes a required capability change. Teams determine how to intervene in their existing way of working to realize it.

    The relationship becomes:

    Product Backlog Item → Implementation → Product Increment

    and:

    Capability Backlog Item → Intervention → Capability Increment

    This distinction is important.

    Suppose teams need to become better at identifying significant product-quality risks.

    One team may introduce collaborative risk analysis during refinement.

    Another may change how testers and Product Owners work together.

    Another may introduce examples, modelling or lightweight risk classification.

    Another may discover that its existing practices are sufficient with a relatively small change.

    The interventions can differ because the teams and their contexts differ.

    What should remain common is the required organizational ability.

    This allows capability development to provide direction without unnecessarily standardizing how autonomous teams work.

    The Capability Reference Model

    But how does the Capability Owner know what belongs in the Capability Backlog?

    A backlog alone cannot provide the answer.

    There first needs to be an explicit understanding of the capability itself.

    A Capability Reference Model can describe the significant organizational abilities, expectations and conditions associated with the capability.

    Assessment then compares the realized capability with that reference.

    Perhaps the reference says that teams should be able to identify significant product-quality risks early enough to influence development.

    Assessment shows that several teams cannot reliably do this.

    That creates a development need.

    The need can become a Capability Backlog Item and be prioritized against other capability-development needs.

    This gives the reference model and backlog different purposes:

    The Capability Reference Model describes what good means.

    The Capability Backlog describes what we believe should change next.

    The reference provides continuity.

    The backlog provides focus.

    The Capability Increment

    Once a Capability Backlog Item has been prioritized, a team can take it into development.

    Suppose the item says:

    Teams can use identified product-quality risks to prioritize testing effort.

    The team explores why it cannot reliably do this today.

    Perhaps risk information arrives too late. Perhaps business impact is poorly understood. Perhaps testing is planned before risks have been discussed. Perhaps the team lacks experience with risk analysis.

    The team then chooses interventions appropriate to its situation.

    It might change refinement.

    It might experiment with risk workshops.

    It might seek coaching.

    It might change collaboration between Product Management and Testing.

    It might introduce supporting tools.

    Those interventions are analogous to implementation decisions in product development.

    But what is the resulting increment?

    It is not the workshop.

    It is not the training.

    It is not the tool.

    The increment is the change in organizational ability.

    Before the sprint, the team could not reliably use product-quality risks to prioritize testing.

    After development, it should be able to.

    That is the capability increment.

    Done Means Capable

    This has an important consequence for the Definition of Done.

    Suppose the team planned three interventions:

    • conduct training;
    • introduce a risk model;
    • use it during refinement.

    At the end of the sprint, all three activities have been completed.

    Is the Capability Backlog Item Done?

    Not necessarily.

    Completing the interventions demonstrates that development work occurred.

    It does not demonstrate that the capability increment exists.

    The relevant question is:

    Can the team now do what the Capability Backlog Item says it should be able to do?

    That requires evidence.

    The team might demonstrate the ability using real product work. It might be observable during refinement. A simulation might sometimes provide sufficient evidence. Other capabilities may require operational evidence over a longer period.

    The form of evidence will vary.

    The principle does not.

    A Capability Backlog Item is not Done because its interventions were completed. It is Done when there is sufficient evidence that the required capability has been realized.

    In other words:

    Done means capable.

    Reassessment Becomes Part of Development

    Capability assessment is often treated as something that happens before an improvement initiative and perhaps again much later.

    Agile capability development suggests something more continuous.

    Assessment identifies a capability-development need.

    That need enters the Capability Backlog.

    A team develops a capability increment through interventions.

    Evidence is collected.

    The resulting capability is reassessed against the relevant part of the Capability Reference Model.

    The cycle becomes:

    Reference → Assess → Prioritize → Develop → Intervene → Reassess → Learn

    If the required ability exists, the Capability Backlog Item may be Done.

    If it does not, the interventions may have failed or been insufficient.

    That is not necessarily wasted work. It is learning.

    The team may try another intervention. The Capability Owner may reconsider the backlog item. The assessment may reveal another underlying weakness. In some cases, experience may even challenge the Capability Reference Model itself.

    Capability development becomes iterative.

    Autonomy Without Abandoning Direction

    This may resolve the tension that started the question.

    Team autonomy does not have to mean that organizational capability development is left entirely to local initiative.

    And organizational capability development does not have to mean prescribing one standardized way of working.

    The organization can establish:

    what ability is needed,

    while autonomous teams retain substantial freedom over:

    how that ability is realized.

    Assessment closes the loop.

    Teams are not judged primarily on whether they adopted the prescribed practice. They are assessed on whether the required capability exists.

    Two teams may therefore realize the same capability differently.

    That is not necessarily inconsistency.

    If both realizations satisfy the relevant capability expectations, it may simply be autonomy working as intended.

    The reference provides coherence without requiring uniform implementation.

    When Capability Development Crosses Team Boundaries

    Not every capability can be developed by one team acting independently.

    Capabilities interact.

    Product Management affects Requirements Engineering. Requirements Engineering affects Testing. Architecture affects Engineering and Release Management. Operations provides evidence that influences several of them.

    An intervention may therefore contribute to several capability changes at once.

    Collaborative product-risk analysis, for example, might strengthen Product Management’s ability to make quality expectations explicit, Requirements Engineering’s ability to identify significant conditions and Testing’s ability to focus evidence on important risks.

    Likewise, one capability change may require several coordinated interventions.

    This is where broader organizational development becomes necessary.

    Transformation as Orchestrated Intervention

    A transformation program can be viewed from this perspective as a temporary mechanism for orchestrating interventions across multiple capabilities.

    Consider adopting SAFe.

    The visible transformation introduces roles, events, structures, backlogs, planning mechanisms and governance arrangements.

    Those are interventions.

    Behind them are organizational capabilities that the transformation is intended to strengthen: Portfolio Management, Product Management, Architecture, Requirements Engineering, Engineering, Testing, Release Management, cross-team coordination and others.

    The important question is therefore not simply:

    Did we successfully implement SAFe?

    It is:

    Did the capabilities we intended to improve actually improve?

    PI Planning may have been implemented perfectly while cross-team coordination remains weak.

    Product Management roles may have been appointed while product direction remains fragmented.

    New testing practices may have been introduced while the organization remains unable to provide timely evidence about product quality.

    The interventions can be successfully implemented while the capability development fails.

    This is why the intervention should not become the reference against which improvement is judged.

    Capability Stewardship Continues

    Transformation programs are temporary.

    Capabilities are not.

    A transformation may accelerate development across many capabilities and coordinate interventions that would otherwise be difficult to perform.

    Eventually the program ends.

    The Capability Reference Model remains.

    The Capability Backlog remains.

    New evidence appears.

    The environment changes.

    New weaknesses emerge.

    The capability continues to develop.

    The Capability Owner therefore provides continuity beyond any particular intervention or transformation.

    The role is not to own every practice used to realize the capability.

    It is to steward the capability itself: maintain its reference, understand its current state, prioritize what needs to change, and ensure that development and reassessment continue.

    Reflection

    The original problem seemed to be about autonomy.

    How can an organization deliberately improve ways of working when Agile teams are supposed to determine for themselves how they work?

    Perhaps the problem appears because we confuse the capability with its realization.

    The organization does not necessarily need to prescribe the practice.

    It needs to make the required ability explicit.

    A Capability Reference Model can describe that ability.

    Assessment can show where the realized capability is insufficient.

    A Capability Owner can prioritize the required changes through a Capability Backlog.

    Autonomous teams can discover suitable interventions.

    Reassessment can determine whether those interventions actually produced the required capability increment.

    That is remarkably similar to the way we already develop products.

    Perhaps capabilities themselves could be developed using Agile principles.

  • Practice Note: A Reference Model Does Not Preserve Understanding by Itself

    A recurring idea in this body of work is that organizations need an explicit reference if they want to preserve, develop and assess understanding over time.

    Without such a reference, understanding gradually becomes implicit. People leave. Context disappears. Decisions remain while their rationale is forgotten. Eventually, whatever happens to survive — an existing system, a process, a framework, a set of requirements — may become the implicit definition of what should exist.

    The Reference Model provides an alternative. It makes important concepts, relationships, expectations, assumptions and boundaries explicit enough that future people do not have to reconstruct everything from the realization itself.

    But there is a potential problem with this argument. If documentation does not preserve understanding simply because it exists, why should a Reference Model?

    It should not. A Reference Model is necessary for Continuity, but it is not sufficient.

    Making Understanding Explicit

    The first step toward preserving understanding is making enough of it explicit.

    This is important because artifacts tend to survive longer than the reasoning that created them. A system may continue operating long after its designers have left. A capability may retain its processes after nobody remembers why they were introduced. A product backlog may continue evolving after its connection to the original product intent has weakened.

    A Reference Model helps by separating the intended understanding of the subject from its current realization. That creates a stable point from which the subject can be developed and assessed without treating the current implementation as the definition of what is correct.

    But making understanding explicit is not the same as preserving it. A Reference Model is itself an artifact. It can survive while the understanding required to interpret it disappears.

    A Model Must Remain Interpretable

    Imagine finding a carefully structured Reference Model ten years after it was created. The concepts are still there. The relationships are documented. Important expectations are described. Nothing has been deleted.

    Yet some of the terminology has changed. Several concepts no longer mean quite what they once did. Assumptions that were obvious to the original authors are no longer obvious to the current organization. Nobody remembers why one relationship was considered significant while another was deliberately excluded.

    The model survived. Its interpretation did not.

    This is the same problem that affects ordinary documentation. Preserving information does not automatically preserve the context required to reconstruct its meaning.

    A Reference Model therefore needs more than correct statements. Where necessary, it must preserve enough rationale, assumptions, definitions and context for future people to understand what those statements were intended to mean.

    Otherwise, the model may remain readable while becoming progressively less understandable.

    A Model Must Remain Connected to Reality

    There is another risk. Once a Reference Model becomes authoritative, its contents can begin to acquire authority simply because they are in the model.

    An expectation may originally have been based on operational evidence. An assumption may have reflected a particular business condition. A relationship may have been inferred from several observations. A constraint may have represented a deliberate decision rather than an unavoidable fact.

    Over time, those distinctions can disappear.

    Eventually someone asks:

    Why do we believe this?

    And the answer becomes:

    Because it is in the Reference Model.

    At that point, the mechanism intended to prevent Reference Substitution risks becoming a Reference Object itself.

    A Reference Model should therefore remain connected, where relevant, to the basis for its claims. That does not mean every statement needs an academic citation. It means that important distinctions between observation, evidence, assumption, decision, interpretation and intent should not disappear simply because they have been synthesized into a model.

    Agreement does not solve this problem either. An entire organization may agree with what the Reference Model says. That agreement is useful evidence that the model represents shared understanding. It is not necessarily evidence that every claim within the model remains correct.

    The Reference Model must remain open to challenge from evidence.

    The Boundary Must Be Explicit

    No Reference Model can contain everything.

    The moment we describe a product, capability, system or body of knowledge, we make decisions about what belongs inside the subject and what remains outside it. Those decisions matter because the boundary helps define the object being understood.

    Without an explicit boundary, absence becomes ambiguous. If something is missing from the Reference Model, is it outside the subject? Was it deliberately excluded? Is it considered insignificant? Is it unknown? Or did somebody simply forget it These possibilities have very different meanings.

    A useful Reference Model therefore needs an explicit subject and sufficiently clear boundaries. Not because boundaries can eliminate every ambiguity, but because they allow people to reason about what the model claims to represent.

    Before asking whether a Reference Model is complete, we need to know what it is intended to be complete about.

    The Problem of Appropriate Abstraction

    Reference Models need abstraction. If a Capability Reference specifies a particular process, role structure and toolset as the definition of good capability, it may prevent legitimate alternative realizations.

    If a Product Reference simply reproduces the current architecture, requirements and behavior, the existing product becomes the model for its own future. The Reference Model has then become too concrete. Instead of preserving intent independently of realization, it preserves the realization.

    But abstraction has an opposite failure mode. Consider a Capability Reference that says an organization should have effective governance, appropriate competence, efficient processes and continuous improvement. It is difficult to disagree with. It is also difficult to do much with.

    What would distinguish a strong realization from a weak one? What evidence would matter? What should Development attempt to improve? What exactly should an assessor assess?

    Excessive abstraction can make a Reference Model universally acceptable precisely because it no longer makes sufficiently meaningful distinctions.

    The challenge is therefore not to maximize abstraction. It is to find the appropriate abstraction.

    A Reference Model should be abstract enough to permit alternative realizations, but specific enough to distinguish meaningful alternatives.

    This principle applies beyond any particular domain.

    A Capability Reference must avoid prescribing one organizational implementation while remaining specific enough to describe meaningful organizational ability.

    A Product Reference must remain independent of today’s implementation while being concrete enough to guide Development and Assessment.

    A model of a body of knowledge must allow understanding to evolve without reducing that understanding to principles so generic that the observations, relationships and reasoning that gave them meaning disappear.

    The object changes. The problem remains the same.

    A Model Must Be Maintained Through Learning

    Even a well-bounded, appropriately abstract and interpretable Reference Model eventually becomes outdated if understanding continues to evolve while the model does not.

    New evidence appears.

    Operational experience challenges assumptions.

    Assessments reveal weaknesses in the reference itself.

    Development creates possibilities that were not previously considered.

    The environment changes.

    Stakeholders learn that they value something differently than expected.

    A Reference Model that cannot change eventually stops representing current understanding.

    But simply updating it creates another Continuity problem. If yesterday’s model says one thing and today’s says another, future people may need to understand not only what changed but why.

    Otherwise maintenance preserves the latest answer while destroying the reasoning that connects one state of understanding to the next.

    A living Reference Model therefore needs stewardship.

    Changes should be deliberate. Important revisions should remain reconstructable. New evidence should be able to challenge old assumptions. Learning should feed back into the reference rather than accumulating elsewhere until the model gradually drifts away from what the organization actually knows.

    Continuity does not require the Reference Model to remain unchanged. It requires understanding to remain continuous while the Reference Model evolves.

    The Reference Model Is Not the Understanding

    This leads to an important qualification.

    A Reference Model does not contain organizational understanding in the same way that a database contains records.

    Understanding exists through the relationship between the model, the evidence behind it, the context in which it is interpreted, the people who use it, the realizations it guides and the learning that changes it.

    The Reference Model gives that understanding structure.

    It makes important parts explicit.

    It provides an independent reference for Development and Assessment.

    It helps future people reconstruct why the subject is understood in a particular way.

    But the artifact itself is not Continuity.

    A perfectly preserved Reference Model can still become incomprehensible, outdated, disconnected from evidence, incorrectly scoped or so abstract that it ceases to guide meaningful judgment.

    The same lesson that led to Reference Models therefore applies to Reference Models themselves.

    Artifacts do not preserve understanding merely because they survive.

    A Reference Model does not preserve understanding by itself.

    It creates a structure through which understanding can be preserved, challenged, developed and continued over time.

  • Practice Note: Agreement Is Not Evidence

    In assessments, workshops and organizational discussions, agreement often feels reassuring. Several people describe the situation in the same way. Stakeholders confirm one another’s explanations. A workshop reaches consensus about why something works as it does. After enough people have repeated the same account, it becomes tempting to treat the matter as established.

    Everyone agrees. Therefore, we know.

    But those are two different things. Agreement tells us something about what people believe. It does not necessarily tell us whether what they believe is true.

    When Agreement Looks Like Evidence

    Imagine interviewing five people about why a particular governance process exists. All five give essentially the same explanation. That consistency is useful information. It suggests that the organization has a shared understanding of the process.

    But suppose all five originally learned the explanation from the same person. We do not really have five independent pieces of evidence. We have one explanation that has propagated through five people.

    Or perhaps the explanation has simply been repeated for so long that nobody remembers where it came from. It has become organizational knowledge through repetition rather than through continued connection to its original rationale. The agreement is real. The explanation may still be wrong.

    This creates an important distinction in assessment.

    Agreement can be evidence of agreement without being evidence for the claim being agreed upon.

    If several stakeholders independently describe the same organizational problem, their agreement may tell us that the perception is widespread. That can itself matter. Shared perceptions influence behavior, decisions and organizational relationships.

    But if the question is whether the underlying claim is correct, agreement alone is not enough. Suppose everyone agrees that a particular control is required by regulation. The agreement demonstrates that the organization believes the control to be mandatory. It does not establish that the regulation actually requires it.

    Suppose everyone agrees that a system limitation exists because of an architectural constraint. The agreement tells us that this explanation is commonly accepted. It does not establish that the constraint still exists.

    Suppose everyone agrees that a capability works well. That tells us something about organizational confidence or perception. It does not demonstrate that the capability reliably produces the intended effects under the conditions that matter.

    Agreement Is Not Corroboration

    The distinction becomes particularly important when evidence is collected through interviews and workshops. Multiple participants can easily create an impression of corroboration. But corroboration depends on more than the number of people repeating a claim. It depends on whether the supporting evidence is meaningfully independent.

    Ten people repeating an explanation that originated from the same document may represent one source, not ten.

    Five teams using the same dashboard may appear to provide five confirmations of performance while all relying on the same underlying measurement.

    Several documents may repeat the same statement while ultimately tracing back to one original assumption.

    Counting confirmations without understanding their provenance can therefore create false confidence.

    When Agreement Does Matter

    This does not make agreement unimportant. Agreement can reveal shared understanding. Disagreement can reveal ambiguity, hidden assumptions, competing perspectives or different experiences of the same subject. Consensus can be valuable when an organization must choose an intention, define a boundary, establish priorities or decide which trade-offs it is willing to accept.

    Some organizational questions are inherently matters of choice.

    What do we want this capability to achieve?

    Which outcomes do we value?

    What level of risk are we willing to accept?

    Where should we place the boundary of this product or capability?

    For questions like these, agreement may be part of establishing the reference itself. The organization is not discovering an external fact. It is making its intent explicit.

    But other questions concern reality.

    Does the capability currently produce the intended effect?

    Does this architectural constraint actually exist?

    Did this decision originate for the reason we now believe?

    Does the product behave reliably under the conditions we expect?

    Those claims require evidence.

    Understanding What Kind of Evidence We Have

    This is why agreement should not be confused with corroboration.

    Corroboration strengthens a claim by connecting it to additional evidence that does not merely reproduce the same assumption. That evidence might come from observed behaviour, measurements, historical records, operational outcomes, contemporaneous decisions, technical artifacts or genuinely independent accounts.

    The purpose is not to distrust people.

    It is to understand what kind of evidence we actually have.

    A shared explanation may be highly valuable evidence about organizational understanding while remaining weak evidence about historical fact. A unanimous workshop may establish organizational intent while saying little about current performance. A commonly accepted assumption may provide a useful starting point while still requiring validation.

    Agreement and Continuity

    This distinction also matters for Continuity.

    Organizations inherit explanations just as they inherit documents, processes and systems. Over time, an interpretation can lose its connection to the evidence or rationale that originally supported it. The explanation survives because people continue to repeat it.

    Eventually, agreement itself becomes the justification.

    “We have always understood it this way.”

    At that point, shared understanding can conceal Understanding Debt rather than prevent it.

    Preserving understanding therefore requires more than preserving consensus. Where a claim matters, future people should ideally be able to distinguish what was recorded, what was observed, what was inferred, what was decided, and what became commonly believed.

    Agreement belongs in that picture. But it should remain what it is.

    Agreement is evidence that people agree.

    It is not, by itself, evidence that they are right.