Marcel van Oosterwijk

Category: Capabilities

Organizational capabilities, responsibilities, governance, ways of working, transformation, and how organizations develop and sustain their ability to achieve outcomes.

  • Practice Note: A Reference Model Is Not a Prescription

    Two teams use exactly the same testing practices.

    One receives a positive capability assessment. The other does not.

    Is the assessment inconsistent?

    Not necessarily.

    One team develops a relatively simple internal application. The other develops a safety-critical subsystem.

    The practices may be identical, but the expectations should not be.

    This raises an interesting question.

    How can a generic capability reference model remain useful when adequate capability depends on context?

    From Generic to Specific

    A capability reference model describes what an organization should be capable of doing, without necessarily prescribing how that capability must be realized.

    Consider Quality Assurance.

    A generic reference model might describe capabilities related to quality strategy, verification, validation, test design, defect management and quality evaluation.

    These concepts can apply across many organizations.

    But their realization depends on context.

    An organization developing medical devices faces different risks, obligations and quality expectations from one developing internal business applications.

    Even within the same organization, different products and subsystems may require different approaches.

    A safety-critical component might demand extensive traceability and formal verification. A user interface might place greater emphasis on usability and exploratory testing. A data integration component might require particular attention to data integrity and failure handling.

    The underlying capability concepts remain recognizable.

    What constitutes adequate capability changes.

    Contextualizing the Reference Model

    This suggests a progression:

    Generic capability → Organizational context → Product or subsystem context

    At the generic level, the reference model provides a common vocabulary and a basis for reasoning about capability.

    At the organizational level, we consider the organization’s objectives, responsibilities, operating environment, risks and constraints.

    At the product or subsystem level, we consider the specific characteristics and quality expectations of the subject being assessed.

    Each level adds context.

    But that doesn’t necessarily mean creating an entirely new capability model at every level.

    The reference model can remain conceptually stable while the expectations, evidence and appropriate realization become increasingly specific.

    For example, risk-based testing may be a relevant capability in both a business application and a safety-critical subsystem.

    But what constitutes sufficient risk analysis, test coverage and supporting evidence may differ substantially.

    The capability is recognizable in both contexts.

    Its adequate realization is not necessarily identical.

    Assessing Capability or Checking Compliance?

    This distinction matters because reference models can easily become checklists.

    Does the organization have a test strategy?

    Does it use automated testing?

    Does it perform reviews?

    Does it maintain traceability?

    These questions may be useful, but the existence of a practice doesn’t necessarily establish that the capability is adequate.

    An organization can perform all the expected activities while remaining unable to achieve the intended outcomes.

    Conversely, an organization may realize a capability effectively through practices that differ from those anticipated when the reference model was created.

    If we assess compliance with prescribed practices, we risk confusing the means with the capability.

    Capability assessment should evaluate fitness for purpose, not conformity to a universal implementation.

    This also means that identical practices need not produce identical assessment results.

    And different practices may legitimately support the same capability rating.

    The question is whether the subject can achieve what is required of it in its particular context.

    The Risk of Contextualizing Too Much

    There is a complication.

    If every organization, product or subsystem receives its own completely customized model, we may lose the consistency that made the reference model useful in the first place.

    Assessments become difficult to compare. Common capability concepts disappear behind local terminology. Improvement approaches become unnecessarily fragmented.

    But contextualizing too little creates the opposite problem.

    We evaluate subjects against generic expectations that may be irrelevant, insufficient or inappropriate.

    The challenge is therefore to distinguish between what should remain stable and what should vary.

    The reference model provides the conceptual structure.

    Context determines which capabilities matter, what adequate performance means and what evidence is needed to assess them.

    Sometimes the context may even reveal capabilities or dimensions missing from the generic model.

    That should be an opportunity to improve the reference model, rather than a reason to abandon it.

    A Reference Model Is Not a Prescription

    Perhaps the value of a capability reference model lies precisely in this distinction.

    It allows us to reason consistently about different subjects without assuming that they should all operate identically.

    It provides a shared language for understanding, assessing and developing capabilities.

    But it does not eliminate the need for professional judgment.

    We still need to understand the subject, its purpose, its environment and the outcomes it must achieve.

    Only then can we determine what adequate capability looks like.

    A reference model tells us what to reason about. Context tells us what good looks like.

  • Exploration: Is Moving Upstream a Professional Quality?

    A tester can execute tests perfectly against the wrong requirements.

    A developer can implement an elegant solution to the wrong problem.

    A project manager can deliver a project successfully without achieving the change it was intended to create.

    A consultant can implement an improvement initiative without improving anything.

    In each case, the professional may have performed the assigned work competently.

    But was it good professional work?

    Looking Beyond the Assignment

    Consider a tester who discovers that a requirement is ambiguous.

    The immediate response might be to request clarification so that testing can continue.

    But perhaps the ambiguity suggests a more fundamental problem with requirements management.

    And perhaps that problem originates in how stakeholders communicate, how decisions are made or how the organization understands customer needs.

    The tester can keep moving upstream.

    From testing to requirements. From requirements to product development. From product development to organizational decision-making.

    At each step, the question changes.

    Not simply How can I do my work better?

    But Why does this work need to be done, and what determines whether it creates value?

    This pattern isn’t unique to testing.

    A developer might question the architecture rather than simply implement another workaround.

    A project manager might question whether the planned deliverables will produce the intended outcomes.

    A consultant might question whether the requested intervention addresses the capability that actually needs improvement.

    In each case, moving upstream means looking beyond the immediate assignment toward the conditions and purposes that give it meaning.

    Is This Professional Development?

    We often associate professional development with increasing expertise.

    A tester becomes better at testing. A developer becomes better at programming. A consultant becomes more proficient in a methodology.

    But perhaps professional development also has another dimension.

    The ability to understand how one’s work contributes to something larger.

    That doesn’t necessarily mean moving into management or changing professions.

    A tester doesn’t need to become a business analyst to recognize weaknesses in requirements.

    A developer doesn’t need to become a product manager to question whether a feature solves the intended problem.

    And a consultant doesn’t need to become a strategist to recognize that implementing a framework is not the same as developing an organizational capability.

    Perhaps moving upstream isn’t about changing what we do.

    It’s about changing how we understand what we do.

    How Far Upstream?

    There is a problem with this reasoning.

    We can always ask another question.

    Why are the requirements poor? Why does the organization work this way? Why were these priorities chosen? Why does the business pursue these objectives?

    Eventually, questioning the purpose of the work can consume more attention than performing it.

    And not every upstream question is useful.

    Sometimes the problem is understood, the assignment is appropriate and the most valuable contribution is excellent execution.

    Specialization exists for good reasons. Deep expertise matters.

    A professional who continually challenges the assignment but cannot perform it well isn’t necessarily demonstrating superior professional quality.

    So perhaps moving upstream is not inherently better.

    The important quality might instead be knowing when the boundaries of the assignment prevent us from addressing the problem.

    And having the judgment to recognize when those boundaries should be questioned.

    When Implementation Becomes Easier

    AI adds another dimension to this question.

    As AI becomes increasingly capable of producing code, tests, documents, analyses and other artifacts, some implementation activities become easier.

    That doesn’t eliminate the need for expertise.

    But it may change where professional judgment contributes the most value.

    If producing a solution becomes easier, choosing the right problem becomes more important.

    AI can help implement what we ask for.

    It cannot make our intentions appropriate merely by implementing them successfully.

    Perhaps this makes the ability to move upstream more valuable.

    Not because everyone should become a strategist, but because the consequences of poorly understood intentions may become easier to realize.

    We may become increasingly effective at doing things without becoming equally effective at determining whether those things are worth doing.

    A Different Dimension of Professional Quality?

    Perhaps we should distinguish three things.

    Competence: Can I perform the assigned work well?

    Understanding: Do I know why the work matters?

    Judgment: Can I recognize when performing the assigned work well will not achieve its intended purpose?

    The first is essential.

    The second provides context.

    The third allows a professional to challenge the relationship between the assignment and its purpose.

    But even this distinction raises questions.

    How much upstream understanding should we expect from a professional?

    When does questioning an assignment demonstrate responsibility, and when does it become interference?

    Can organizations genuinely encourage professionals to challenge assumptions while continuing to evaluate them primarily on how efficiently they complete assigned tasks?

    And what happens when a professional recognizes an upstream problem but has no authority to influence it?

    Perhaps moving upstream is not simply an individual quality. It may also depend on whether the organization is capable of using the understanding that professionals develop.

    Perhaps the Question Is Different

    Maybe professional quality isn’t about moving upstream as far as possible.

    Nor is it about constantly questioning everything.

    It may be about retaining the ability to connect our work to its purpose, and recognizing when that connection is missing.

    A good professional knows how to perform the work.

    A better professional may also recognize when performing that work well is insufficient.

    But that leaves an uncomfortable question.

    If we recognize that the assignment will not achieve its intended purpose, is it still professional to complete it exactly as requested?

  • Practice Note: The Project Is Not the Subject of the Change

    Projects have a peculiar ability to become the center of attention.

    We establish plans, milestones, budgets, governance structures and reporting mechanisms. We monitor progress, manage risks and eventually determine whether the project succeeded.

    Yet the organization rarely wanted a project in the first place.

    It wanted something to change.

    A testing improvement project exists to improve testing capability. A system replacement project exists to change a system or product. A transformation programme exists to develop organizational capabilities.

    The project exists in service of that change.

    But this raises an interesting distinction.

    The Project Is a Subject Too

    A project can itself be understood as a subject.

    It has boundaries, responsibilities, capabilities and characteristics that can be assessed and improved.

    There is nothing wrong with managing a project as a subject.

    The problem arises when we confuse it with the subject it exists to change.

    Consider a system replacement project.

    The project is one subject: a temporary organization coordinating change.

    The system being replaced is another. The organizational capability depending on that system may be a third.

    Each can be understood, developed and assessed.

    But evidence about one does not necessarily tell us much about the others.

    Two Different Claims of Success

    Suppose a project finishes on schedule, within budget and with all agreed deliverables accepted.

    That is meaningful evidence about the project.

    But has the intended change occurred?

    Does the new system deliver the expected outcomes? Has the organizational capability improved? Have the risks that motivated the project been reduced?

    Those are different questions requiring different evidence.

    The difficulty is that project performance is often easier to observe than the effects of the change.

    Milestones can be counted. Deliverables can be accepted. Activities can be marked complete.

    Capability development and organizational outcomes are harder to assess.

    So the visible subject can gradually substitute for the important one.

    A training programme is completed, so competence is assumed to have improved.

    A process is implemented, so behaviour is assumed to have changed.

    A system goes live, so the transformation is considered successful.

    But completing an intervention is not evidence that it achieved its intended effect.

    The Project as an Intervention

    Seen from the perspective of the subject being changed, the project is a temporary mechanism for coordinating interventions.

    It may introduce technologies, practices, roles, structures or governance arrangements intended to move another subject toward a desired state.

    The project therefore contains an implicit hypothesis:

    If we perform these interventions, we expect the subject to change in these ways.

    The project plan describes what we currently believe is necessary to produce that change.

    But what happens when an intervention is completed and the expected effect does not appear?

    Simply completing the remaining activities more efficiently may not help.

    The evidence may be telling us that our understanding of the subject, or of how to change it, was incomplete.

    The ability to adjust the interventions then becomes more important than adherence to the original plan.

    The Temporary and the Continuing Subject

    There is another complication.

    Projects are deliberately temporary.

    We create them with the intention of eventually dismantling them.

    Project teams dissolve. Governance structures disappear. Budgets close. Responsibilities are transferred.

    But the subject being changed may need to continue for decades.

    During the project, considerable understanding accumulates within the temporary organization.

    People learn why decisions were made, which alternatives were rejected, what assumptions proved incorrect and where uncertainty remains.

    What happens to that understanding when the project disappears?

    Traditional handover often concentrates on transferring deliverables, documentation and operational responsibilities.

    But continuity requires more than possession of artifacts.

    Can those who inherit the subject understand why it is the way it is?

    Do they know what outcomes were intended, which assumptions shaped the solution and what remains uncertain?

    Can they continue assessing and developing the subject without depending on the people who participated in the project?

    A project may successfully deliver a new realization while leaving the organization poorly equipped to understand or evolve it.

    The project is supposed to disappear. The understanding required to continue the subject should not disappear with it.

    What Does Project Completion Mean?

    Perhaps project completion deserves a broader interpretation.

    We need evidence that the project has been managed effectively.

    We need different evidence that the intended change has occurred.

    And we need confidence that the organization can continue managing and developing the changed subject after the project has ended.

    The project doesn’t need to preserve itself.

    Its temporary structures can disappear.

    But the understanding, responsibility and ability to learn must continue somewhere.

    Because the project is not the subject of the change.

    It is the temporary subject responsible for helping another subject become something different — and remain capable of evolving afterward.

  • Practice Note: Transformation Is Not an Intervention

    A recent discussion about Agile transformation made me wonder whether “mindset shift” is really the right way to describe transformation.

    Mindset matters. But organizations do not transform simply because people start thinking differently.

    They transform when the organization becomes capable of behaving differently.

    And that changes how we should think about transformation itself.

    Introducing Change Is Not Transforming

    Organizations introduce new frameworks, methodologies, technologies, governance structures and operating models.

    They reorganize teams, establish new roles, implement tools and train employees.

    These activities may be necessary.

    But they are interventions, not transformations.

    An organization can adopt Agile terminology, introduce Scrum ceremonies and appoint Product Owners without becoming particularly capable of delivering value iteratively.

    It can migrate infrastructure to the cloud without becoming more adaptable.

    It can introduce AI tools without becoming better at using information or making decisions.

    The intervention has happened.

    But has the organizational capability changed?

    From Interventions to Capabilities

    Perhaps transformation should begin with a different question.

    Not “What should we introduce?”

    But “What must the organization become capable of doing?”

    That requires understanding the current capability, defining the desired capability and identifying what prevents the organization from achieving it.

    Only then can we determine which interventions might help.

    And those interventions rarely operate independently.

    Introducing a new development methodology may require changes in requirements management, architecture, governance, competence development and decision-making.

    Improving one practice without addressing the surrounding organizational mechanisms may accomplish surprisingly little.

    Transformation therefore becomes an exercise in orchestrating interventions toward a desired change in organizational capability.

    Did Anything Actually Change?

    There is another problem.

    Organizations often measure transformation through the completion of interventions.

    How many employees have been trained? How many teams have adopted the framework? Has the new operating model been implemented? Has the technology been deployed?

    These measures tell us whether activities have been completed.

    They do not necessarily tell us whether the organization has become more capable.

    To establish that, we need evidence.

    Can teams now make decisions they previously couldn’t? Can the organization respond more effectively to changing requirements? Can it deliver acceptable outcomes more reliably?

    And do these improvements persist after the transformation programme has ended?

    A temporary improvement achieved through extraordinary effort is not necessarily a sustained organizational capability.

    The transformation is not the framework, method or technology we introduce. It is the sustained change in organizational capability that those interventions produce.

    Transformation as a Learning Process

    This also suggests that transformation cannot simply be a predefined sequence of activities.

    We may have a clear understanding of the capability we want to develop, but we cannot always know in advance which interventions will produce the intended effects.

    We need to observe what happens.

    Did the intervention work? What changed? What remained unchanged? Did we discover new constraints or dependencies?

    The answers should influence what we do next.

    In that sense, transformation is an iterative learning process:

    Current capability → Desired capability → Interventions → Observed effects → Learning → Adjusted interventions.

    The interventions are hypotheses about how to develop the capability.

    Their effects provide evidence about whether those hypotheses were correct.

    And the transformation progresses as the organization uses that evidence to adjust its approach.

    What Are We Actually Transforming?

    Perhaps the most important question in any transformation initiative is deceptively simple:

    What will the organization be capable of doing afterward that it cannot do today?

    If we cannot answer that question, it becomes difficult to determine which interventions are necessary.

    And even harder to establish whether the transformation has succeeded.

    Because introducing change is relatively easy to observe.

    Establishing that the organization has become sustainably more capable is a different matter.

  • Practice Note: The Illusion of the Feedback Loop

    Most feedback loops aren’t loops.

    Imagine telling an employee in a supermarket that the price on the shelf doesn’t match the price at checkout.

    The employee apologizes. Perhaps they correct the price for you. They might even inform their supervisor.

    Problem solved?

    For you, perhaps.

    But will someone correct the shelf label? Will anyone investigate why the discrepancy occurred? And if it keeps happening, will the organization change anything?

    The organization has received feedback and responded to it.

    But has it learned anything?

    When Feedback Travels Backwards

    Organizations have developed countless mechanisms for collecting feedback.

    Customer complaints, retrospectives, quality metrics, incident reports, assessments and lessons learned.

    We collect information, discuss findings, make decisions and initiate improvement actions.

    We call these feedback loops.

    But information travelling backwards through an organization doesn’t automatically create a feedback loop.

    The information must be capable of influencing the subsequent behaviour of the system.

    That doesn’t mean every observation must result in a change. Sometimes feedback confirms that everything is working as intended.

    What matters is whether the system can respond when the evidence warrants it.

    Closing Actions Is Not Closing the Loop

    Consider an organization that identifies a recurring problem.

    An improvement action is agreed upon, assigned to someone and eventually marked as completed.

    The action has been closed.

    But did it solve the problem?

    Perhaps the intervention addressed a symptom rather than its cause. Perhaps it introduced new problems. Perhaps it worked in one situation but not in others.

    Unless someone evaluates the effect, the organization doesn’t actually know.

    And if the next occurrence of the problem leads to the same discussion and another improvement action, the organization may be repeating a process without learning from it.

    A feedback loop requires more than information and intervention.

    It requires evidence about the consequences of the intervention.

    Closing an action is not the same as closing a feedback loop.

    When Organizations Become Good at Responding

    There is an interesting paradox here.

    An organization can become highly effective at handling individual problems without becoming better at preventing them.

    Customer complaints are resolved quickly. Incidents are closed within agreed timeframes. Retrospectives produce action lists. Assessments generate improvement plans.

    Every mechanism appears to be working.

    Yet the same underlying problems continue to occur.

    The organization has optimized its ability to respond.

    But perhaps it hasn’t developed an equivalent ability to learn.

    This distinction matters because responding to a problem and changing the conditions that produced it are different capabilities.

    The first restores normal operations.

    The second improves the organization’s ability to operate.

    Both are valuable, but they shouldn’t be confused.

    Where Does the Loop Close?

    Perhaps we should evaluate feedback mechanisms differently.

    Instead of asking how much feedback we collect, how quickly we respond or how many actions we complete, we might ask:

    What can actually change as a result of this feedback?

    Can it challenge existing assumptions? Influence priorities? Change practices? Redirect investments? Improve organizational capabilities?

    And when something changes, do we evaluate whether the intended improvement was achieved?

    These questions concern more than the existence of feedback mechanisms.

    They concern the organization’s ability to use evidence to adapt its behaviour.

    Because a process doesn’t become a feedback loop merely because information travels backwards.

    Perhaps the loop doesn’t close when we act on feedback. It closes when we learn whether our response actually worked.