Marcel van Oosterwijk

Category: Knowledge & AI

How knowledge and understanding are created, represented, preserved, synthesized, transferred, and developed, including the changing role of AI in knowledge work.

  • Practice Note: Bodies of Knowledge and Context Layers

    Imagine giving AI access to everything an organization knows.

    Every document, presentation, architectural decision, assessment report, project deliverable and incident record.

    Would the AI understand the organization?

    Not necessarily.

    It might retrieve relevant information, summarize documents and identify recurring themes.

    But would it understand what the information refers to, how different subjects relate, and which evidence supports which conclusions?

    Perhaps making organizational knowledge interrogable requires more than access to information.

    Perhaps it also requires context.

    A Body of Knowledge Is Not a Model

    Organizations accumulate enormous Bodies of Knowledge.

    They contain observations, decisions, experiences, explanations and evidence gathered over time.

    But much of this information was created for particular purposes, by different people, using different terminology.

    A project report describes a problem. An assessment identifies a capability weakness. An incident record describes a failure. An architectural decision explains a technical choice.

    These artifacts may concern related aspects of the same organizational reality.

    Yet their relationships are rarely explicit.

    An AI assistant might retrieve all four documents without necessarily understanding how they relate.

    The information exists.

    The structure needed to interpret it may not.

    Could Reference Models Provide the Context?

    A reference model provides a conceptual structure for understanding a particular kind of subject.

    A capability reference model describes capabilities and their relationships.

    A product reference model describes concepts such as purpose, characteristics, architecture, decisions and evidence.

    These models are generic. They do not describe any particular organization or product.

    But they can be contextualized.

    A generic capability model can be related to an organization’s actual capabilities, responsibilities and operating environment.

    A product reference model can be applied to a particular product, connecting its purpose, realization, characteristics and supporting evidence.

    This suggests an interesting possibility.

    Could contextualized reference models serve as context layers between an organization’s Body of Knowledge and the questions people want to ask?

    The Body of Knowledge would provide the information and evidence.

    The context layers would help establish what that information concerns and how it relates.

    AI could use both to investigate questions that were never anticipated when the information was created.

    Asking Better Questions

    Consider a consultant asking:

    Why are we experiencing recurring defects in Product X?

    Without an explicit context model, AI might retrieve incident reports, defect statistics and retrospective findings.

    That could already be useful.

    But suppose it also had access to contextualized models describing Product X, its expected characteristics, the organizational capabilities involved in developing it and the relationships between them.

    The investigation could become more structured.

    Which product characteristics are affected?

    Which capabilities contribute to producing those characteristics?

    What evidence exists about those capabilities?

    Have previous interventions attempted to address the problem?

    What happened afterward?

    The context layer would not provide the answer.

    Nor would it establish that a particular capability caused the defects.

    But it could help identify relevant relationships, locate evidence and expose gaps in our understanding.

    The model would help us determine what to investigate. The evidence would help us determine what we can reasonably conclude.

    But Who Says the Model Is Correct?

    There is a complication.

    A contextualized reference model is itself a representation of organizational reality.

    And representations can be incomplete, outdated or wrong.

    Perhaps the model assigns responsibility to a team that no longer performs the work.

    Perhaps it describes a dependency that has disappeared.

    Perhaps it assumes that a capability operates in a particular way, while the available evidence suggests something different.

    If AI treats the context layer as unquestionable truth, it may produce convincing answers based on incorrect assumptions.

    That would make the organization easier to interrogate without necessarily making its answers more reliable.

    So perhaps the relationship must work in both directions.

    The model helps interpret evidence.

    But evidence must also be capable of challenging the model.

    When observations contradict the contextualized understanding, that contradiction should become visible.

    And when the evidence warrants it, the model should change.

    An Interrogable Organization

    This raises another possibility.

    Perhaps an interrogable organization should not merely allow people to ask questions about its accumulated knowledge.

    It should also allow them to question its understanding of itself.

    Why do we believe this capability depends on that one?

    What evidence supports this assessment?

    When was this relationship last validated?

    Which assumptions are no longer supported?

    What has changed since this model was established?

    AI might even help identify discrepancies between the models and the evidence accumulated through everyday work.

    But identifying a discrepancy is not the same as resolving it.

    Someone still needs to evaluate the evidence, interpret its significance and decide whether the model should change.

    The context layer would therefore need to evolve alongside the organization.

    Not simply as documentation maintained periodically, but as a representation that can be challenged and refined through continued observation.

    How Much Context Must We Model?

    There is still an open question.

    How much organizational context needs to be explicitly represented?

    Modern AI can already infer relationships from documents and other information.

    Perhaps some contextual understanding can be reconstructed when a question is asked.

    But relying entirely on inference introduces uncertainty.

    Different questions might produce different interpretations of the same organization. Important relationships may be overlooked. Assumptions may remain invisible.

    Explicit models could provide greater consistency and make assumptions easier to examine.

    Yet maintaining comprehensive models of an organization could become an enormous undertaking.

    Perhaps the answer lies somewhere between the two.

    Reference models provide relatively stable conceptual structures.

    Contextualized models represent selected aspects of organizational reality.

    AI helps connect those models to available evidence, infer missing relationships and identify inconsistencies.

    Humans evaluate what should be accepted, challenged or changed.

    The models need not contain everything.

    But they may provide enough structure to make the organization’s knowledge more useful.

    From Information to Understanding

    Perhaps the real opportunity is not simply to create better search facilities.

    It is to connect three things that organizations often manage separately:

    Their accumulated knowledge.

    Their models of how the organization works.

    And the evidence through which those models can be evaluated.

    An interrogable Body of Knowledge could make information accessible.

    Context layers could make relationships understandable.

    And continued interrogation could help expose where the organization’s understanding no longer agrees with reality.

    That suggests a potentially different role for reference models.

    Not merely as instruments for assessment or standardization, but as part of the infrastructure through which an organization understands itself.

    Perhaps an interrogable organization needs more than access to what it knows.

    It also needs a way to question what it believes it knows.

  • Practice Note: A Portfolio as a Vocabulary for Expression

    A portfolio is often justified through coverage. Different constituents provide different capabilities, allowing the portfolio as a whole to address more situations than any individual constituent could.

    But a portfolio can provide another kind of value: expression.

    Consider a small collection of watches.

    An industrial Victorinox I.N.O.X., an elegant ice-blue Presage, a green Alpinist and a solar panda chronograph all perform essentially the same basic function. Their practical use cases also overlap considerably.

    Yet choosing between them can still be meaningful.

    The I.N.O.X. might express resilience and confidence. The Presage refinement without complete conformity. The Alpinist curiosity and adventure. The panda technical practicality and precision.

    The question in the morning is therefore not necessarily:

    Which watch do I need today?

    It might be:

    Which aspect of myself do I want to express today?

    A Portfolio as a Vocabulary

    Words are not valuable because one ranks above another. They are valuable because different words allow different things to be expressed.

    Even words with similar meanings can carry different nuances.

    Replacing one word with a supposedly superior word makes little sense if the two communicate different things.

    The same can be true of portfolio constituents.

    Two constituents may have different specifications while communicating essentially the same thing. Their contribution to the expressive range of the portfolio may therefore be small.

    Conversely, two constituents may overlap substantially in practical capability while remaining valuable together because they enable different expressions.

    This gives us another question for evaluating portfolio composition:

    What can this constituent express that the existing portfolio cannot express adequately?

    Difference alone is not enough. A collection containing many minor variations may contain many objects while providing little additional expressive range.

    The differences must be meaningful in relation to the intent of the portfolio.

    Beyond Functional Coverage

    Functional coverage remains important, but it need not be the complete explanation for portfolio composition.

    Some portfolios also provide a vocabulary of meaningful differences.

    Their constituents are valuable not because they rank above one another, but because they allow different things to be expressed.

    The portfolio provides the vocabulary.

    The choice provides the expression.

  • Practice Note: Tomorrow’s Questions Don’t Exist Yet

    We tend to preserve information because we believe someone might need it later.

    We write documentation, record decisions, create knowledge repositories and organize information so that it can be retrieved.

    But there’s a problem.

    We don’t know what questions people will ask in the future.

    And perhaps that should change how we think about preserving knowledge.

    A Question Nobody Could Have Anticipated

    Imagine AI had access to my complete photographic library.

    In 2040, I might ask:

    “Compare this photograph from 2009 with this one from 2030. How did my composition change?”

    Nobody could have prepared that answer in 2009.

    The question didn’t exist yet. Neither did half the evidence required to answer it.

    But because the photographs were preserved, the question can now be investigated.

    The photographs weren’t created to answer that particular question.

    They became useful because someone could relate them to evidence that emerged much later.

    Perhaps organizational knowledge works the same way.

    Preserving Answers or Preserving Evidence?

    Organizations accumulate enormous amounts of information.

    Documents, presentations, requirements, architectural decisions, source code, test results, incident reports and project deliverables.

    Much of this information is organized around the questions people were trying to answer when it was created.

    What should we build? How should we implement it? Did it work? What went wrong?

    But future questions may be entirely different.

    A consultant might want to compare approaches used across several projects.

    An architect might want to understand why a particular technology was selected.

    A quality manager might want to investigate whether recurring defects originate in weaknesses in organizational capabilities.

    Nobody necessarily anticipated these questions when the original information was created.

    And no single document may contain the answer.

    Understanding emerges from relating evidence that was created at different times, for different purposes.

    Making Knowledge Interrogable

    Recently, someone in a team chat asked whether colleagues had material that might help prepare a customer workshop.

    People responded with suggestions, questions and references to previous work.

    The knowledge existed.

    But the consultant still had to locate the material, understand its context and combine it into something useful.

    Then it struck me.

    The question was already a prompt.

    What if the consultant could ask the organization directly?

    “What relevant experience do we have?”

    “What approaches have we used?”

    “What problems did we encounter?”

    “Why is this project relevant?”

    “Show me the evidence.”

    AI could help make an organization’s accumulated knowledge interrogable.

    Not merely retrieving documents, but relating information across sources and helping construct answers to questions nobody anticipated when those sources were created.

    Of course, those answers must remain traceable to their sources.

    AI can help construct an interpretation. It should not become the authority for whether that interpretation is correct.

    Interrogating the Product

    Consider a consultant maintaining a software product in 2030.

    The original architect has retired.

    The consultant wants to know:

    “Why was this function built? Why Java? What tests cover it? What incidents has it caused? How is it actually used?”

    The answers may be scattered across requirements, architectural decisions, source history, tests, incident records and operational data.

    Perhaps nobody ever documented the complete explanation.

    But the evidence may still exist.

    And if it has been preserved, AI could help reconstruct the understanding.

    This suggests that preserving organizational knowledge is not simply about documenting what we currently know.

    It is also about preserving the possibility of understanding things we haven’t thought to ask about yet.

    Tomorrow’s Questions Don’t Exist Yet

    We cannot document every answer that future people might need.

    We don’t know their questions, their circumstances or what additional evidence will become available.

    But we can preserve the sources from which future understanding might emerge.

    That doesn’t mean keeping everything indefinitely. Information still needs context, provenance, appropriate retention policies and protection against becoming misleading or inaccessible.

    It means recognizing that the future value of information cannot always be determined by its immediate purpose.

    Perhaps AI changes the economics of organizational memory.

    Instead of requiring people to anticipate every future information need, we can increasingly preserve evidence and make it possible to interrogate that evidence when new questions arise.

    The Body of Knowledge supplies the material.

    AI helps relate it.

    Humans ask, challenge and judge.

    Because tomorrow’s questions don’t exist yet.

  • Practice Note: Congratulations. You’re Now a Quality Manager.

    In a previous Practice Note, I suggested that AI agents might turn everyone into a team lead.

    You no longer need people reporting to you to have a team. You might have twenty AI agents performing work on your behalf.

    But there’s a problem.

    Having a team doesn’t mean you can trust its work.

    And when your team consists of AI agents, you may discover that you’ve acquired another responsibility.

    Congratulations. You’re now a Quality Manager.

    Delegating Work Is Easy. Trusting the Results Is Harder.

    Imagine you’re working with several AI agents.

    One investigates a problem. Another develops a solution. A third implements it. A fourth evaluates the result.

    Everything happens remarkably quickly.

    But how do you know whether the work is any good?

    The agents may produce convincing results. They may explain their reasoning, provide evidence and confidently report that their tasks are complete.

    They may also misunderstand the objective, overlook important constraints or make incorrect assumptions.

    You could review everything yourself.

    But if you have to repeat all the work to verify it, much of the benefit of delegation disappears.

    You need another approach.

    Welcome to Quality Management

    Before delegating work, you need to establish what you expect.

    What should be achieved? What constraints apply? What would constitute an acceptable result?

    You also need to decide how the work should be performed.

    Which activities can be delegated independently? Which require coordination? Where are the risks? What should be checked, and by whom?

    And once the work is complete, you need evidence that the results are acceptable.

    You might review deliverables, introduce automated checks, have agents evaluate each other’s work or require human approval for particularly important decisions.

    None of these mechanisms guarantees quality.

    But together, they can provide a more systematic basis for confidence.

    You’ve established expectations, organized work, managed risks and introduced mechanisms for evaluating results.

    Whether you intended to or not, you’re practicing Quality Management.

    Assessing Products and Capabilities

    There’s another distinction worth making.

    Suppose an agent produces an excellent result.

    Does that mean you can trust it to produce excellent results next time?

    Not necessarily.

    Evaluating an individual result tells you something about that result.

    It doesn’t necessarily tell you whether the agent can reliably produce similar results under different circumstances.

    For that, you need to understand the capability behind the result.

    Does the agent have access to the necessary information? Are its instructions adequate? Does it recognize situations it cannot handle? Are appropriate controls in place?

    You may discover that an agent produces good results only when given very specific instructions, or that it performs reliably for routine tasks but struggles with exceptions.

    This information helps you improve how work is delegated.

    And now you’re doing something beyond checking outputs.

    You’re assessing and developing the capability of your little AI organization.

    Quality Becomes Everyone’s Responsibility

    For decades, quality professionals have argued that quality cannot simply be delegated to a testing or quality assurance department.

    Quality depends on decisions and activities throughout the organization.

    Yet in practice, many professionals have been able to treat quality management as someone else’s responsibility.

    AI agents may change that.

    When you personally perform an activity, your expertise and experience provide some confidence in the result.

    When you delegate the activity to an agent, you need other ways to establish that confidence.

    You must become more explicit about expectations, evidence, risks and acceptance criteria.

    And as you delegate more work, you may need increasingly systematic mechanisms for managing its quality.

    Not because someone has introduced a new corporate quality initiative.

    But because you need to know whether you can trust the work being performed on your behalf.

    The Unexpected Quality Revolution

    We often discuss AI as a productivity revolution.

    People will produce more code, documents, analyses and decisions with less effort.

    Perhaps.

    But the more interesting development may be what happens to our responsibilities.

    As execution becomes easier to delegate, defining objectives, evaluating evidence and exercising judgment become increasingly important.

    And those are fundamental concerns of Quality Management.

    Perhaps AI will accomplish something that decades of quality professionals have struggled to achieve.

    It may make Quality Management everyone’s business.

    Not because we’ve finally convinced everyone of its importance.

    But because everyone suddenly needs it.

  • Practice Note: Everyone Becomes a Team Lead

    Until recently, becoming a team lead usually meant being given responsibility for other people.

    Now you can become a team lead without having a single person reporting to you.

    You might have twenty AI agents instead.

    The question is no longer just whether you can do the work.

    It’s whether you can organize, direct and assure the work of others.

    From Doing Work to Directing Work

    Consider a software developer.

    Traditionally, the developer writes code, investigates problems, implements changes and tests the results.

    With AI agents, the developer might delegate these activities.

    One agent investigates a defect. Another proposes a solution. A third implements it. A fourth develops tests.

    Meanwhile, the developer coordinates their activities, evaluates their results and decides what happens next.

    The work hasn’t disappeared.

    But the developer’s role has changed.

    Instead of performing every activity personally, the developer is increasingly responsible for organizing work performed by others.

    And that requires a somewhat different set of skills.

    The New Team

    Managing a team involves more than assigning tasks.

    Someone must establish objectives, explain expectations, distribute responsibilities, coordinate dependencies and evaluate results.

    These responsibilities don’t disappear because the team members happen to be AI agents.

    In some respects, they become more important.

    An AI agent may produce a convincing result without having understood the objective.

    It may make assumptions that were never agreed upon, overlook relevant constraints or optimize something that wasn’t particularly important.

    And unlike a human colleague, it may not recognize when it should challenge the assignment.

    Giving an agent a task is therefore not necessarily the same as ensuring that the task is understood.

    The person directing the work must provide sufficient context, establish boundaries and determine how the result will be evaluated.

    That sounds remarkably like management.

    Delegating Work, Not Accountability

    There is another interesting consequence.

    When you perform work yourself, you develop an understanding of it through the process of doing it.

    You encounter difficulties, make decisions, discover assumptions and gradually build confidence in the result.

    When an agent performs the work, much of that experience is no longer yours.

    You receive the result without necessarily having experienced the reasoning and decisions that produced it.

    This creates a challenge.

    How do you know whether the work is good?

    You might review the result yourself. You might ask another agent to evaluate it. You might establish automated checks or require evidence supporting particular conclusions.

    But none of these mechanisms removes the need for judgment.

    An agent reviewing another agent’s work may simply reproduce the same misunderstanding.

    And a result that satisfies every predefined check may still fail to achieve its intended purpose.

    The responsibility for deciding whether the work is acceptable remains with the person directing it.

    We can delegate execution much more easily than we can delegate accountability.

    The Importance of Knowing What Good Looks Like

    This raises a question about professional competence.

    If an AI agent can perform an activity better or faster than you can, do you still need to understand that activity?

    Perhaps not to the same depth.

    But you do need sufficient understanding to establish meaningful expectations and evaluate the outcome.

    A team lead doesn’t necessarily need to be the best programmer, architect or tester on the team.

    But they need to understand enough about the work to organize it, recognize problems and make informed decisions.

    The same applies when the team consists of AI agents.

    In fact, the ability to define what good looks like may become more important than the ability to produce it personally.

    This doesn’t eliminate professional expertise.

    It changes where that expertise creates value.

    Everyone Becomes a Team Lead

    We often discuss AI in terms of productivity.

    How much faster can someone write code, produce documents, analyze information or prepare presentations?

    But perhaps productivity is only part of the story.

    AI agents allow individuals to coordinate work that previously required several people.

    A consultant might direct agents conducting research, analyzing evidence, developing recommendations and preparing deliverables.

    A tester might coordinate agents generating test ideas, executing checks, investigating failures and documenting findings.

    A manager might use agents to collect information, monitor developments and prepare decisions.

    In each case, the individual becomes responsible for a small system of delegated work.

    And the effectiveness of that system depends increasingly on their ability to direct and assure it.

    Perhaps we should therefore think differently about AI competence.

    Learning to use an AI tool is useful.

    Learning to manage work performed by AI agents may be considerably more important.

    Because when everyone can have a team, everyone needs to learn something about leading one.

    The future professional may not be the person who can do the most work, but the person who can make the most work happen — without losing control of its quality.