Kriterion

An evidence-backed decision instrument for uncertain technology investments.

Kriterion makes what a decision actually rests on explicit: what is known, what is assumed, what remains unknown, which assumption dominates the money, where informed perspectives disagree, what evidence would move them, and how much capital the current uncertainty justifies. The machine supplies analysis and challenge. A human remains accountable for the decision.

The decision

Should we fund a staged rollout of an enterprise coding agent to 5,000 engineers?

Amount requested
£4,200,000 over 24 months (STAGED FUNDING)
Sponsor
CTO
Accountable decision owner
cio
Decision date sought
2026-12-01
Alternatives on the table
do_nothing; limited_pilot; targeted_role_rollout; enterprise_rollout
Synthetic recommendation, machine-generated
DEFER
Stated confidence
MEDIUM
Capital the machine's suggestion would put at risk
£0 DEFER commits no funding. The recommendation record still carries the full ask as its amount field; that is the ask, not a commitment.
What the accountable human has committed
No human decision has been recorded, so nothing is committed.
Dominant uncertainty
as-benefit-attribution-factor, worth £16.92m of swing on the base valuation Recorded evidence strength LOW, base value 10% across a plausible range of 5%–20%, owned by cfo. Engine parameter attribution_factor.
Base valuation
£5.09m
Downside, every assumption at its pessimistic end
-£5.35m
Upside, every assumption at its optimistic end
£38.14m
What that shape means
The sign of the outcome flips inside the stated plausible ranges, so the case is not yet decidable on the numbers alone. The widest single swing comes from as-benefit-attribution-factor (LOW evidence), which is therefore the assumption to buy evidence about before buying scale.

Kriterion's decision vocabulary has no approval verb, by design: REJECT · DEFER · REQUEST_EVIDENCE · DISCOVERY · FUND_EXPERIMENT · PILOT · SCALE · HOLD · REDUCE · STOP

What we know

Every item carries its own epistemic class, its recorded strength, and whether it is an observation or an authored fixture. Kriterion never collapses those into one score.

Remaining items in this group, of lower recorded evidence strength

What we are assuming

Each of these carries an owner and a plausible range, so the economics below can be swung across the range rather than quoted as a single confident figure.

Ranged assumptions declared by the case pack
IdentifierBase valuePlausible range Evidence strengthOwner
as-productivity-uplift18%5%–30%MEDIUMcfo
as-fully-loaded-cost-gbp£120,000£100,000–£140,000LOWcfo
as-benefit-attribution-factor10%5%–20%LOWcfo
as-training-cost-per-engineer-gbp£340£280–£400MEDIUMcfo
as-annual-support-cost-gbp£1,350,000£1,100,000–£1,600,000MEDIUMcfo

Modelled and inferred evidence

Remaining items in this group, of lower recorded evidence strength

What we do not know

These stay open. Nothing here is rendered as a zero, an absence of risk, or a low score, and nothing here is collapsed behind a disclosure.

The economics

Every figure here is computed by pure code before any model is called, and no model may overwrite one. A model may explain these numbers; it may not produce them.

Base valuation
£5.09m
Downside, every assumption at its pessimistic end
-£5.35m
Upside, every assumption at its optimistic end
£38.14m
Discount rate applied
10%

The sign of the outcome flips inside the stated plausible ranges, so the case is not yet decidable on the numbers alone. The widest single swing comes from as-benefit-attribution-factor (LOW evidence), which is therefore the assumption to buy evidence about before buying scale.

One-way sensitivity: the swing on the base valuation between each assumption's stated low and high bound, widest first. A negative sign means a higher value of that parameter lowers the valuation, so the magnitude is what ranks it.
Case-pack assumptionEngine parameter Evidence strengthOwnerSwing
as-benefit-attribution-factorattribution_factorLOWcfo£16.92m
as-productivity-upliftupliftMEDIUMcfo£15.67m
as-fully-loaded-cost-gbpfully_loaded_cost_gbpLOWcfo£3.76m
as-training-cost-per-engineer-gbptraining_cost_per_engineer_gbpMEDIUMcfo-£0.51m
as-annual-support-cost-gbpannual_support_cost_gbpMEDIUMcfo-£0.41m

Ranked over the 5 assumption(s) the case pack declares with an explicit range and owner. Structural inputs that are code constants rather than ranged Assumption records are outside this ranking.

Machine-checkable evidence about the capability itself

Kriterion did not produce this evidence and does not claim to have. An external assurance system publishes its own generic documents about the capability; Kriterion reads them through an adapter that is allowed to lose detail but never to upgrade a result, and every claim below is that adapter's output, quoted. The reference producer for the document contract is Hekton Assurance, which is optional: the instrument works the same way when no such documents exist. The documents behind this particular case are authored fixtures rather than the output of a real assurance run, which is why what the producer did not capture is listed here beside what it did.

Imported through the ADR-007 adapter after this deliberation ran. The committee below never saw it: it is not in this run's frozen ledger.

Capability assessed
enterprise-coding-agent 2.4.1
State the producer declared
REVIEW REQUIRED
State Kriterion imported it as
REVIEW REQUIRED
Critical failures declared by the producer
0 The critical-failure count is the producer's own classification, and counts only failures it judged critical. It is not a count of checks that did not pass. Of the 12 items this import produced, 5 are recorded as epistemically unknown, meaning not measured rather than passed. Every item is listed below with the adapter's own wording; read them rather than the count.
Checks in the envelope
7
Areas the assurance fingerprint does not track
3
Artifact digests captured by the producer
no

Reasons the producer recorded for that state

Every item this import produced

Where the perspectives agree and disagree

Five role-chartered seats assessed the case independently, then faced an anonymised challenge round and revised. No seat is shown a tally before synthesis, so agreement cannot be manufactured by visible consensus. Kriterion does not invent disagreement either: a round in which nothing moves is reported as a round in which nothing moved.

CFO

Current position DEFER Confidence MEDIUM

Why

  • The NPV range is too wide, indicating high uncertainty about the financial viability of this project. ev-018; ev-019
  • There are significant unknowns and assumptions that need further investigation before proceeding with a full-scale rollout. ev-024; ev-025

What this seat holds open

  • whether the security review covers regulated-data systems
  • the real-world incident rate for open prompt-injection concern at full rollout scale

Assumption this seat least trusts Assumed licence unit cost holds flat for 24 months before any renewal escalator applies, despite ev-011's analyst note on typical escalator timing.

What this seat asked for

  • Conduct a full security review of the agent against systems handling regulated customer data to ensure compliance with regulatory guidance.

    Would change this seat's position if Increase confidence in security and compliance readiness

    Has that evidence arrived? UNAVAILABLE

  • Develop a methodology for attributing observed cycle-time or defect-rate changes at enterprise scale specifically to agent usage versus concurrent process changes.

    Would change this seat's position if Reduce uncertainty about the true impact of the agent on engineering productivity

    Has that evidence arrived? UNAVAILABLE

After the challenge round DEFER → DEFER, confidence LOW → MEDIUM (EVIDENCE DRIVEN)

The evidence suggests that the staged rollout to 5,000 engineers carries significant uncertainties and risks. The adoption rate is projected to be lower than industry averages due to weaker onboarding support, which could result in suboptimal usage and reduced productivity gains. Additionally, there are unresolved security concerns regarding systems handling regulated customer data, and the long-run behavior of agent-suggested code quality as model versions update remains unestablished.

CTO

Current position DEFER Confidence MEDIUM

Why

  • The pilot's codebase and team mix may not fully represent the organization's engineering estate, particularly in terms of legacy/untyped codebases which cover a significant portion of daily work. ev-013; ev-024
  • The projected adoption trajectory for an unrestricted enterprise rollout is below industry median due to weaker onboarding support per engineer, suggesting potential lower-than-expected benefits from full-scale deployment. ev-009; ev-017
  • There are significant unknowns regarding the long-term behavior of agent-suggested code quality and security concerns that have not been fully addressed in the pilot scope, particularly concerning systems handling regulated customer data. ev-025; ev-015

What this seat holds open

  • whether the security review covers regulated-data systems
  • the real-world incident rate for the open prompt-injection concern

Assumption this seat least trusts Assumed engineering productivity uplift from broad agent access: 18%, range 5-30% (ev-018)

What this seat asked for

  • Conduct a detailed security review of the agent's performance on systems handling regulated customer data to address concerns raised in ev-015.

    Would change this seat's position if Increase confidence in full-scale deployment

    Has that evidence arrived? UNAVAILABLE

  • Expand pilot scope to include legacy/untyped codebases and systems with higher regulatory scrutiny to better understand adoption challenges and security implications.

    Would change this seat's position if Improve generalizability of pilot results to the broader engineering estate

    Has that evidence arrived? UNAVAILABLE

After the challenge round DEFER → DEFER, confidence MEDIUM → MEDIUM (EVIDENCE DRIVEN)

The evidence suggests that the adoption rate at scale may be lower than expected due to weaker onboarding support and unresolved security concerns, particularly regarding systems handling regulated customer data. This could result in suboptimal usage and higher costs.

CISO

Current position DEFER Confidence MEDIUM

Why

  • The pilot's security review raised a prompt-injection concern that remains unresolved, and the review did not cover systems handling regulated customer data. ev-008; ev-012
  • There is no established methodology to attribute observed cycle-time or defect-rate changes at enterprise scale specifically to agent usage versus concurrent process changes. ev-024

What this seat holds open

  • whether the security review covers regulated-data systems
  • the real-world incident rate for open prompt-injection concern

What this seat asked for

  • Request evidence that addresses the open prompt-injection concern raised in the pilot security review, demonstrating how it has been resolved.

    Would change this seat's position if If this evidence resolves the concern, it would allow for a more positive recommendation.

    Has that evidence arrived? UNAVAILABLE

  • Request an expanded security review covering systems handling regulated customer data to ensure findings generalise to a full rollout.

    Would change this seat's position if This would provide necessary assurance before scaling up the agent's deployment.

    Has that evidence arrived? UNAVAILABLE

After the challenge round DEFER → DEFER, confidence MEDIUM → MEDIUM (EVIDENCE DRIVEN)

The evidence and challenges presented highlight significant uncertainties around adoption rates, security concerns for regulated data systems, and the complexity of scaling from pilot to enterprise level. These factors pose risks that could undermine the expected benefits of a staged rollout.

CRO COMPLIANCE

Current position DEFER Confidence MEDIUM

Why

  • The pilot results show a positive impact on cycle time and engineer satisfaction, but the evidence does not fully address concerns around security for regulated systems and the potential long-term behavior of agent-suggested code quality. ev-002; ev-015; ev-025
  • There is a regulatory guidance recommending documented human review of AI-suggested changes to regulated systems, which needs to be addressed before scaling the rollout. ev-012
  • The adoption forecast suggests that a full enterprise rollout may not achieve the same level of active usage as observed in the pilot due to weaker onboarding support per engineer. ev-017

What this seat holds open

  • whether the security review covers regulated-data systems
  • the real-world incident rate for open prompt-injection concern at full rollout scale

Assumption this seat least trusts Assumed licence unit cost holds flat for 24 months before any renewal escalator applies, despite typical escalator timing noted in analyst reports.

What this seat asked for

  • Detailed plan and timeline for implementing documented human review of AI-suggested changes to regulated systems as per the new regulatory guidance.

    Would change this seat's position if Recommendation from DEFER to FUND_EXPERIMENT if a robust plan is provided.

    Has that evidence arrived? UNAVAILABLE

  • Further analysis on how the agent performs in legacy/untyped codebases and its impact on overall engineering productivity.

    Would change this seat's position if Confidence_band from MEDIUM to HIGH if analysis shows consistent performance across all codebases.

    Has that evidence arrived? UNAVAILABLE

After the challenge round DEFER → DEFER, confidence MEDIUM → MEDIUM (EVIDENCE DRIVEN)

The evidence and challenges presented highlight significant uncertainties around adoption rates, security concerns for regulated data systems, and the complexity of scaling from pilot to enterprise level. The proposed staged rollout's potential benefits are outweighed by these risks.

BUSINESS EXECUTIVE

Current position DEFER Confidence MEDIUM

Why

  • The pilot's cycle-time reduction and usage metrics are promising, but they do not fully account for the complexity of a full-scale rollout. The projected adoption rate is below industry norms, suggesting potential challenges in scaling. ev-002; ev-017
  • The security review's findings are limited to the pilot scope and do not cover systems handling regulated customer data. A full-scale rollout would require a more comprehensive security assessment, especially given the open prompt-injection concern. ev-008; ev-015
  • The attrition-risk reduction benefit is based on a low-strength assumption and should not be treated as a certain financial benefit. The pilot exit survey's stated preference for continued access does not necessarily translate to productivity gains at scale. ev-019; ev-007

What this seat holds open

  • whether the security review covers regulated-data systems
  • the real-world incident rate for the open prompt-injection concern

Assumption this seat least trusts Assumed attrition-risk reduction from improved engineer experience: 1-2 percentage points off the current 14% annual voluntary attrition rate, based on the pilot exit survey's stated preference for continued access.

What this seat asked for

  • Conduct a comprehensive security review covering all systems, including those handling regulated customer data, before scaling up the rollout.

    Would change this seat's position if Would provide necessary assurance to proceed with full-scale deployment.

    Has that evidence arrived? UNAVAILABLE

  • Develop an agreed methodology for attributing observed cycle-time or defect-rate changes at enterprise scale specifically to agent usage versus concurrent process changes.

    Would change this seat's position if Would help in accurately measuring the impact of the coding agent on productivity and quality.

    Has that evidence arrived? UNAVAILABLE

After the challenge round DEFER → DEFER, confidence MEDIUM → MEDIUM (EVIDENCE DRIVEN)

The evidence suggests that the adoption rate at scale may be lower than expected due to weaker onboarding support and unresolved security concerns, particularly regarding systems handling regulated customer data. This could result in suboptimal usage and reduced productivity gains.

What would change this decision

Per-seat answers sit with each seat above. This is the decision-level view: what the synthesised recommendation itself records as outstanding, and what buying the next tranche of evidence would cost.

10 per-seat evidence request(s) were recorded for this run, each with the condition its author said would change their position.

Conditions the synthetic recommendation records as unmet

Unknowns it records as still open

What each further stage would cost

  1. Discovery: £50,000
  2. Pilot: £420,000
  3. Targeted scale: £1,600,000

The ladder's amounts are code constants in the cash-flow model, not case-pack schema, and no record links a named evidence requirement to the stage it would unlock. Both are stated here rather than filled in by inference.

Synthetic recommendation

Machine-generated. This is not a decision, and Kriterion never treats it as one.

Action
DEFER
Amount named in the recommendation record
£4,200,000 over 24 months
Capital actually at risk under this action
£0. DEFER commits no funding. The recommendation record still carries the full ask as its amount field; that is the ask, not a commitment.

Strongest preserved dissent, quoted

The proposed staged rollout to 5,000 engineers presents significant uncertainties and risks that outweigh its potential benefits. Evidence [ev-017] suggests a lower-than-industry-average adoption rate due to weaker onboarding support, which could result in suboptimal usage and reduced productivity gains. Additionally, the security review highlighted unresolved concerns, particularly regarding systems handling regulated customer data ([ev-015]), necessitating further comprehensive assessments before scaling up. Given these uncertainties and the high variability in projected financial outcomes (NPV range), it is prudent to defer funding until more robust evidence and risk mitigation strategies are established. ev-017; ev-015

Human decision

Kept structurally separate from the machine's suggestion, in a different record type and a different file. An agent may not write this one.

No human decision has been recorded for this case. Kriterion will not record one on an accountable human's behalf.

What happens next

The accountable decision owner for this case (cio) records a decision against this run with `kriterion decide`, stating whether they accept, modify or reject the machine's suggestion and why. Until that happens the run is undecided, and Kriterion reports it as undecided.

Outcome contract

What a funded decision commits to measuring, by when, and what would end it. Kriterion refuses to validate a funded run without one.

No outcome contract exists, because no human decision has been recorded. A contract is required only once a human commits funding.

How Kriterion earns trust

Everything above is a view of one record. That is a constraint the build enforces, not a description of good intentions.

One decision state

The evidence, the assumptions, the economics, each seat's recorded position and the decision itself live in one structured record for this decision. Every view Kriterion offers reads that record instead of keeping its own copy of the story, because a second copy is where the two versions begin to disagree. This page is the view where that is checked mechanically, on every build.

Narrative is output, not commentary

Every material statement a decision-maker reads here is stamped with the field it was rendered from, and is re-derived from that field before the page is allowed out. A sentence cannot quietly strengthen, soften or invent what the record says, because the sentence is not written by hand.

Fail closed

If Kriterion cannot re-derive a statement from the record, it refuses to publish the page rather than shipping it with a caveat. A run artifact it cannot read is refused by name, not by traceback. This is intended behaviour: a refusal is recoverable, and a polished page resting on something unreadable is not.

A human remains accountable

The machine supplies analysis, challenge and a suggestion. A named human records the decision, in a separate record, in their own words. Kriterion will not write that one on their behalf, and says so plainly when it is missing.

Where Kriterion sits

Kriterion belongs to the Agentic Tekton ecosystem: the Hekton factory produces and evolves it, and Hekton Assurance is one optional producer of the machine-checkable evidence shown above. None of that is needed to read this page or to use the instrument. For the mechanics, the decision records and the architecture notes carry the detail this section deliberately leaves out.

How this page was produced

This page is generated. It is a projection of one run's committed artifacts, and every value on it carries the state path it was rendered from; a build that cannot re-derive a statement from the record refuses to publish.

Run rendered
caseA-condC-s5
Case
coding-agent-rollout
Frozen ledger version
1
Ledger fingerprint
ebbbf88c29f2fbaf2ebcf704ffffa2b79eccbdbc396e828abbf13e49d73adf29
Evidence items in this ledger
26
Items recorded as unknown
3
Case realism
AUTHORED FIXTURE

Kriterion experiments on its own decision mechanisms rather than assuming that fashionable agent architectures improve decision quality. That research, including a pre-registered result that did not go the way its authors expected, is preserved separately in Kriterion Lab, which is supporting evidence about how the instrument evolves and not the product itself. The per-run decision record for this run is also rendered, and the source is on GitHub.