Accountability Architecture

Proposed reform — not current law

A proposed UK accountability architecture

Eight measures built on levers government already holds. Procurement is the first enforcement layer; criminal law is the backstop, not the plan.

PROPOSED REFORM — NOT CURRENT LAW. These measures describe what the editors argue procurement and statute should require.

1 · Auditable procurement

High-risk contracts require the evidence base up front: design documentation, data provenance, model and version records, operating-envelope definitions and access for independent testing.

  • What is not contracted for will not exist when it is needed.
  • Documentation scales with risk and function — not maximal paperwork for everything.
2 · Measurable human control

Test review time, information quality, uncertainty presentation, authority to refuse and performance at expected operational tempo — as acceptance conditions and in service.

  • A human in the loop is a claim; these are its measurements.
3 · Continuous legal review

Article 36 review plus defined thresholds for material software change, with re-testing and recertification when they are crossed, and deployed-version identification throughout.

4 · Continuing supplier duty

When credible evidence of harmful out-of-envelope use emerges: preserve and assess, report and restrict, escalate or suspend through defined legal process, cooperate with investigation and remedy.

  • Safeguarded so suppliers do not become private foreign-policy authorities: pre-agreed triggers, expedited regulator or court process, an emergency route for imminent grave harm.
5 · Responsibility allocation

Contracts identify, before deployment, which actor controls data, updates, deployment settings, incident reporting and remedial action — so investigation allocates rather than excavates.

6 · Independent assurance

A technically competent assurance function that neither the procuring team nor the supplier controls, with access, clearances and the authority to require re-review.

7 · Evidence and remedy

Tamper-evident logs, protected records, whistleblower safeguards, incident disclosure calibrated to security needs, and meaningful routes to investigation and victim remedy.

8 · Graduated enforcement

Remediation, contract suspension, fines, debarment, senior-manager consequences — and criminal investigation where the relevant evidential and fault thresholds are met.

  • Procurement is the first enforcement layer. Criminal law is a backstop, not the only accountability mechanism.

National assurance and international verification

PROPOSED REFORM — this agency does not exist and is not presented as inevitable.

An IAEA for high-risk military AI functions — but not a copy of it

The nuclear analogy earns its place for four things: independent declarations, inspection practice, common technical methods and confidence-building. Its limits are equally instructive: software can be copied, updated and hidden; military AI is dual-use; there is no scarce material to count; non-state actors matter; and lawfulness usually turns on context of use, not possession.

The IAEA asks whether declared nuclear material and facilities match reality. This agency would ask what decisions a system is permitted to make, in which conditions, which version was deployed, and whether a harmful use can be reconstructed.

The remit would be limited to defined high-risk functions:

  • target selection and prioritisation;
  • engagement without a new human decision;
  • lethal swarm coordination;
  • machine-speed coupling of warning, target generation and response;
  • material changes to certified systems.
Bounded possible powers:
  1. minimum technical standards for testing, logging, change control and incident reporting;
  2. confidential registration of systems, envelopes and deployed versions;
  3. review of national assurance and weapons-review processes against the treaty minimum — not their replacement;
  4. accreditation and audit of independent test laboratories;
  5. agreed routine inspections and tightly governed challenge inspections;
  6. incident investigation with protected access to logs and version histories;
  7. export and end-use verification support;
  8. sanitised public findings;
  9. referral of evidence to competent national and international authorities.
Limits, stated plainly
  • non-parties and proxies remain outside the regime;
  • secrecy and covert software limit verification;
  • unequal capacity risks making assurance a rich-state monopoly unless assistance is funded;
  • certification can drift into political legitimation of approved systems — the regime's own standing hazard.

SOURCES [21] IAEA safeguards and verification, International Atomic Energy Agency[5] ICRC position on autonomous weapon systems, International Committee of the Red Cross

For government and Parliament

“Fit for purpose” must include “capable of lawful use.” The Government–industry partnership is advancing faster than the architecture needed to test, trace and govern it. The answer is not to stop innovation, or to outsource legal judgment to suppliers. It is to build accountability into procurement: measurable human control, review of material software changes, auditable evidence, continuing supplier duties and scrutiny independent of both supplier and procurer. These safeguards protect civilians, the state, responsible companies and the legitimacy of UK defence policy.

⚖ Legal content on this site is editorial. Reforms marked PROPOSED REFORM are recommendations, not current law. All legal content requires review by qualified UK public-law, IHL, export-control and international-criminal-law counsel before publication.