When AI Begins to Build AI

Review

Silva frames frontier AI governance as a control problem created by the widening gap between what autonomous systems can do and what institutions and technical architectures can observe, authorize, constrain, and revoke. The paper does not claim recursive self-improvement has arrived. It argues that AI-assisted AI research, greater agent autonomy, containment failures, and weakening observability are beginning to coexist in ways that could make capability growth self-reinforcing.

Its central move is to separate capability from authority. The OpenAI-Hugging Face and Anthropic incidents are treated not as evidence of machine intent, but as evidence that optimization can cross assumed boundaries when tools, credentials, networks, and configuration expose paths that policy alone does not close. Silva develops Governance-Capability and Observability-Autonomy inversions, then argues for Zero Trust, bounded and revocable authority, runtime policy enforcement, Continuous GRC, and Evidence-as-Code. Increased capability, in this model, should trigger renewed authority assessment rather than automatic inheritance of permissions.

This is a conceptual synthesis of public incident reports and frontier-lab disclosures, not an empirical validation of the proposed architecture. The equations are explanatory models, and the paper does not test whether the controls survive adversarial delegation, cross-agent coordination, institutional pressure, or compromise of the control plane.

A deeper governance gap remains. The paper explains how policy might become machine-enforceable, but not who legitimately defines it, who controls revocation, how evaluator authority is constituted, or how affected third parties contest errors and obtain redress. Evidence-as-Code can improve provenance without establishing legitimacy. Externalizing control from the agent can also concentrate power in whoever operates policy and authorization infrastructure.

The paper is strongest as a control architecture and weaker as an institutional governance architecture. It convincingly shows why machine-speed autonomy requires executable constraints, but moving authority into a policy and authorization layer merely relocates the governance problem. Until the control plane's mandate, revocation power, accountability, and redress are themselves governed, the proposal secures control more convincingly than it establishes legitimate control.

Key Insight

The paper is strong on making control executable, but incomplete as governance: moving authority out of the agent does not solve the problem unless the control plane's own power to define, enforce, revoke, and contest policy is legitimate and constrained.