Corrigibility

What Is Corrigibility in AI?

Corrigibility is an AI safety property that enables a software agent to accept authorized correction, interruption, and goal updates. A corrigible system cooperates when operators pause, inspect, modify, or redirect it, rather than resisting to preserve its current objective. In practice, it means human control remains meaningful throughout an agent’s operation, a central concern in AI corrigibility research.

How a Corrigible AI System Works

Corrigibility depends on controls that are both visible and enforceable. An agent must not merely promise to stop. The surrounding software must be able to make it stop.

1. Define approved operators and secure control channels.
2. Receive a pause, update, or inspection request.
3. Verify the requester’s authority and the command’s scope.
4. Stop work safely or apply the approved change.
5. Record its state, actions, and any unfinished tasks.
6. Resume only after an authorized instruction permits it.

Corrigibility vs. Reliability, Alignment, and Access Control

These concepts support safer software, but they answer different questions.

Benefits of Corrigibility for AI Applications

Corrigible design makes automation easier to supervise and revise.

• Supports safer iteration when requirements, prompts, or policies change.
• Makes automated actions more reversible during incidents.
• Gives developers clearer oversight of agent behavior.
• Improves incident response through pause controls and recorded state.
• Allows controlled updates without treating the agent’s first goal as permanent.

Practical Limits and Challenges

Corrigibility is valuable, but it does not make an application correct by itself.

• Ambiguous instructions can lead to an authorized but harmful action.
• Weak authentication can let the wrong person issue a stop or update command.
• Hidden tool calls and incomplete logs make behavior difficult to inspect.
• Conflicting commands require clear precedence rules.
• A compliant agent can still generate incorrect output or follow a flawed request.

How to Design More Corrigible Software Agents

Build control into the architecture, testing process, and permissions model.

1. Apply least-privilege permissions so agents can access only necessary tools.
2. Add explicit stop controls that interrupt active tasks and queued work.
3. Require approval gates for high-impact actions such as publishing or deleting data.
4. Keep audit logs of commands, tool calls, state changes, and outcomes.
5. Test in sandboxes and verify that interrupted tasks can recover safely.
6. Run tests that confirm authorized updates and shutdown requests are obeyed.

Corrigibility in Emergent

When building agentic software, corrigibility starts with clear requirements, controlled integrations, and repeatable testing. Emergent helps teams create and iterate on AI-powered web and mobile products through its AI agent builder and AI app builder. Clear workflows, permission boundaries, and testable changes help developers retain human control as an application evolves.