How it works
AI products spread decisions across more hands than conventional software: the model provider, the prompt author, the evaluation team, the policy team, the vendor whose agent is called further down the line. Each owns a part. Nobody owns the outcome.
The shape is old. Thirty years ago Helen Nissenbaum named four barriers to accountability in computerised systems: the problem of many hands, the problem of bugs, blaming the computer, and software ownership without liability. Generative systems raise all four at once. Work that belongs to no one falls into the gaps between roles, and everyone assumes someone else has checked. When a harm surfaces, the investigation finds a chain of reasonable steps and nobody who decided. The person affected has no one to answer them, and the team has no one with the standing to change course.
The tooling records the wrong thing. A 2026 study of the artefacts around agentic coding tools found that providers’ terms attach duty to the output as a class, while the platforms log events, and no artefact anywhere records that the verification the terms require actually took place. Responsibility, the authors write, ‘is settled in two layers that do not refer to each other’.
Field marks
- Asked who decided how a feature behaves, the team names a model, a vendor or a meeting.
- Harms are reported to a channel, not to a person.
- The decision log records what shipped, not who chose it.
- Oversight is assigned to people without the authority, or the means, to stop a release.
Where it shows up
Interviewing 26 practitioners from 19 organisations, Rakova and colleagues found a lack of accountability common, and that ‘Responsible AI work falls through the cracks due to role uncertainty’. One interviewee’s version: ‘we run into “Whose job is this?”’. Developers building agents in 2026 prioritised product and business risks before downstream ones and ‘lacked mature controls for containing agentic risks’.
When a company tried to say the model decided, a tribunal declined to hear it. Air Canada’s chatbot told a passenger he could claim a bereavement fare after travelling; the airline refused, and argued before British Columbia’s Civil Resolution Tribunal that the chatbot was ‘a separate legal entity that is responsible for its own actions’. The tribunal called that ‘a remarkable submission’ and held that ‘It should be obvious to Air Canada that it is responsible for all the information on its website.’ The award was C$812.02; the principle travels further than the sum. The UK Competition and Markets Authority put the same principle into guidance in March 2026: a business is ‘responsible for what an AI agent does in the same way you are responsible for what an employee does’, even if someone else built or supplies the agent.
Frameworks and law ask for an owner. The NIST AI Risk Management Framework asks that roles and responsibilities for managing AI risk ‘are documented and are clear to individuals and teams throughout the organization’, and that executive leadership take responsibility for decisions about that risk. The EU AI Act will require deployers of high-risk systems to ‘assign human oversight to natural persons who have the necessary competence, training and authority’. The Digital Omnibus on AI, in force from 27 July 2026, moved that duty to 2 December 2027 for most listed uses and 2 August 2028 for systems embedded in regulated products.
Designed out
- Name an owner. Give every consequential decision a person, not a team or a tool, with the authority to change it.
- Write decisions down where they are made. Record the choice, the owner and the alternatives considered, in the same place the work happens.
- Give oversight authority. Whoever owns the outcome can stop the release.