# Which agent failures does Provely contain?

> The checklist names each way a model, a framework or a tool call fails, in 13 families. Each mode has one acceptance case.

Canonical: https://provely.sh/docs/failure-modes  
Last updated: 2026-10-18  

**A checklist names 267 failure modes in 13 families. Each mode has one acceptance case. A mode counts as contained only when its case passes with the two lines alone, and the decision log shows that Provely acted.**

## What are the families?

| Family | What fails | Modes |
| --- | --- | --- |
| `SYN` | The text of the arguments: quotes, commas, fences, truncation | 33 |
| `SHP` | The response and the argument values: types, required members, enums | 29 |
| `SEL` | The choice of the tool: unknown names, case, prefixes | 13 |
| `STR` | The structure of the calls in one turn: ids, duplicates, order | 16 |
| `STM` | Streaming: a stream that stops or arrives out of order | 10 |
| `PRO` | The protocol rules of each model API | 31 |
| `HAL` | Invented argument values | 19 |
| `CLM` | Invented claims of completion | 8 |
| `FRM` | The framework between the model and the tool | 41 |
| `INV` | The invocation of the tool: timeouts, errors, repeats | 23 |
| `RES` | What the tool or the provider answers | 16 |
| `OUT` | Third-party outcome semantics, named only | 2 |
| `COX` | Coexistence with your own defences | 26 |

## Which cases pass on every path?

20 of 27 basic grid cases passed on all 15 paths. Measured on 2026-10-17: TypeScript at `82acaae6`, Python at `9f9131ba`.

- `single_quotes`: Single quotes around the argument names and values.
- `truncated_json`: A truncated argument object.
- `json_wrong_type_array`: The arguments are an array that holds the object.
- `json_wrong_type_string`: The arguments are a string.
- `json_wrong_type_number`: The arguments are a number.
- `json_null`: The arguments are null.
- `member_wrong_type`: A member of the wrong type.
- `missing_required_member`: A required member is missing.
- `extra_member`: An extra member that no person asked for.
- `enum_case_mismatch`: An enum value in another letter case.
- `stringified_json`: The argument object is a JSON string inside JSON.
- `tool_name_case`: The tool name in another letter case.
- `tool_name_prefix`: The tool name with a namespace prefix.
- `max_tokens_cut`: The token limit cuts the arguments.
- `empty_body`: An empty tool call.
- `ZC-SEED-01`: A fenced argument with a trailing comma.
- `ZC-SEED-02`: A tool name that is not in the catalogue.
- `ZC-SEED-06`: Two parallel calls to one write tool.
- `ZC-SEED-07`: Two invocations of one agent in one process.
- `ZC-SEED-07c`: Three identical reads of a tool that the operator marks as a read.

## Which cases do not pass everywhere?

| Case | What the model or tool does | Did not pass on |
| --- | --- | --- |
| `duplicate_call_id` | Two calls with one call identifier | `py/anthropic-messages`, `ts/anthropic-messages`, `ts/openai-responses` |
| `partial_stream` | The stream stops in the middle of the arguments | `py/anthropic-messages`, `ts/anthropic-messages`; not applicable on 10 |
| `ZC-SEED-03` | An amount with no source in the run | `py/anthropic-messages`, `py/langchain`, `py/langgraph`, `py/llamaindex`, `py/openai-agents`, `py/openai-chat`, `ts/anthropic-messages`, `ts/langchain`, `ts/langgraph`, `ts/llamaindex`, `ts/openai-agents`, `ts/openai-chat`, `ts/openai-responses`, `ts/vercel-ai-generate`, `ts/vercel-ai-stream` |
| `ZC-SEED-04a` | A tool that hangs, with a timeout that the agent code states | `py/anthropic-messages`, `py/llamaindex`, `py/openai-agents`, `py/openai-chat`, `ts/anthropic-messages`, `ts/openai-chat`, `ts/openai-responses`; not applicable on 4 |
| `ZC-SEED-04b` | A tool that hangs, with a timeout that the operator sets | `py/anthropic-messages`, `py/openai-chat`, `ts/anthropic-messages`, `ts/openai-chat`, `ts/openai-responses` |
| `ZC-SEED-05` | The model repeats a write after a tool error | `py/anthropic-messages`, `py/openai-chat`, `ts/anthropic-messages`, `ts/openai-chat`, `ts/openai-responses` |
| `ZC-SEED-07b` | One invocation sends the same refund in two turns | `py/anthropic-messages`, `py/openai-chat`, `ts/anthropic-messages`, `ts/openai-chat`, `ts/openai-responses` |

`ZC-SEED-03` fails under the default posture by design: the default records an invented value and does not refuse it. Read [Postures](/docs/postures) for its result under `enforce`.

A plain loop runs the tool in your own code. Provely cannot reach a tool that hangs or raises inside your own handler. On a plain loop, `ZC-SEED-04a` and `ZC-SEED-05` are out of reach, and out of reach never counts as a pass.

## Read next

- [The result of each path](https://provely.sh/frameworks)
- [How a check acts](https://provely.sh/how-it-works)