ProvelyTwo lines put a check at every step your agent takes, and Provely fixes what it can.
Coverage is not yet complete. Being updated continuously.
Fixed on every path where it appliesMeasured, not fixed everywhereFails on at least one pathNot measured yet
Step 1. Why does the plain code break?
The model sends get_weather({'city': 'Paris'}). The model writes the arguments of get_weather with single quotes, so they are not JSON.
import OpenAI from 'openai';
const getWeather = ({ city }: { city: string }) => `Sunny in ${city}`;
const parameters = { type: 'object', properties: { city: { type: 'string' } } };
const reply = await new OpenAI().chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'What is the weather in Paris?' }],
tools: [{ type: 'function', function: { name: 'get_weather', parameters } }],
});
const call = reply.choices[0]!.message.tool_calls![0]!;
console.log(getWeather(JSON.parse(call.function.arguments)));
$ node weather.ts
SyntaxError: Expected property name or '}' in JSON at position 1 (line 1 column 2)
exit 1
Step 2. What does a fix by hand take?
A try/catch adds 5 lines of code. It catches the error, and the run gives no answer:
$ node weather-defended.ts
Could not read the tool arguments: Expected property name or '}' in JSON at position 1 (line 1 column 2)
exit 0
A real repair adds 28 lines of code, 34 with the comments. Each comment states what its guard does.
import OpenAI from 'openai';
const getWeather = ({ city }: { city: string }) => `Sunny in ${city}`;
const parameters = { type: 'object', properties: { city: { type: 'string' } } };
function repairJson(text: string): string {
// the model may wrap the JSON in a Markdown fence
const fence = text.trim().match(/^```(?:json)?\s*([\s\S]*?)\s*```$/);
const source = fence ? fence[1]! : text;
let out = '';
let quote: string | null = null;
for (let i = 0; i < source.length; i += 1) {
const char = source[i]!;
if (quote === null) {
// the model may use single quotes: every string gets double quotes
if (char === "'" || char === '"') quote = char;
out += quote === null ? char : '"';
} else if (char === '\\') {
// an escaped single quote is not valid JSON: keep the quote, drop the backslash
const next = source[i + 1] ?? '';
out += next === "'" ? "'" : char + next;
i += 1;
} else if (char === quote) {
quote = null;
out += '"';
} else {
// a double quote inside a single-quoted string must be escaped
out += char === '"' ? '\\"' : char;
}
}
return out;
}
const reply = await new OpenAI().chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'What is the weather in Paris?' }],
tools: [{ type: 'function', function: { name: 'get_weather', parameters } }],
});
const call = reply.choices[0]!.message.tool_calls![0]!;
// the model may send arguments that are not JSON: tell the user, do not crash
try {
console.log(getWeather(JSON.parse(repairJson(call.function.arguments))));
} catch (error) {
console.log(`Could not read the tool arguments: ${(error as Error).message}`);
}
$ node weather-repaired.ts
Sunny in Paris
exit 0
Fixed by hand, for 2 kinds of fault.
Step 3. What do the two lines of Provely do?
Add 2 lines to the plain file: lines 2 and 3. Provely fixes the call before your code reads it.
import OpenAI from 'openai';
import { Provely } from '@provely/sdk';
new Provely();
const getWeather = ({ city }: { city: string }) => `Sunny in ${city}`;
const parameters = { type: 'object', properties: { city: { type: 'string' } } };
const reply = await new OpenAI().chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'What is the weather in Paris?' }],
tools: [{ type: 'function', function: { name: 'get_weather', parameters } }],
});
const call = reply.choices[0]!.message.tool_calls![0]!;
console.log(getWeather(JSON.parse(call.function.arguments)));
$ node weather.ts
Sunny in Paris
exit 0
Auto fixed by Provely: repair_in_place · single_quoted_strings · get_weather
Lines of code: 0 → breaks · 5 → caught, no answer · 28 → fixes 2 kinds of fault · 2 → Provely's checks
Recorded on 2026-10-17. Every output above is the output of that run.
How does Provely handle a multi-hop agent run?
In words
- The ask goes to the model, and the model answers with a tool call.
- Provely checks the answer before the framework runs the call.
- The tool runs, and Provely checks the result before the model reads it.
- The model calls the next tool, and the same two checks run again, until the final answer.
- One run memory spans every hop: completed writes, the step order and the counts.
A multi-hop run is a loop. The model answers, a tool runs, the result goes back to the model, and the model calls the next tool, until the final answer. Provely checks every hop at two points.
- The model's answer, before your framework runs the call. Provely repairs broken arguments. It refuses an invented tool or value, or it returns the error to the model so the model can correct it. It stops a tool that hangs.
- The tool's result, before the model reads it. Provely catches an error page that says "succeeded", a replayed result and a broken value. The model does not build its next step on a false result.
- One memory for the whole run. Provely remembers each completed write, so the same refund later in the same run is refused. A signed rule can set the order of steps (look up the charge before the refund) and a limit (at most 2 refunds in a run).
- More than one agent. Each agent in a run keeps its own plan and its own list of tools.
Not covered yet: a retry across separate agent calls; pauses for human approval; resuming from a checkpoint; parallel runs; durable workflow engines. The coverage page shows what is measured. The intent firewall will check that a whole workflow reached its goal in the real systems.
What comes next?
Coming soon. The intent firewall is coming soon.
It will check that what an agent did matches what a person authorised, in the real systems, with a signed receipt. Read the planned intent firewall.