mcp: embed example calls in top-level tool descriptions (#723)

* mcp: embed example calls in top-level tool descriptions

agents (esp. claude sonnet) hallucinate param names from training-data
priors — `pr_number` instead of `pull_number`, `summary` instead of
`body`, full subcommand strings jammed into `git({command})` like it
were `shell({command})`. each error burns a tool round-trip plus a
follow-up ToolSearch, ~40+ events / 24h, no observable recovery cost
to us but visible to users in agent logs.

cheapest fix: add a sample formatted function call to every affected
tool's top-level description. example anchors are more reliable than
schema descriptions alone because the model treats descriptions as
narrative but call examples as canonical structure. for `git` and
`shell` (whose `command` fields collide), include explicit
counter-examples disambiguating which tool owns which shape.

no schema aliases / coercion yet — try the cheap thing first; if the
next audit window still shows the same hallucination rate, layer
aliases on top per #585's recommendation.

closes #585, closes #701

* mcp: drop negative anchors from tool descriptions

negation is a footgun in tool descriptions — telling the model "NOT
pr_number" makes pr_number more salient, not less. let the positive
example carry the schema and trust the model to read it.

removes:
- "the parameter is pull_number (a number), NOT pr_number" and
  similar across checkout_pr, get_pull_request, list_pull_request_reviews,
  get_review_comments, create_pull_request_review
- "NOT summary, message, or content" on report_progress
- "WRONG: git({ command: 'log --oneline' })" counter-example on git
- redundant param-type restatements after the example (e.g. "depth is a
  number, not a string" on git_fetch, "description is required" on shell)

keeps a single positive example per tool. for tools with multiple call
shapes (git, git_fetch, push_branch), two positive examples instead of
one + a counter-example.
This commit is contained in:
Colin McDonnell
2026-05-13 22:45:08 +00:00
committed by pullfrog[bot]
parent 868576a474
commit b8ac42e875
11 changed files with 41 additions and 24 deletions
+10 -10
View File
@@ -218,11 +218,11 @@ export function PushBranchTool(ctx: ToolContext) {
name: "push_branch",
description:
"Push the current branch to the remote repository. Omit branchName to push the current branch (recommended). " +
'Example: `push_branch({})` to push the current branch. Example: `push_branch({ branchName: "pr-1" })` to push a specific local branch. ' +
"If specifying branchName, use the LOCAL branch name (e.g., 'pr-1'), not the remote branch name. " +
"The correct remote and remote branch are determined automatically from branch config set by checkout_pr. " +
"Requires a clean working tree. Runs the repository prepush hook (if configured) before the network push — hook failure means tests/lint or similar in that script failed, not necessarily a Pullfrog timeout. " +
"Never force push unless explicitly requested. Pushes to the default branch are blocked in restricted mode. " +
"If the response reports a timeout, the underlying push may have actually succeeded — verify with `git log origin/<branch>` (or this tool with command 'log') before retrying, otherwise you'll push a duplicate.",
"Never force push unless explicitly requested. Pushes to the default branch are blocked in restricted mode.",
parameters: PushBranch,
execute: execute(async ({ branchName, force }) => {
// permission check
@@ -445,19 +445,17 @@ const subcommandPattern = regex("^[a-z][a-z0-9-]*$");
const Git = type({
command: type(subcommandPattern).describe("Git command (e.g., 'status', 'log', 'diff')"),
args: type.string
.array()
.describe(
'Additional arguments for the git command, as a JSON array of strings (NOT a single string). e.g. args: ["HEAD"], args: ["--oneline", "-20"]. Passing a single string fails validation.'
)
.optional(),
args: type.string.array().describe("Additional arguments for the git command").optional(),
});
export function GitTool(ctx: ToolContext) {
return tool({
name: "git",
description:
"Run git commands. For push/fetch, use the dedicated MCP tools (push_branch, git_fetch). " +
"Run a git subcommand. `command` is a single subcommand; flags and positional args go in `args`. " +
'Example: `git({ command: "log", args: ["--oneline", "-n", "20"] })`. ' +
'Example: `git({ command: "diff", args: ["origin/main..HEAD"] })`. ' +
"For push/fetch, use the dedicated MCP tools (push_branch, git_fetch). " +
"git pull is not available — use git_fetch then this tool with command 'merge'.",
parameters: Git,
execute: execute(async (params) => {
@@ -529,7 +527,9 @@ const DEEPEN_RETRY_DEPTH = 1000;
export function GitFetchTool(ctx: ToolContext) {
return tool({
name: "git_fetch",
description: "Fetch refs from remote repository. Use this instead of git fetch directly.",
description:
"Fetch refs from remote repository. Use this instead of git fetch directly. " +
'Example: `git_fetch({ ref: "main" })`. With depth: `git_fetch({ ref: "pull/1234/head", depth: 1 })`.',
parameters: GitFetch,
execute: execute(async (params) => {
rejectIfLeadingDash(params.ref, "ref");