* audit learnings: reshape reflection prompt + bake tool quirks into descriptions (#619)
Cross-repo audit of the 48 repos with non-null learnings turned up two
recurring failure modes:
1. ~25-30% of bullets across the most-active repos are pullfrog-tool
quirks ("shell timeout is in milliseconds", "git args must be a JSON
array", "create_pull_request_review drops out-of-hunk comments",
"push_branch may report timeout when push succeeded", "checkout_pr
shallow.lock retries", "commit_id needs full 40-char SHA"). These are
universal across repos and should live in tool descriptions, not be
rediscovered and stored 48 times. Tool descriptions now surface them.
2. Bullets are routinely 200-1000 chars (paragraph-length), and 12 of 48
repos are at the 10k cap. The reflection prompt now: caps bullets at
~240 chars (one specific fact), bans PR/review/commit/date-anchored
facts that decay within weeks, bans tool-quirk learnings, and tells
the agent that cap pressure means compress+prune existing bullets,
not skip new findings.
Co-authored-by: Cursor <cursoragent@cursor.com>
* learnings: add server-generated TOC, fixed section taxonomy, raise cap to 100k (#707)
Cap goes 10k → 100k. Reads stay bounded because the seeded file now
opens with a server-generated table of contents listing every `## `
section's line range — agents read the TOC, then `read_file offset/limit`
just the sections relevant to the current task instead of slurping the
whole file.
## Section taxonomy (fixed)
`## Build & test`, `## CI`, `## Conventions`, `## Architecture`,
`## Gotchas`. Free-form `### ` sub-headings inside a section are fine.
Pre-taxonomy free-text rows get wrapped in a `## Legacy` carve-out on
first seed so they remain visible while the agent gradually re-curates
them during reflection turns.
## Storage shape unchanged
`Repo.learnings` still holds raw markdown (no schema migration). The TOC
is a pure read-side affordance: prepended at seed time, stripped from
the agent-edited file before persist. Markers
`<!-- pullfrog-learnings-toc:* -->` delimit the strip region. Agent
edits inside the markers are discarded.
## Round-trip semantics
`seedLearningsFile` now returns `{ path, canonicalSeed }` where
`canonicalSeed` is the post-TOC body — same shape `readLearningsFile`
returns at end-of-run, so `persistLearnings` byte-compares them
directly to skip the no-op PATCH. Empty-repo first runs end up with the
section scaffold both as seed and as read-back, so untouched runs still
short-circuit cleanly.
## Reflection prompt
Adds explicit section-placement guidance (place each new bullet under
the most relevant `## `; do NOT add new top-level headings; do NOT
edit anything between the TOC markers). Carries forward the bullet
hygiene from the previous commit: ≤240 chars per bullet, no
pullfrog-tool quirks (those belong in tool descriptions), no
PR/review/commit/date references. The "near cap" framing is replaced
with "compress and prune within a section when it grows noisy" since
the cap pressure that drove cramming is gone.
Co-authored-by: Cursor <cursoragent@cursor.com>
* anneal round 1: line-anchored taxonomy detect, partial-merge, line-boundary truncation, scaffold-empty UI
Multi-lens review of the TOC + taxonomy diff surfaced a cluster of
correctness and operational bugs. Fixes:
- `hasAnyTaxonomyHeading` used `String.includes("## X")` which
false-positives on `### X` (the `## ` substring sits inside `### `),
prose containing `## CI`, fenced code documenting markdown, etc.
Replaced with a line-anchored predicate that reuses `parseHeadings`
so detection and TOC construction stay consistent.
- The "any heading present → pass through verbatim" rule meant a body
with one taxonomy heading would seed without the other four. Worse,
requiring all five would flip a body back into Legacy when the agent
legitimately pruned a section to empty. New `partial` kind: keep
existing content in place, append missing sections in canonical order
so the agent always has the full scaffold without losing pruning
intent.
- `stripLearningsToc` collapsed `\n{3,}` globally; `canonicalSeed`
doesn't, so an untouched body with intentional triple-newline spacing
would compare unequal and burn a spurious LearningsRevision row each
run. Drop the global collapse — only the leading newlines that the
strip itself introduces are normalized.
- 100k truncation via `slice(0, 100_000)` could cut mid-line, breaking
`parseHeadings` (whole-line `^## `) on the next seed and flipping a
cut body back into Legacy. New `truncateAtLineBoundary` cuts at the
last newline before the cap.
- `LearningsSection.tsx` rendered a scaffold-only body as "has
learnings" instead of the empty placeholder. Added a
`hasOnlyEmptyScaffold` guard so the console behaves the same as
pre-PR for the empty case.
- Seed log line distinguishes `kind=structured/partial/legacy-wrapped/
empty` instead of `existing=yes/no`, so operators can spot legacy
migration activity in logs.
- New tests cover: substring false-positive (`### Build & test`,
in-prose mentions), partial-taxonomy merge (no Legacy wrap),
full-taxonomy structured pass-through, last-newline truncation,
triple-newline preservation.
Deferred (documented in PR body): deploy-ordering footgun (action
before API), rollback for rows >10k, Gemini sanitizer dropping
`description` on `anyOf` branches, reflection-on-failed-runs.
Co-authored-by: Cursor <cursoragent@cursor.com>
* anneal r2: hard-truncate fallback when line boundary discards >4k
Round-2 review caught a regression in `truncateAtLineBoundary`: when the
only newline within the first 100k chars sits near the start (e.g. one
heading + 100k+ char single line — pathological pasted log dumps), the
line-boundary cut discards almost all of the body. losing one partial
line is preferable to losing kilobytes; threshold the fallback at 4k.
Co-authored-by: Cursor <cursoragent@cursor.com>
* move TOC out of file: prompt-side rendering, server-parsed headings
drops the in-file TOC + fixed taxonomy in favor of:
- file on disk = verbatim Repo.learnings (no markers, no scaffold)
- server parses headings (mdast-util-from-markdown) at run-context time
and returns them as RepoSettings.learningsHeadings
- action renders heading TOC into the LEARNINGS prompt section as
parenthesized line ranges like `Build & test (L1-L42)` with hierarchy
via 2-space indent off the shallowest depth
- reflection prompt teaches agent-curated structure with a soft 300-line
per-section cap and explicit guidance to restructure flat legacy lists
cuts 8 helpers (ensureSections, stripLearningsToc, assembleFile,
buildTocBlock, parseHeadings, buildSectionScaffold, hasAnyTaxonomyHeading,
LEARNINGS_SECTIONS) and the canonicalSeed round-trip dance.
action seedLearningsFile is now { path } only; main.ts byte-compares the
trimmed read-back against (current ?? "").trim() to gate the persist
PATCH. truncateAtLineBoundary kept for safety.
new tests:
- test/learningsToc.test.ts (11 parser cases incl. fenced-code, blockquote,
arbitrary h1-h6 nesting, startLine-points-at-heading invariant)
- action/utils/learningsTocRender.test.ts (7 renderer cases)
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
Pullfrog is a GitHub bot that brings the full power of your favorite coding agents into GitHub. It's open source and powered by GitHub Actions.
Tag @pullfrog — Tag @pullfrog in a comment anywhere in your repo. It will pull in any relevant context using the action's internal MCP server and perform the appropriate task.
Prompt from the web — Trigger arbitrary tasks from the Pullfrog dashboard
Automated triggers — Configure Pullfrog to trigger agent runs in response to specific events. Each of these triggers can be associated with custom prompt instructions.
issue created
issue labeled
PR created
PR review created
PR review requested
and more...
Pullfrog is the bridge between your preferred coding agents and GitHub. Use it for:
🤖 Coding tasks — Tell @pullfrog to implement something and it'll spin up a PR. If CI fails, it'll read the logs and attempt a fix automatically. It'll automatically address any PR reviews too.
🔍 PR review — Coding agents are great at reviewing PRs. Using the "PR created" trigger, you can configure Pullfrog to auto-review new PRs.
🤙 Issue management — Via the "issue created" trigger, Pullfrog can automatically respond to common questions, create implementation plans, and link to related issues/PRs. Or (if you're feeling lucky) you can prompt it to immediately attempt a PR addressing new issues.
Literally whatever — Want to have the agent automatically add docs to all new PRs? Cut a new release with agent-written notes on every commit to main? Pullfrog lets you do it.
Standalone Usage
You can also use pullfrog/pullfrog as a step in your own workflows. The action exposes a result output that can be consumed by subsequent steps.
Example: Auto-generate release notes on new tags
name:Releaseon:push:tags:['v*']permissions:contents:writejobs:release:runs-on:ubuntu-lateststeps:- name:Checkoutuses:actions/checkout@v4with:fetch-depth:0- name:Generate release notesid:notesuses:pullfrog/pullfrog@v0with:prompt:| Generate release notes for ${{ github.ref_name }}.
Compare commits between this tag and the previous tag.
Format as markdown: summary paragraph, then ### Features, ### Fixes, ### Breaking Changes sections.
Omit empty sections. Be concise.env:ANTHROPIC_API_KEY:${{ secrets.ANTHROPIC_API_KEY }}# write to file to avoid shell escaping issues with special characters- name:Create GitHub releaserun:| notesfile="$RUNNER_TEMP/release-notes-$GITHUB_RUN_ID.md"
printf '%s' "$NOTES" > "$notesfile"
gh release create ${{ github.ref_name }} --title "${{ github.ref_name }}" --notes-file "$notesfile"env:GH_TOKEN:${{ github.token }}NOTES:${{ steps.notes.outputs.result }}
Example: Structured Output with Zod Schema
You can force the agent to return structured JSON output by providing a JSON schema. This allows you to reliably parse and use the agent's response in subsequent workflow steps.
You can define your JSON schema directly or uou can use any validation library that converts to JSON Schema. Here's an example using Zod:
name:Release Checkon:pull_request:types:[closed]jobs:check-release:if:github.event.pull_request.merged == trueruns-on:ubuntu-lateststeps:- uses:actions/checkout@v4- name:Install dependenciesrun:npm install --no-save --no-package-lock zod @actions/core- name:Generate Schemaid:schemarun:| node -e '
import { z } from "zod";
import { setOutput } from "@actions/core";
const schema = z.object({
version: z.string().describe("Semantic version number (e.g. 1.0.0)"),
isBreaking: z.boolean().describe("Whether this release contains breaking changes"),
changelog: z.array(z.string()).describe("List of changes in this release"),
});
setOutput("schema", JSON.stringify(z.toJSONSchema(schema)));
'- name:Analyze PRid:analysisuses:pullfrog/pullfrog@v0with:prompt:| Analyze this PR and determine semantic versioning impact.
Return a JSON object matching the provided schema.output_schema:${{ steps.schema.outputs.schema }}env:ANTHROPIC_API_KEY:${{ secrets.ANTHROPIC_API_KEY }}- name:Process Resultrun:| # Parse the JSON result using fromJSON()
echo "Version: ${{ fromJSON(steps.analysis.outputs.result).version }}"
echo "Breaking: ${{ fromJSON(steps.analysis.outputs.result).isBreaking }}"