David Blass dd26d35137 learnings: audit fixes — preamble in TOC, server-side line-boundary truncation, empty-repo intro (#743)
* learnings: surface preamble in TOC, mirror line-boundary truncation server-side, fix empty-repo intro copy

three audit fixes on top of the recent learnings overhaul (#717):

- `parseLearningsHeadings` now prepends a synthetic `(preamble)` entry
  when a body has non-whitespace content before the first heading. the
  prompt instructs the agent NOT to slurp the whole file when a TOC is
  present, so without this any preamble lines were silently invisible
  (realistic transitional case: an agent partially restructures a
  legacy free-text body and leaves bullets above the first `## `).

- server-side PATCH route now applies the same line-boundary-aware
  truncation as the action (defense in depth via a shared
  `truncateAtLineBoundary` + `MAX_LEARNINGS_LENGTH` exported from
  `action/internal`). the raw `.slice` it used before could leave a
  mid-heading tail on any caller that bypassed the client-side
  truncate, breaking the next-seed TOC parse. removes the duplicated
  cap constant.

- `buildLearningsSection` intro no longer asserts "accumulated by
  previous agent runs" — false for fresh repos with zero history. new
  copy is tense-neutral and works for empty + populated bodies. also
  nudges the agent to re-read after mid-run edits (the inlined TOC
  ranges are a run-start snapshot).

Co-authored-by: Cursor <cursoragent@cursor.com>

* learnings prompt: tighten to single evergreen test, allow tool-quirk bullets when they prevent repeat waste

The blanket "no pullfrog tool quirks" ban was wrong — if the agent burned
calls discovering a quirk this run, recording the workaround prevents the
next run from repeating the waste. Reframe around one litmus ("would a
future run do its work better because this bullet exists?") and trust it
to subsume the scattered don'ts. Drop the 3+ months timeframe (arbitrary)
and the four-example pullfrog/PR/date/play-by-play list (the rule
underneath is "don't anchor facts to repo state that will move"). Cuts
~10 lines from a prompt the model was already mostly ignoring; the
remaining anchor list is narrower and more enforceable.

* audit-learnings-r2: align wiki + tighten re-read nudge

- wiki/prompt.md described the post-run reflection prompt as "bans pullfrog-tool quirks (those belong in tool descriptions, not per-repo learnings), bans PR/review/commit/date references" — that's stale after the prompt rewrite. update to: single-litmus framing, expanded anchor list (now includes version pins + line numbers), and explicit allowance for tool-quirk workarounds when discovery burned calls.
- buildLearningsSection re-read nudge said "re-read after editing" which can be read as "re-read the section you edited". in fact any edit shifts the line numbers of every later section in the TOC, not just the edited one. tighten to make that explicit. mirror the new wording in the wiki example block. update the test substring assertion accordingly.

* postRun: refresh JSDoc to match the reflection prompt rewrite

`buildLearningsReflectionPrompt`'s JSDoc still listed "PR-/review-/commit-/date-anchored facts" and "rediscovery of pullfrog-tool quirks" as failure modes the prompt pushes back on. after b586b4f8 the prompt no longer bans tool-quirk bullets (it explicitly allows them when the agent burned calls discovering the quirk), and the anchor list expanded to cover branch refs, version pins, and line numbers too. update the JSDoc so it describes the prompt that actually exists, and call out the cross-repo drift tradeoff that comes with allowing tool-quirk bullets.

* fix(mcp/issueEvents): narrow event.event before Set.has lookup

octokit's listEventsForTimeline union includes timeline-event members where `event` is `event?: string`. `("event" in event)` does not narrow that property to non-undefined, so `relevantEventTypes.has(event.event)` was passing `string | undefined` to a `Set<string>.has`. typescript only flagged this once `cf-worker-indexing` started seeing the file via the type graph that now reaches mcp through the new `truncateAtLineBoundary` re-export in `action/internal/index.ts`. fix the latent bug at the source: require `typeof event.event === "string"` before the Set lookup.

* learnings: split truncation helpers into MCP-free module

re-exporting `truncateAtLineBoundary` + `MAX_LEARNINGS_LENGTH` from `action/utils/learnings.ts` through `action/internal/index.ts` accidentally pulled the entire MCP type graph into the SDK barrel: `learnings.ts` imports `ToolContext` from `mcp/server.ts`, which transitively wires every tool module under `action/mcp/` into anything that imports from `pullfrog/internal`. for `cf-worker-indexing/tsconfig.json` (`customConditions: ["@pullfrog/source"]`) and the root `tsc` (which compiles the proprietary app routes that import from `pullfrog/internal`), this expanded the type-checked surface and surfaced two latent issues in unrelated files (`mcp/issueEvents.ts`, `utils/subprocess.ts`). a 6-line pure string helper has no business dragging mcp/server.ts into anyone else's type graph.

move both symbols to `action/utils/learningsTruncate.ts`. `learnings.ts` re-exports them so existing callers keep working; `internal/index.ts` re-exports from the truncate-only module so the SDK barrel stays MCP-free.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Colin McDonnell <colinmcd94@gmail.com>
2026-05-19 21:47:10 +00:00
2026-01-16 08:00:16 +00:00
2026-03-12 05:22:51 +00:00
2025-08-27 16:53:48 -07:00
2026-01-19 08:41:56 +00:00
2026-03-12 05:22:51 +00:00

Green Pullfrog logo
Pullfrog

Bring your favorite coding agent into GitHub


🚀 Pullfrog is in beta! We're onboarding users in waves. Get on the waitlist →


What is Pullfrog?

Pullfrog is a GitHub bot that brings the full power of your favorite coding agents into GitHub. It's open source and powered by GitHub Actions.

  • Tag @pullfrog — Tag @pullfrog in a comment anywhere in your repo. It will pull in any relevant context using the action's internal MCP server and perform the appropriate task.
  • Prompt from the web — Trigger arbitrary tasks from the Pullfrog dashboard
  • Automated triggers — Configure Pullfrog to trigger agent runs in response to specific events. Each of these triggers can be associated with custom prompt instructions.
    • issue created
    • issue labeled
    • PR created
    • PR review created
    • PR review requested
    • and more...

Pullfrog is the bridge between your preferred coding agents and GitHub. Use it for:

  • 🤖 Coding tasks — Tell @pullfrog to implement something and it'll spin up a PR. If CI fails, it'll read the logs and attempt a fix automatically. It'll automatically address any PR reviews too.
  • 🔍 PR review — Coding agents are great at reviewing PRs. Using the "PR created" trigger, you can configure Pullfrog to auto-review new PRs.
  • 🤙 Issue management — Via the "issue created" trigger, Pullfrog can automatically respond to common questions, create implementation plans, and link to related issues/PRs. Or (if you're feeling lucky) you can prompt it to immediately attempt a PR addressing new issues.
  • Literally whatever — Want to have the agent automatically add docs to all new PRs? Cut a new release with agent-written notes on every commit to main? Pullfrog lets you do it.

Standalone Usage

You can also use pullfrog/pullfrog as a step in your own workflows. The action exposes a result output that can be consumed by subsequent steps.

Example: Auto-generate release notes on new tags

name: Release
on:
  push:
    tags: ['v*']

permissions:
  contents: write

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Generate release notes
        id: notes
        uses: pullfrog/pullfrog@v0
        with:
          prompt: |
            Generate release notes for ${{ github.ref_name }}.
            Compare commits between this tag and the previous tag.
            Format as markdown: summary paragraph, then ### Features, ### Fixes, ### Breaking Changes sections.
            Omit empty sections. Be concise.
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

      # write to file to avoid shell escaping issues with special characters
      - name: Create GitHub release
        run: |
          notesfile="$RUNNER_TEMP/release-notes-$GITHUB_RUN_ID.md"
          printf '%s' "$NOTES" > "$notesfile"
          gh release create ${{ github.ref_name }} --title "${{ github.ref_name }}" --notes-file "$notesfile"
        env:
          GH_TOKEN: ${{ github.token }}
          NOTES: ${{ steps.notes.outputs.result }}

Example: Structured Output with Zod Schema

You can force the agent to return structured JSON output by providing a JSON schema. This allows you to reliably parse and use the agent's response in subsequent workflow steps.

You can define your JSON schema directly or uou can use any validation library that converts to JSON Schema. Here's an example using Zod:

name: Release Check
on:
  pull_request:
    types: [closed]

jobs:
  check-release:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install dependencies
        run: npm install --no-save --no-package-lock zod @actions/core

      - name: Generate Schema
        id: schema
        run: |
          node -e '
            import { z } from "zod";
            import { setOutput } from "@actions/core";
            const schema = z.object({
              version: z.string().describe("Semantic version number (e.g. 1.0.0)"),
              isBreaking: z.boolean().describe("Whether this release contains breaking changes"),
              changelog: z.array(z.string()).describe("List of changes in this release"),
            });
            setOutput("schema", JSON.stringify(z.toJSONSchema(schema)));
          '

      - name: Analyze PR
        id: analysis
        uses: pullfrog/pullfrog@v0
        with:
          prompt: |
            Analyze this PR and determine semantic versioning impact.
            Return a JSON object matching the provided schema.
          output_schema: ${{ steps.schema.outputs.schema }}
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

      - name: Process Result
        run: |
          # Parse the JSON result using fromJSON()
          echo "Version: ${{ fromJSON(steps.analysis.outputs.result).version }}"
          echo "Breaking: ${{ fromJSON(steps.analysis.outputs.result).isBreaking }}"
S
Description
Self-hosted Ollama-powered code review bot for Gitea Actions based on pullfrog
Readme MIT 27 MiB
Languages
TypeScript 97.9%
Shell 0.7%
Dockerfile 0.7%
JavaScript 0.7%