ShockVPN/shockbot
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rename agent key to opencode and add skill invocation coverage (#541)

Browse Source 
* rename agent key to opencode and add skill invocation coverage.

add skill-invoke tests for claude and opencode with local play-based validation signals, update CI matrices, and include the current tracked refactors in this branch for review.

Made-with: Cursor

* exclude agent-specific skill-invoke tests from wrong agent in CI matrix

* address review follow-up and preserve workflow run UI tweak.

switch changed-agents ci coverage to exercise the opentoad implementation path while keeping the opencode expectation, and include the local workflow run client interaction updates requested on this branch.

Made-with: Cursor

* remove opentoad agent filename from runtime.

rename the opencode harness implementation file from opentoad.ts to opencode.ts and update ci coverage input accordingly so action code no longer carries the old filename.

Made-with: Cursor

* ensure security prompt bypass is set on every test fixture.

this keeps adversarial and permissions harnesses from being blocked by the default prompt-injection refusal path during CI security tests.

Made-with: Cursor

---------

Co-authored-by: pullfrog[bot] <226033991+pullfrog[bot]@users.noreply.github.com>
This commit is contained in:
Colin McDonnell
2026-04-15 19:38:36 +00:00
committed by pullfrog[bot]
parent 2799cce4bf
commit 4b3c5ca905
32 changed files with 158 additions and 219303 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/run.ts
+7 -7
View File
@@ -27,14 +27,14 @@ import {
* filters can be test names, tags, or agent names:
* node test/run.ts # run all tests (excludes adhoc-tagged tests)
* node test/run.ts smoke # run tests named "smoke" or tagged "smoke"
* node test/run.ts opentoad # run all tests for opentoad only
* node test/run.ts opencode # run all tests for opencode only
* node test/run.ts security # run all tests tagged "security"
* node test/run.ts agnostic # run all agnostic-tagged tests (with opentoad)
* node test/run.ts agnostic # run all agnostic-tagged tests (with opencode)
* node test/run.ts adhoc # run all adhoc-tagged tests
* node test/run.ts smoke opentoad # run smoke tests for opentoad only
* node test/run.ts smoke opencode # run smoke tests for opencode only
*
* special tags:
* - "agnostic": runs with opentoad only, excluded when filtering by agent
* - "agnostic": runs with opencode only, excluded when filtering by agent
* - "adhoc": excluded from default runs, must be explicitly requested
*
* by default, runs in a Docker container for isolation.
@@ -303,7 +303,7 @@ async function runTestForAgent(ctx: RunContext): Promise<ValidationResult> {
if (!Object.hasOwn(env, "PULLFROG_MODEL")) {
const defaultModels: Record<string, string> = {
claude: "anthropic/claude-sonnet-4-6",
opentoad: "anthropic/claude-sonnet-4-6",
opencode: "anthropic/claude-sonnet-4-6",
};
const model = defaultModels[ctx.agent];
if (model) {
@@ -414,11 +414,11 @@ async function main(): Promise<void> {
const isAgnostic = hasTag(testInfo, "agnostic");
if (isAgnostic) {
// agnostic tests: skip if only filtering by agent, otherwise run with opentoad
// agnostic tests: skip if only filtering by agent, otherwise run with opencode
if (parsed.filters.length === 0 && parsed.agentFilters.length > 0) {
continue;
}
runs.push({ testInfo, agent: "opentoad" });
runs.push({ testInfo, agent: "opencode" });
} else {
// determine which agents to run for this test
const testAgents = testInfo.config.agents ?? agents;