goalline How it worksClaude Code hookGitHub ActionBenchmarkCommandsGitHub

goalline / GitHub Action

A GitHub Action that catches tests edited to pass

The goalline Action runs on every pull request. It replays the old versions of the tests the pull request edited against its new code, and puts a card on the job summary listing every expectation that moved, with the old and new values.

Add it to a workflow

name: goalline
on: pull_request
jobs:
  goalpost:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 20
      - run: npm ci
      - uses: intikhab49/goalline@v0
        with:
          command: npx jest {files}

The action fetches the history it needs, so the default shallow checkout is fine. It is listed on the GitHub Marketplace.

Vitest and pytest

      - uses: intikhab49/goalline@v0
        with:
          command: npx vitest run {files}
      - uses: actions/setup-python@v7
        with:
          python-version: '3.12'
      - run: pip install -e . pytest
      - uses: intikhab49/goalline@v0
        with:
          command: python -m pytest {files}

Inputs

InputDefaultWhat it does
commandrequiredThe test command, with {files} where the test files go: npx jest {files}, npx vitest run {files}, python -m pytest {files}.
checkgoalpostgoalpost (the old tests on the new code) or prove-red (the new tests on the old code).
base(see description)The ref to compare against. Default: origin/<the pull request base branch>, else origin/main.
cwd(see description)The folder the tests run in, inside the repository (for a monorepo).
tests(see description)Globs that say which files are tests, comma separated. Default: the usual names for the runner.
repo(see description)The repository to check. Default: the checked-out workspace.
fail-on-flagtrueFail the step when goalline flags something the change does not explain.
ackstrueRead .goalpost-acks.json (written by goalline guard --ack): a flagged test the change explains there shows its reason on the card and as a warning, and does not fail the step. Only entries the change added count. false: every flag counts.

Outputs: flagged (true when something is flagged that the change doesn't explain), explained (how many flagged test files the change explains) and report (the path of the Markdown card).

The card on the job summary

This is the real card for a change that made add(2, 3) return 6 and edited its test to match, as Markdown:

### Goalpost: 1 test file(s) need a look

The old versions of the tests this change edited or deleted, run on the new code (merge-base `62fac20` → head `94ac494`). An old test that now fails means behaviour changed and the test was edited to match.

**Needs a look**: the change also removed or rewrote lines of the old test.

- `test/add.test.mjs`
  - ✗ adds two numbers: Expected 5, received 6

<details><summary>Every file</summary>

| Test file | Result | Detail |
|---|---|---|
| `test/add.test.mjs` | ⚠️ MOVED | 1 of 1 old tests fail on the new code<br>✗ adds two numbers: Expected 5, received 6 |

</details>

**FLAGGED (1 moved). Old tests no longer pass on the new code: behaviour changed and the tests were edited to match. A person should confirm that was intended.**

Changes the agent explained

An agent stopped by the Claude Code hook restores the behaviour or says why the change is intended, with goalline guard --ack, which writes .goalpost-acks.json. Commit that file with the change, and the card lists each explained test under Explained, with its old and new values and the reason. A warning on the test file points the reviewer at it, and the step doesn't fail for it. A flagged test without a reason still fails the step.

Only the entries the change itself adds count: one already on the base branch explained an earlier change to that test, so it never explains a new one. With acks: false, every flag fails the step.

New tests that test nothing

With check: prove-red, the action runs the new tests a pull request adds against the base code instead, and flags any that already pass there. See prove-red.

Related