Skip to content

Report on every pull request in GitHub Actions

The sarifix GitHub Action does this in one step

rossedman/sarifix runs this check on pull requests (mode: check), with scanner installs and annotations included. This page builds the same workflow by hand, which is useful when you want full control over each step.

Run the check from Check only what a pull request introduced on every pull request. The workflow scans the pull request and its merge base with the same config, posts what the change introduced to the job summary, and uploads the report.

The workflow

Save as .github/workflows/sarifix.yml. It is also in the sarifix repository at examples/workflows/sarifix-pr.yml.

# Runs on every pull request: scans the PR and its merge base with the same
# config, then writes what the PR introduced to the job summary and uploads
# the report. Read-only: no secrets, no write token, no agent.
name: sarifix

on:
  pull_request:

permissions: {}

jobs:
  report:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    permissions:
      contents: read
    env:
      # v0.1.0 predates `plan` and report mode, which this workflow needs.
      # Pin a commit SHA from github.com/rossedman/sarifix until the next
      # release, then that release's tag, and bump it on purpose.
      SARIFIX_REF: main
      TRIVY_VERSION: v0.76.1
      CHECKOV_VERSION: "3.3.30"
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0 # the merge base must be reachable
          persist-credentials: false

      - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
        with:
          go-version: stable
          cache: false

      - uses: aquasecurity/setup-trivy@81e514348e19b6112ce2a7e3ecbafe19c1e1f567 # v0.3.1
        with:
          version: ${{ env.TRIVY_VERSION }}

      - name: Install sarifix and checkov
        run: |
          go install "github.com/rossedman/sarifix/cmd/sarifix@${SARIFIX_REF}"
          pipx install "checkov==${CHECKOV_VERSION}"

      - name: Scan the merge base
        # The PR's sarifix.yaml scans both sides, so a config change in the
        # PR can't make the base look different.
        run: |
          base=$(git merge-base "origin/${GITHUB_BASE_REF}" HEAD)
          git worktree add "${RUNNER_TEMP}/base" "$base"
          sarifix scan --repo "${RUNNER_TEMP}/base" --config "${PWD}/sarifix.yaml" \
            --out-dir "${RUNNER_TEMP}/base-sarif" --log-level warn

      - name: Scan the pull request
        run: sarifix scan --out-dir "${RUNNER_TEMP}/head-sarif" --log-level warn

      - name: Report what the pull request introduced
        run: |
          sarifix fix --mode report \
            --sarif "${RUNNER_TEMP}/head-sarif/*.sarif" \
            --baseline "${RUNNER_TEMP}/base-sarif/*.sarif" \
            --out-dir "${RUNNER_TEMP}/report" --log-level warn
          sarifix diff --base "${RUNNER_TEMP}/base-sarif/*.sarif" \
            --head "${RUNNER_TEMP}/head-sarif/*.sarif" --json > "${RUNNER_TEMP}/report/diff.json"
          {
            echo "### sarifix"
            jq -r '.counts | "\(.new) new, \(.fixed) fixed, \(.unchanged) unchanged findings"' "${RUNNER_TEMP}/report/diff.json"
            echo
            jq -r '"Fixable now: \(.batched) findings in \(.batches | length) batches"' "${RUNNER_TEMP}/report/summary.json"
            echo
            echo "| New finding | Location |"
            echo "| --- | --- |"
            jq -r '.new[] | "| `\(.tool)/\(.ruleId)` | `\(.path):\(.startLine)` |"' "${RUNNER_TEMP}/report/diff.json"
          } >> "$GITHUB_STEP_SUMMARY"

      - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
        with:
          name: sarifix-report
          path: ${{ runner.temp }}/report/

How it fits together:

  • Both sides use the pull request's sarifix.yaml. The merge base is checked out as a worktree and scanned with --config pointing at the pull request's file. A pull request that changes the config is then judged by its own rules on both sides, rather than looking like it fixed or introduced everything a rule covers.
  • fetch-depth: 0 makes the merge base reachable. A shallow clone can't compute it.
  • Read-only. contents: read is the only permission and no secrets are used, so it is safe to run on pull requests from forks.
  • Pinned tools. Scanner versions are pinned, so a new rule in a scanner release shows up as a deliberate version bump, not as a wave of new findings on an unrelated pull request.

What a run produces

The workflow's scan and report steps were run against the terragoat pull request from the pull request tutorial, with the GitHub variables they read set by hand. The job summary they wrote:

### sarifix
15 new, 2 fixed, 718 unchanged findings

Fixable now: 2 findings in 2 batches

| New finding | Location |
| --- | --- |
| `Checkov/CKV2_AWS_6` | `terraform/aws/reports.tf:1` |
| `Checkov/CKV2_AWS_62` | `terraform/aws/reports.tf:1` |
...
| `Trivy/AWS-0132` | `terraform/aws/reports.tf:1` |

The sarifix-report artifact holds sarifix.sarif (every finding with its status), summary.json (batches and skip counts) and diff.json (new, fixed and unchanged findings).

Adapting it

  • Other scanners: install them in the install step. The scan steps run whatever sarifix.yaml lists.
  • Fail the build on new findings: add a step that exits non-zero when jq '.counts.new' diff.json is above zero. Start with a report-only workflow and turn this on once the numbers are trusted.
  • Scan the default branch on a schedule: drop the merge-base step and run sarifix fix --mode report without --baseline. The report then covers the whole backlog.

Note

The workflow uses sarifix fix --mode report, which came after the v0.1.0 release (v0.1.0 has only scan and diff). Until the next release, pin SARIFIX_REF to a commit SHA from main rather than main itself; after that, pin the release tag. Release binaries can be verified as described in Security.