Skip to content

ci: post test results as a PR comment - #17

Merged
joeykamsteeg merged 11 commits into
developfrom
feature/test-report-comment-pr
Aug 3, 2026
Merged

joeykamsteeg merged 11 commits into
developfrom
feature/test-report-comment-pr

Conversation

@joeykamsteeg

Copy link
Copy Markdown
Member

Summary

  • bun test now runs with --reporter=junit --reporter-outfile=test-report.xml so it produces a structured report.
  • A new step publishes that report as a single, updating PR comment (EnricoMi/publish-unit-test-result-action, pinned to the v2.24.0 commit SHA) showing pass/fail/skip counts and any failing test names.
  • The report step only runs for pull_request events (push-to-develop/main and workflow_dispatch runs are unaffected) and uses if: always() so it still comments when tests fail.
  • pull-requests: write permission scoped to the test job only.
  • Full OpenSpec change (proposal/design/specs/tasks) at openspec/changes/add-test-report-pr-comment/.

Test plan

  • Verified locally that bun test --reporter=junit --reporter-outfile=... produces valid JUnit XML.
  • Validated workflow YAML parses correctly.
  • Open this PR and confirm a test report comment appears with correct counts.
  • Push a commit with a failing test and confirm the same comment updates in place (not duplicated).
  • Confirm a push to main/develop and a manual workflow_dispatch run do not attempt to comment.

🤖 Generated with Claude Code

Runs bun test with the JUnit reporter and publishes a single, updating
PR comment (via publish-unit-test-result-action) summarizing pass/fail
counts and failing tests, so reviewers see results without opening the
Actions log.
check_run defaulted to true, which needs a checks:write permission we
don't grant (job only has contents:read/pull-requests:write), causing
a 403 on POST /check-runs. We only need the PR comment, not a separate
check run, so disable it instead of widening permissions.
Replace EnricoMi/publish-unit-test-result-action with junit-to-ctrf +
ctrf-io/github-test-reporter so the PR comment renders as a CTRF-style
summary table (tests/passed/failed/skipped/pending/other/flaky/duration)
with an "updated" footer. update-comment + comment-tag keep it a
single, upserted comment across re-runs, matching the requested
comment design and the existing update-in-place requirement.
@kamsteegsoftware kamsteegsoftware deleted a comment from github-actions Bot Aug 3, 2026
@kamsteegsoftware kamsteegsoftware deleted a comment from github-actions Bot Aug 3, 2026
update-comment appends each run's report to the same comment, which
stacked duplicate result tables across re-runs instead of keeping the
comment as a single up-to-date report. overwrite-comment replaces the
tagged comment's content in place, which is the actual "stays up to
date" behavior that was intended.
@github-actions

github-actions Bot commented Aug 3, 2026 •

Copy link
Copy Markdown

test: Run #32

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Duration ⏱️
136 136 0 0 664ms

🎉 All tests passed!

Github Test Reporter by CTRF 💚

🔄 This comment has been updated

- Drop Pending/Other/Flaky columns via a custom Handlebars template
  (.github/ctrf-pr-report.hbs), since github-test-reporter's built-in
  pull-request-report table hardcodes all 8 columns with no way to
  configure them out.
- junit-to-ctrf always zeroes summary.start/stop, which is why
  Duration showed "not captured". Capture real timestamps around the
  bun test step and patch them into the CTRF report with jq before
  publishing.
Sync main specs/ with the delta specs (new pr-test-report-comment
capability, updated ci-test-workflow requirement) and archive the
completed change now that the workflow ships in .github/workflows/test.yml.
Sync main specs/ with the delta specs (new cache-upstash-adapter
capability, updated package-exports requirements) and archive the
completed change now that the Upstash adapter has shipped.
@joeykamsteeg
joeykamsteeg marked this pull request as ready for review August 3, 2026 21:12
@joeykamsteeg
joeykamsteeg merged commit a5df6a4 into develop Aug 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant