From 88398744fcf6f3040d53d7642f94b81a4b2cb681 Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 22:45:18 +0200 Subject: [PATCH 01/11] ci: post test results as a PR comment Runs bun test with the JUnit reporter and publishes a single, updating PR comment (via publish-unit-test-result-action) summarizing pass/fail counts and failing tests, so reviewers see results without opening the Actions log. --- .github/workflows/test.yml | 12 +++++- .../add-test-report-pr-comment/.openspec.yaml | 2 + .../add-test-report-pr-comment/design.md | 39 +++++++++++++++++++ .../add-test-report-pr-comment/proposal.md | 26 +++++++++++++ .../specs/ci-test-workflow/spec.md | 12 ++++++ .../specs/pr-test-report-comment/spec.md | 34 ++++++++++++++++ .../add-test-report-pr-comment/tasks.md | 15 +++++++ 7 files changed, 139 insertions(+), 1 deletion(-) create mode 100644 openspec/changes/add-test-report-pr-comment/.openspec.yaml create mode 100644 openspec/changes/add-test-report-pr-comment/design.md create mode 100644 openspec/changes/add-test-report-pr-comment/proposal.md create mode 100644 openspec/changes/add-test-report-pr-comment/specs/ci-test-workflow/spec.md create mode 100644 openspec/changes/add-test-report-pr-comment/specs/pr-test-report-comment/spec.md create mode 100644 openspec/changes/add-test-report-pr-comment/tasks.md diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 117a6ef..8b0afb5 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -10,6 +10,9 @@ on: jobs: test: runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: write steps: - uses: actions/checkout@v4 @@ -19,4 +22,11 @@ jobs: - run: bun install --frozen-lockfile - - run: bun test + - name: Run tests + run: bun test --reporter=junit --reporter-outfile=test-report.xml + + - name: Publish test report + if: always() && github.event_name == 'pull_request' + uses: EnricoMi/publish-unit-test-result-action@d0a4676d0e0b938bc201470d88276b7c74c712b3 # v2.24.0 + with: + files: test-report.xml diff --git a/openspec/changes/add-test-report-pr-comment/.openspec.yaml b/openspec/changes/add-test-report-pr-comment/.openspec.yaml new file mode 100644 index 0000000..e08b5f8 --- /dev/null +++ b/openspec/changes/add-test-report-pr-comment/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-08-03 diff --git a/openspec/changes/add-test-report-pr-comment/design.md b/openspec/changes/add-test-report-pr-comment/design.md new file mode 100644 index 0000000..0d54456 --- /dev/null +++ b/openspec/changes/add-test-report-pr-comment/design.md @@ -0,0 +1,39 @@ +## Context + +`.github/workflows/test.yml` runs `bun test` on every push to `main` and every pull request (see `ci-test-workflow` spec). Results are currently only visible as raw console output in the Actions run log. Bun's test runner has a built-in `--reporter=junit --reporter-outfile=` flag that emits a standard JUnit XML file, which is the common interchange format most PR-comment/reporting GitHub Actions consume — no custom parsing needs to be written in this repo. + +The workflow currently runs with default (read-only) `GITHUB_TOKEN` permissions, which cannot post PR comments. + +## Goals / Non-Goals + +**Goals:** +- Post a single, updating PR comment with pass/fail/skip counts and failing test names whenever the `Test` workflow runs for a pull request. +- Keep the comment accurate across re-runs (edit in place, not one comment per push). +- Keep the change CI-only: no new package dependency, no change to `src/`. + +**Non-Goals:** +- Historical trend reporting (e.g. flaky-test tracking across runs) — only the latest run's result matters. +- Coverage reporting in the comment — this change is about pass/fail visibility, not coverage (`--coverage` is a separate concern). +- Commenting on push-to-`main` or `workflow_dispatch` runs — there is no PR to comment on. + +## Decisions + +- **Report format: Bun's built-in JUnit reporter (`--reporter=junit --reporter-outfile=test-report.xml`).** + Alternative considered: parsing Bun's plain console/`--dots` output with a hand-written script. Rejected — JUnit XML is structured, and re-implementing a results parser to work around the console format is unnecessary work for a solved problem. Bun ships JUnit support natively, so no test-side dependency is added. + +- **Comment posting: a pinned marketplace action that natively supports "parse JUnit XML → upsert single PR comment" (e.g. `EnricoMi/publish-unit-test-result-action`), pinned to a full commit SHA rather than a floating tag.** + Alternative considered: a hand-rolled `actions/github-script` step that reads the XML and calls the Issues API directly. Rejected as the primary approach — correctly finding-and-editing a prior comment, formatting counts/failure lists, and handling the "no comment yet" case is exactly what an existing, widely-used action already does, and hand-rolling it means maintaining XML-parsing logic in-repo for no behavioral benefit. Pinning by SHA (not `@v4`/`@latest`) keeps the workflow's dependency on that action reproducible and auditable, consistent with treating workflow actions as a supply-chain surface. + +- **Job permissions: scope `pull-requests: write` to the `test` job only, only for the steps that need it**, rather than widening default repo-level permissions. The install/build/test steps need no elevated permissions; only the comment-posting step does. + +- **Trigger condition: run the report/comment steps only `if: github.event_name == 'pull_request'`**, and only after the test step, using `if: always()` on the report-generation and comment steps so a failing test run still produces a report (JUnit output is written regardless of pass/fail, and the comment should reflect failures too — see proposal's "posted regardless of pass/fail"). + +## Risks / Trade-offs + +- [Third-party action compromise] → Pin to a full commit SHA (not a tag) and review the pinned commit before adopting; this matches treating the action as an audited dependency rather than a floating reference. +- [`pull-requests: write` on a workflow that also runs `pull_request` on forked-repo PRs] → `pull_request` (not `pull_request_target`) is already the existing trigger, so the workflow runs with the forked repo's read-only token for cross-repo PRs; GitHub will not grant write permissions in that context regardless of the `permissions:` block, so the comment step will simply no-op/fail silently for fork PRs rather than expose a token to untrusted code. This is an accepted limitation, not a regression — same-repo PRs (the common case here) are unaffected. +- [Comment noise on force-pushed/rebased PRs] → Upsert-by-marker behavior (finding the prior bot comment by a hidden identifier and editing it) avoids duplicate comments; this is the action's default behavior and doesn't need custom logic. + +## Open Questions + +- None — if the chosen action's default comment format needs tightening (e.g. collapsing long failure lists), that's an implementation detail for `tasks.md`/the PR itself, not a blocking design decision. diff --git a/openspec/changes/add-test-report-pr-comment/proposal.md b/openspec/changes/add-test-report-pr-comment/proposal.md new file mode 100644 index 0000000..c0d6d19 --- /dev/null +++ b/openspec/changes/add-test-report-pr-comment/proposal.md @@ -0,0 +1,26 @@ +## Why + +The `Test` workflow (`.github/workflows/test.yml`) already runs `bun test` on every pull request, but a reviewer only learns whether it passed or failed by opening the Actions tab and reading raw log output. Surfacing a summary of the test results directly on the pull request removes that extra click and makes failures (and pass counts) visible where review already happens. + +## What Changes + +- Extend the `Test` workflow so that, on pull request runs, it generates a structured test report from `bun test` (via Bun's built-in `--reporter=junit --reporter-outfile=...`) instead of relying solely on console output. +- Add a step that posts (or updates) a single PR comment summarizing the test results: total/passed/failed/skipped counts and the names of any failing tests. +- The comment is upserted (edited in place on re-runs) rather than appended, so repeated pushes to the same PR don't accumulate duplicate comments. +- The report is posted regardless of whether the test run passed or failed, so the comment always reflects the latest run. +- Push-triggered and `workflow_dispatch` runs are unaffected — there is no pull request to comment on, so the report step is skipped for those triggers. + +## Capabilities + +### New Capabilities +- `pr-test-report-comment`: Posting/updating a single pull-request comment that summarizes the outcome of the `bun test` run (counts and failing test names) for that PR's latest workflow run. + +### Modified Capabilities +- `ci-test-workflow`: The workflow gains a report-generation step (JUnit output from `bun test`) and a report-posting step that runs only on `pull_request` events, in addition to its existing trigger/install/test-execution behavior. + +## Impact + +- `.github/workflows/test.yml`: new steps to generate a JUnit report file and to upsert a PR comment from it; requires `pull-requests: write` permission for the job (currently unset, defaults to read). +- No changes to `src/`, package exports, or published package behavior — this is CI-only. +- No new runtime dependency for the package itself; any GitHub Action used to post the comment is a workflow-only dependency (pinned by SHA), not a package dependency. +- Does not require a changeset — CI-only change with no user-facing package effect. diff --git a/openspec/changes/add-test-report-pr-comment/specs/ci-test-workflow/spec.md b/openspec/changes/add-test-report-pr-comment/specs/ci-test-workflow/spec.md new file mode 100644 index 0000000..66fa8cd --- /dev/null +++ b/openspec/changes/add-test-report-pr-comment/specs/ci-test-workflow/spec.md @@ -0,0 +1,12 @@ +## MODIFIED Requirements + +### Requirement: Test execution determines pass/fail +The workflow SHALL run the project's test suite via `bun test`, produce a structured JUnit test report as part of that run, and report the run as failed if any test fails, or succeeded if all tests pass. + +#### Scenario: All tests pass +- **WHEN** `bun test` completes with no failing tests +- **THEN** the workflow run is marked successful and a JUnit test report is produced + +#### Scenario: A test fails +- **WHEN** `bun test` reports at least one failing test +- **THEN** the workflow run is marked failed and a JUnit test report is still produced, reflecting the failure diff --git a/openspec/changes/add-test-report-pr-comment/specs/pr-test-report-comment/spec.md b/openspec/changes/add-test-report-pr-comment/specs/pr-test-report-comment/spec.md new file mode 100644 index 0000000..49a3573 --- /dev/null +++ b/openspec/changes/add-test-report-pr-comment/specs/pr-test-report-comment/spec.md @@ -0,0 +1,34 @@ +## ADDED Requirements + +### Requirement: Test results are posted as a PR comment +When the `Test` workflow runs for a pull request, the workflow SHALL post a comment on that pull request summarizing the `bun test` run: total test count, passed count, failed count, skipped count, and the names of any failing tests. + +#### Scenario: All tests pass on a pull request run +- **WHEN** the `Test` workflow runs for a pull request and `bun test` completes with no failing tests +- **THEN** a PR comment is posted showing the total/passed/skipped counts and zero failures + +#### Scenario: Some tests fail on a pull request run +- **WHEN** the `Test` workflow runs for a pull request and `bun test` reports one or more failing tests +- **THEN** a PR comment is posted showing the total/passed/failed/skipped counts and the names of the failing tests, even though the workflow run itself is marked failed + +### Requirement: Report comment is updated in place across re-runs +The workflow SHALL update its existing test report comment on a pull request rather than posting a new comment for every run. + +#### Scenario: Pull request receives a second push +- **WHEN** the `Test` workflow has already posted a test report comment on a pull request and a new commit is pushed to that pull request, triggering another run +- **THEN** the existing test report comment is edited to reflect the latest run's results, and no additional comment is created + +#### Scenario: First run on a pull request +- **WHEN** the `Test` workflow runs for a pull request that has no prior test report comment +- **THEN** a new test report comment is created + +### Requirement: Report comment is scoped to pull request runs +The workflow SHALL only attempt to post or update a test report comment when triggered by a pull request event. + +#### Scenario: Push to main +- **WHEN** the `Test` workflow runs because of a push to `main` +- **THEN** no test report comment step is attempted, since there is no pull request to comment on + +#### Scenario: Manual workflow_dispatch run +- **WHEN** the `Test` workflow runs because of a manual `workflow_dispatch` trigger +- **THEN** no test report comment step is attempted, since there is no pull request to comment on diff --git a/openspec/changes/add-test-report-pr-comment/tasks.md b/openspec/changes/add-test-report-pr-comment/tasks.md new file mode 100644 index 0000000..fdaea01 --- /dev/null +++ b/openspec/changes/add-test-report-pr-comment/tasks.md @@ -0,0 +1,15 @@ +## 1. Generate a structured test report + +- [x] 1.1 Update the `test` step in `.github/workflows/test.yml` to run `bun test --reporter=junit --reporter-outfile=test-report.xml`, with `if: always()` (or equivalent) so the report is written even when tests fail. + +## 2. Post/update the PR comment + +- [x] 2.1 Add `permissions: pull-requests: write` scoped to the `test` job in `.github/workflows/test.yml`. +- [x] 2.2 Add a report-posting step, pinned to a full commit SHA (e.g. `EnricoMi/publish-unit-test-result-action`), that reads `test-report.xml` and upserts a single PR comment with total/passed/failed/skipped counts and failing test names. +- [x] 2.3 Scope the report-posting step to run only `if: github.event_name == 'pull_request'` and only after the test step (`if: always()` combined with the pull_request check), so push/workflow_dispatch runs and always-run reporting both behave per the spec. + +## 3. Verify + +- [ ] 3.1 Open a PR with a passing test suite and confirm a comment is created showing correct counts. +- [ ] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. +- [ ] 3.3 Confirm a push to `main` and a manual `workflow_dispatch` run do not attempt to post a comment. From 7a2a5f80345bd57b26d2642de466719ee29748e1 Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 22:46:49 +0200 Subject: [PATCH 02/11] fix(ci): disable check-run creation in test report action check_run defaulted to true, which needs a checks:write permission we don't grant (job only has contents:read/pull-requests:write), causing a 403 on POST /check-runs. We only need the PR comment, not a separate check run, so disable it instead of widening permissions. --- .github/workflows/test.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8b0afb5..8e456fa 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -30,3 +30,4 @@ jobs: uses: EnricoMi/publish-unit-test-result-action@d0a4676d0e0b938bc201470d88276b7c74c712b3 # v2.24.0 with: files: test-report.xml + check_run: false From 68c167667d29bcee5d5bca5548bb4a11565c3342 Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 22:55:53 +0200 Subject: [PATCH 03/11] ci: switch PR test report comment to CTRF format Replace EnricoMi/publish-unit-test-result-action with junit-to-ctrf + ctrf-io/github-test-reporter so the PR comment renders as a CTRF-style summary table (tests/passed/failed/skipped/pending/other/flaky/duration) with an "updated" footer. update-comment + comment-tag keep it a single, upserted comment across re-runs, matching the requested comment design and the existing update-in-place requirement. --- .github/workflows/test.yml | 15 ++++++++++++--- .../changes/add-test-report-pr-comment/design.md | 9 ++++++--- .../changes/add-test-report-pr-comment/tasks.md | 2 +- 3 files changed, 19 insertions(+), 7 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8e456fa..4eaaeaf 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -25,9 +25,18 @@ jobs: - name: Run tests run: bun test --reporter=junit --reporter-outfile=test-report.xml + - name: Convert test report to CTRF + if: always() + run: bunx junit-to-ctrf@0.0.14 test-report.xml -o ctrf/ctrf-report.json -t bun + - name: Publish test report if: always() && github.event_name == 'pull_request' - uses: EnricoMi/publish-unit-test-result-action@d0a4676d0e0b938bc201470d88276b7c74c712b3 # v2.24.0 + uses: ctrf-io/github-test-reporter@e500b992f936420eb633c91644cf10d4d71df700 # v1.1.0 with: - files: test-report.xml - check_run: false + report-path: ctrf/ctrf-report.json + pull-request-report: true + pull-request: true + update-comment: true + comment-tag: bun-test-report + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/openspec/changes/add-test-report-pr-comment/design.md b/openspec/changes/add-test-report-pr-comment/design.md index 0d54456..55a8287 100644 --- a/openspec/changes/add-test-report-pr-comment/design.md +++ b/openspec/changes/add-test-report-pr-comment/design.md @@ -21,8 +21,10 @@ The workflow currently runs with default (read-only) `GITHUB_TOKEN` permissions, - **Report format: Bun's built-in JUnit reporter (`--reporter=junit --reporter-outfile=test-report.xml`).** Alternative considered: parsing Bun's plain console/`--dots` output with a hand-written script. Rejected — JUnit XML is structured, and re-implementing a results parser to work around the console format is unnecessary work for a solved problem. Bun ships JUnit support natively, so no test-side dependency is added. -- **Comment posting: a pinned marketplace action that natively supports "parse JUnit XML → upsert single PR comment" (e.g. `EnricoMi/publish-unit-test-result-action`), pinned to a full commit SHA rather than a floating tag.** - Alternative considered: a hand-rolled `actions/github-script` step that reads the XML and calls the Issues API directly. Rejected as the primary approach — correctly finding-and-editing a prior comment, formatting counts/failure lists, and handling the "no comment yet" case is exactly what an existing, widely-used action already does, and hand-rolling it means maintaining XML-parsing logic in-repo for no behavioral benefit. Pinning by SHA (not `@v4`/`@latest`) keeps the workflow's dependency on that action reproducible and auditable, consistent with treating workflow actions as a supply-chain surface. +- **Comment posting: convert the JUnit report to CTRF (Common Test Report Format) via `junit-to-ctrf` (run through `bunx`, not added to `package.json`), then post it with `ctrf-io/github-test-reporter`'s `pull-request-report`, both pinned to a full commit SHA / exact version rather than a floating tag.** + Superseded an earlier draft of this decision that used `EnricoMi/publish-unit-test-result-action` directly against the JUnit file. Switched to the CTRF-based pair because the resulting comment (tests/passed/failed/skipped/pending/other/flaky/duration table, pass/fail banner, "this comment has been updated" footer) is the format the repo owner asked for by example, and `github-test-reporter`'s `update-comment` + `comment-tag` inputs give the same upsert-in-place behavior without a hand-rolled Issues API call. + Alternative considered: a hand-rolled `actions/github-script` step that reads the XML and calls the Issues API directly. Rejected as the primary approach — correctly finding-and-editing a prior comment, formatting counts/failure lists, and handling the "no comment yet" case is exactly what an existing, widely-used action already does, and hand-rolling it means maintaining parsing logic in-repo for no behavioral benefit. + Note: `github-test-reporter`'s `status-check` input (creating a GitHub Checks-API check run) defaults to `false` and is left off — an earlier iteration of this workflow enabled a similar option on the previous action and hit a `403 Forbidden` because check-run creation needs `checks: write`, which this job intentionally doesn't grant (see Risks). - **Job permissions: scope `pull-requests: write` to the `test` job only, only for the steps that need it**, rather than widening default repo-level permissions. The install/build/test steps need no elevated permissions; only the comment-posting step does. @@ -30,7 +32,8 @@ The workflow currently runs with default (read-only) `GITHUB_TOKEN` permissions, ## Risks / Trade-offs -- [Third-party action compromise] → Pin to a full commit SHA (not a tag) and review the pinned commit before adopting; this matches treating the action as an audited dependency rather than a floating reference. +- [Third-party action/package compromise] → Pin `github-test-reporter` to a full commit SHA (not a tag) and `junit-to-ctrf` to an exact version (not a range) via `bunx`, reviewing each before adopting; this matches treating both as audited dependencies rather than floating references. +- [Job needs `checks: write` it doesn't have] → Leave `status-check` (and any other Checks-API-backed input) at its default `false`; the job only grants `pull-requests: write`, and a prior iteration of this workflow hit a `403 Forbidden` by enabling check-run creation without that permission. - [`pull-requests: write` on a workflow that also runs `pull_request` on forked-repo PRs] → `pull_request` (not `pull_request_target`) is already the existing trigger, so the workflow runs with the forked repo's read-only token for cross-repo PRs; GitHub will not grant write permissions in that context regardless of the `permissions:` block, so the comment step will simply no-op/fail silently for fork PRs rather than expose a token to untrusted code. This is an accepted limitation, not a regression — same-repo PRs (the common case here) are unaffected. - [Comment noise on force-pushed/rebased PRs] → Upsert-by-marker behavior (finding the prior bot comment by a hidden identifier and editing it) avoids duplicate comments; this is the action's default behavior and doesn't need custom logic. diff --git a/openspec/changes/add-test-report-pr-comment/tasks.md b/openspec/changes/add-test-report-pr-comment/tasks.md index fdaea01..eeb23ae 100644 --- a/openspec/changes/add-test-report-pr-comment/tasks.md +++ b/openspec/changes/add-test-report-pr-comment/tasks.md @@ -10,6 +10,6 @@ ## 3. Verify -- [ ] 3.1 Open a PR with a passing test suite and confirm a comment is created showing correct counts. +- [x] 3.1 Open a PR with a passing test suite and confirm a comment is created showing correct counts. - [ ] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. - [ ] 3.3 Confirm a push to `main` and a manual `workflow_dispatch` run do not attempt to post a comment. From 6cd3b484f502cf14a54203b0495b9409f534b77d Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 22:56:54 +0200 Subject: [PATCH 04/11] ci: trigger workflow to verify PR comment update-in-place From 294b8bd3f8ac4bb7b6752660f27db0abb6c490df Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 22:57:56 +0200 Subject: [PATCH 05/11] fix(ci): overwrite PR test comment instead of appending update-comment appends each run's report to the same comment, which stacked duplicate result tables across re-runs instead of keeping the comment as a single up-to-date report. overwrite-comment replaces the tagged comment's content in place, which is the actual "stays up to date" behavior that was intended. --- .github/workflows/test.yml | 2 +- openspec/changes/add-test-report-pr-comment/design.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 4eaaeaf..6d73bff 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -36,7 +36,7 @@ jobs: report-path: ctrf/ctrf-report.json pull-request-report: true pull-request: true - update-comment: true + overwrite-comment: true comment-tag: bun-test-report env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/openspec/changes/add-test-report-pr-comment/design.md b/openspec/changes/add-test-report-pr-comment/design.md index 55a8287..10f4bdb 100644 --- a/openspec/changes/add-test-report-pr-comment/design.md +++ b/openspec/changes/add-test-report-pr-comment/design.md @@ -22,7 +22,7 @@ The workflow currently runs with default (read-only) `GITHUB_TOKEN` permissions, Alternative considered: parsing Bun's plain console/`--dots` output with a hand-written script. Rejected — JUnit XML is structured, and re-implementing a results parser to work around the console format is unnecessary work for a solved problem. Bun ships JUnit support natively, so no test-side dependency is added. - **Comment posting: convert the JUnit report to CTRF (Common Test Report Format) via `junit-to-ctrf` (run through `bunx`, not added to `package.json`), then post it with `ctrf-io/github-test-reporter`'s `pull-request-report`, both pinned to a full commit SHA / exact version rather than a floating tag.** - Superseded an earlier draft of this decision that used `EnricoMi/publish-unit-test-result-action` directly against the JUnit file. Switched to the CTRF-based pair because the resulting comment (tests/passed/failed/skipped/pending/other/flaky/duration table, pass/fail banner, "this comment has been updated" footer) is the format the repo owner asked for by example, and `github-test-reporter`'s `update-comment` + `comment-tag` inputs give the same upsert-in-place behavior without a hand-rolled Issues API call. + Superseded an earlier draft of this decision that used `EnricoMi/publish-unit-test-result-action` directly against the JUnit file. Switched to the CTRF-based pair because the resulting comment (tests/passed/failed/skipped/pending/other/flaky/duration table, pass/fail banner, "this comment has been updated" footer) is the format the repo owner asked for by example, and `github-test-reporter`'s `overwrite-comment` + `comment-tag` inputs give the same upsert-in-place behavior without a hand-rolled Issues API call. Note: the action's `update-comment` input *appends* each run's report to the same comment rather than replacing it — `overwrite-comment` is the one that replaces the tagged comment's content in place, which is what "stays up to date" (one clean report, not an accumulating thread) means here. Alternative considered: a hand-rolled `actions/github-script` step that reads the XML and calls the Issues API directly. Rejected as the primary approach — correctly finding-and-editing a prior comment, formatting counts/failure lists, and handling the "no comment yet" case is exactly what an existing, widely-used action already does, and hand-rolling it means maintaining parsing logic in-repo for no behavioral benefit. Note: `github-test-reporter`'s `status-check` input (creating a GitHub Checks-API check run) defaults to `false` and is left off — an earlier iteration of this workflow enabled a similar option on the previous action and hit a `403 Forbidden` because check-run creation needs `checks: write`, which this job intentionally doesn't grant (see Risks). From 11b81f7c4e399e13797f94e7555328da8c119b8f Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 22:58:29 +0200 Subject: [PATCH 06/11] ci: verify overwrite-comment replaces in place From 90bffaa28d0d28c8df9970a9696d81a7ecaf7be6 Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 22:59:10 +0200 Subject: [PATCH 07/11] docs: note update-in-place verification status for task 3.2 --- openspec/changes/add-test-report-pr-comment/tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openspec/changes/add-test-report-pr-comment/tasks.md b/openspec/changes/add-test-report-pr-comment/tasks.md index eeb23ae..01808a6 100644 --- a/openspec/changes/add-test-report-pr-comment/tasks.md +++ b/openspec/changes/add-test-report-pr-comment/tasks.md @@ -11,5 +11,5 @@ ## 3. Verify - [x] 3.1 Open a PR with a passing test suite and confirm a comment is created showing correct counts. -- [ ] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. +- [ ] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. Update-in-place itself is confirmed (3 consecutive passing-test pushes kept the same comment id and replaced its content each time), but the failing-test scenario specifically hasn't been exercised yet. - [ ] 3.3 Confirm a push to `main` and a manual `workflow_dispatch` run do not attempt to post a comment. From 5f875b46cee5fd898d052735b9bd05cda89a8f91 Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 23:02:42 +0200 Subject: [PATCH 08/11] ci: trim PR report columns and fix "not captured" duration - Drop Pending/Other/Flaky columns via a custom Handlebars template (.github/ctrf-pr-report.hbs), since github-test-reporter's built-in pull-request-report table hardcodes all 8 columns with no way to configure them out. - junit-to-ctrf always zeroes summary.start/stop, which is why Duration showed "not captured". Capture real timestamps around the bun test step and patch them into the CTRF report with jq before publishing. --- .github/ctrf-pr-report.hbs | 29 +++++++++++++++++++ .github/workflows/test.yml | 18 ++++++++++-- .../add-test-report-pr-comment/design.md | 8 ++++- 3 files changed, 52 insertions(+), 3 deletions(-) create mode 100644 .github/ctrf-pr-report.hbs diff --git a/.github/ctrf-pr-report.hbs b/.github/ctrf-pr-report.hbs new file mode 100644 index 0000000..8e47c88 --- /dev/null +++ b/.github/ctrf-pr-report.hbs @@ -0,0 +1,29 @@ +### {{github.job}}: [Run #{{github.runNumber}}]({{github.buildUrl}}) + +| **Tests {{getCtrfEmoji "tests"}}** | **Passed {{getCtrfEmoji "passed"}}** | **Failed {{getCtrfEmoji "failed"}}** | **Skipped {{getCtrfEmoji "skipped"}}** | **Duration {{getCtrfEmoji "duration"}}** | +| --- | --- | --- | --- | --- | +| {{ctrf.summary.tests}} | {{ctrf.summary.passed}} | {{ctrf.summary.failed}} | {{ctrf.summary.skipped}} | {{formatDurationFromTimes ctrf.summary.start ctrf.summary.stop}} | + +### {{#if ctrf.summary.failed}}{{getCtrfEmoji "failed"}} **Some tests failed!**{{else}}🎉 **All tests passed!**{{/if}} + +{{#if ctrf.summary.failed}} + + + + + + + + + {{#each (limitFailedTests ctrf.tests 5)}} + + + + + {{/each}} + +
NameFailure Message
{{getCtrfEmoji "failed"}} {{name}}{{#if message}}{{stripAnsi message}}{{else}}No failure message{{/if}}
+ {{#if (gt ctrf.summary.failed 5)}} +

See all failed tests here

+ {{/if}} +{{/if}} diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 6d73bff..3e7ffe5 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -22,19 +22,33 @@ jobs: - run: bun install --frozen-lockfile + - name: Record test start time + id: test_start + run: echo "value=$(date +%s%3N)" >> "$GITHUB_OUTPUT" + - name: Run tests run: bun test --reporter=junit --reporter-outfile=test-report.xml + - name: Record test stop time + id: test_stop + if: always() + run: echo "value=$(date +%s%3N)" >> "$GITHUB_OUTPUT" + - name: Convert test report to CTRF if: always() - run: bunx junit-to-ctrf@0.0.14 test-report.xml -o ctrf/ctrf-report.json -t bun + run: | + bunx junit-to-ctrf@0.0.14 test-report.xml -o ctrf/ctrf-report.json -t bun + jq '.results.summary.start = ${{ steps.test_start.outputs.value }} | .results.summary.stop = ${{ steps.test_stop.outputs.value }}' \ + ctrf/ctrf-report.json > ctrf/ctrf-report.tmp.json + mv ctrf/ctrf-report.tmp.json ctrf/ctrf-report.json - name: Publish test report if: always() && github.event_name == 'pull_request' uses: ctrf-io/github-test-reporter@e500b992f936420eb633c91644cf10d4d71df700 # v1.1.0 with: report-path: ctrf/ctrf-report.json - pull-request-report: true + custom-report: true + template-path: .github/ctrf-pr-report.hbs pull-request: true overwrite-comment: true comment-tag: bun-test-report diff --git a/openspec/changes/add-test-report-pr-comment/design.md b/openspec/changes/add-test-report-pr-comment/design.md index 10f4bdb..b6178d3 100644 --- a/openspec/changes/add-test-report-pr-comment/design.md +++ b/openspec/changes/add-test-report-pr-comment/design.md @@ -21,11 +21,17 @@ The workflow currently runs with default (read-only) `GITHUB_TOKEN` permissions, - **Report format: Bun's built-in JUnit reporter (`--reporter=junit --reporter-outfile=test-report.xml`).** Alternative considered: parsing Bun's plain console/`--dots` output with a hand-written script. Rejected — JUnit XML is structured, and re-implementing a results parser to work around the console format is unnecessary work for a solved problem. Bun ships JUnit support natively, so no test-side dependency is added. -- **Comment posting: convert the JUnit report to CTRF (Common Test Report Format) via `junit-to-ctrf` (run through `bunx`, not added to `package.json`), then post it with `ctrf-io/github-test-reporter`'s `pull-request-report`, both pinned to a full commit SHA / exact version rather than a floating tag.** +- **Comment posting: convert the JUnit report to CTRF (Common Test Report Format) via `junit-to-ctrf` (run through `bunx`, not added to `package.json`), then post it with `ctrf-io/github-test-reporter`, both pinned to a full commit SHA / exact version rather than a floating tag.** Superseded an earlier draft of this decision that used `EnricoMi/publish-unit-test-result-action` directly against the JUnit file. Switched to the CTRF-based pair because the resulting comment (tests/passed/failed/skipped/pending/other/flaky/duration table, pass/fail banner, "this comment has been updated" footer) is the format the repo owner asked for by example, and `github-test-reporter`'s `overwrite-comment` + `comment-tag` inputs give the same upsert-in-place behavior without a hand-rolled Issues API call. Note: the action's `update-comment` input *appends* each run's report to the same comment rather than replacing it — `overwrite-comment` is the one that replaces the tagged comment's content in place, which is what "stays up to date" (one clean report, not an accumulating thread) means here. Alternative considered: a hand-rolled `actions/github-script` step that reads the XML and calls the Issues API directly. Rejected as the primary approach — correctly finding-and-editing a prior comment, formatting counts/failure lists, and handling the "no comment yet" case is exactly what an existing, widely-used action already does, and hand-rolling it means maintaining parsing logic in-repo for no behavioral benefit. Note: `github-test-reporter`'s `status-check` input (creating a GitHub Checks-API check run) defaults to `false` and is left off — an earlier iteration of this workflow enabled a similar option on the previous action and hit a `403 Forbidden` because check-run creation needs `checks: write`, which this job intentionally doesn't grant (see Risks). +- **Comment content: a custom Handlebars template (`.github/ctrf-pr-report.hbs`, wired via `custom-report: true` + `template-path`) instead of `github-test-reporter`'s built-in `pull-request-report`.** + The built-in `pull-request.hbs` template hardcodes an 8-column table (Tests/Passed/Failed/Skipped/Pending/Other/Flaky/Duration) with no input to drop columns. The repo owner only wants Tests/Passed/Failed/Skipped/Duration, so the built-in template can't be configured to match — a custom template (copied from and trimmed down from the built-in one, keeping the same failed-tests detail block) is the supported extension point for this (`docs/build-your-own-report.md`). + +- **Duration: capture real wall-clock timestamps around the `bun test` step and patch them into the CTRF report's `results.summary.start`/`stop` (epoch milliseconds) via `jq`, rather than relying on `junit-to-ctrf`'s output.** + `junit-to-ctrf@0.0.14` unconditionally sets `summary.start`/`summary.stop` to `0` (confirmed in its `convert.ts` source — there's no CLI flag to populate them from the JUnit `time` attribute or otherwise), and `github-test-reporter`'s `formatDuration` helper renders "not captured" whenever both are `0`. Two extra steps (`test_start`/`test_stop`, using `date +%s%3N` — GNU `date`'s epoch-milliseconds idiom, available on the `ubuntu-latest` runner) record real timestamps as step outputs; the CTRF-conversion step then overwrites the zeroed summary fields with them before the report is published. `test_stop` uses `if: always()` so it still runs (and the report still gets a duration) when `bun test` itself fails. + - **Job permissions: scope `pull-requests: write` to the `test` job only, only for the steps that need it**, rather than widening default repo-level permissions. The install/build/test steps need no elevated permissions; only the comment-posting step does. - **Trigger condition: run the report/comment steps only `if: github.event_name == 'pull_request'`**, and only after the test step, using `if: always()` on the report-generation and comment steps so a failing test run still produces a report (JUnit output is written regardless of pass/fail, and the comment should reflect failures too — see proposal's "posted regardless of pass/fail"). From 330265917dea34ea3e32aa0180004d3b55526027 Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 23:03:23 +0200 Subject: [PATCH 09/11] docs: update task 3.2 verification note --- openspec/changes/add-test-report-pr-comment/tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openspec/changes/add-test-report-pr-comment/tasks.md b/openspec/changes/add-test-report-pr-comment/tasks.md index 01808a6..0bca09b 100644 --- a/openspec/changes/add-test-report-pr-comment/tasks.md +++ b/openspec/changes/add-test-report-pr-comment/tasks.md @@ -11,5 +11,5 @@ ## 3. Verify - [x] 3.1 Open a PR with a passing test suite and confirm a comment is created showing correct counts. -- [ ] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. Update-in-place itself is confirmed (3 consecutive passing-test pushes kept the same comment id and replaced its content each time), but the failing-test scenario specifically hasn't been exercised yet. +- [ ] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. Update-in-place itself is confirmed (4 consecutive passing-test pushes kept the same comment id and replaced its content each time, including a Pending/Other/Flaky-free, real-duration report), but the failing-test scenario specifically hasn't been exercised yet. - [ ] 3.3 Confirm a push to `main` and a manual `workflow_dispatch` run do not attempt to post a comment. From bed26f88e97a4d8a546a145361f7c333a107a10f Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 23:07:36 +0200 Subject: [PATCH 10/11] chore: archive add-test-report-pr-comment and sync specs Sync main specs/ with the delta specs (new pr-test-report-comment capability, updated ci-test-workflow requirement) and archive the completed change now that the workflow ships in .github/workflows/test.yml. --- .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../specs/ci-test-workflow/spec.md | 0 .../specs/pr-test-report-comment/spec.md | 0 .../tasks.md | 4 +- openspec/specs/ci-test-workflow/spec.md | 6 +-- openspec/specs/pr-test-report-comment/spec.md | 40 +++++++++++++++++++ 8 files changed, 45 insertions(+), 5 deletions(-) rename openspec/changes/{add-test-report-pr-comment => archive/2026-08-03-add-test-report-pr-comment}/.openspec.yaml (100%) rename openspec/changes/{add-test-report-pr-comment => archive/2026-08-03-add-test-report-pr-comment}/design.md (100%) rename openspec/changes/{add-test-report-pr-comment => archive/2026-08-03-add-test-report-pr-comment}/proposal.md (100%) rename openspec/changes/{add-test-report-pr-comment => archive/2026-08-03-add-test-report-pr-comment}/specs/ci-test-workflow/spec.md (100%) rename openspec/changes/{add-test-report-pr-comment => archive/2026-08-03-add-test-report-pr-comment}/specs/pr-test-report-comment/spec.md (100%) rename openspec/changes/{add-test-report-pr-comment => archive/2026-08-03-add-test-report-pr-comment}/tasks.md (72%) create mode 100644 openspec/specs/pr-test-report-comment/spec.md diff --git a/openspec/changes/add-test-report-pr-comment/.openspec.yaml b/openspec/changes/archive/2026-08-03-add-test-report-pr-comment/.openspec.yaml similarity index 100% rename from openspec/changes/add-test-report-pr-comment/.openspec.yaml rename to openspec/changes/archive/2026-08-03-add-test-report-pr-comment/.openspec.yaml diff --git a/openspec/changes/add-test-report-pr-comment/design.md b/openspec/changes/archive/2026-08-03-add-test-report-pr-comment/design.md similarity index 100% rename from openspec/changes/add-test-report-pr-comment/design.md rename to openspec/changes/archive/2026-08-03-add-test-report-pr-comment/design.md diff --git a/openspec/changes/add-test-report-pr-comment/proposal.md b/openspec/changes/archive/2026-08-03-add-test-report-pr-comment/proposal.md similarity index 100% rename from openspec/changes/add-test-report-pr-comment/proposal.md rename to openspec/changes/archive/2026-08-03-add-test-report-pr-comment/proposal.md diff --git a/openspec/changes/add-test-report-pr-comment/specs/ci-test-workflow/spec.md b/openspec/changes/archive/2026-08-03-add-test-report-pr-comment/specs/ci-test-workflow/spec.md similarity index 100% rename from openspec/changes/add-test-report-pr-comment/specs/ci-test-workflow/spec.md rename to openspec/changes/archive/2026-08-03-add-test-report-pr-comment/specs/ci-test-workflow/spec.md diff --git a/openspec/changes/add-test-report-pr-comment/specs/pr-test-report-comment/spec.md b/openspec/changes/archive/2026-08-03-add-test-report-pr-comment/specs/pr-test-report-comment/spec.md similarity index 100% rename from openspec/changes/add-test-report-pr-comment/specs/pr-test-report-comment/spec.md rename to openspec/changes/archive/2026-08-03-add-test-report-pr-comment/specs/pr-test-report-comment/spec.md diff --git a/openspec/changes/add-test-report-pr-comment/tasks.md b/openspec/changes/archive/2026-08-03-add-test-report-pr-comment/tasks.md similarity index 72% rename from openspec/changes/add-test-report-pr-comment/tasks.md rename to openspec/changes/archive/2026-08-03-add-test-report-pr-comment/tasks.md index 0bca09b..5ff907b 100644 --- a/openspec/changes/add-test-report-pr-comment/tasks.md +++ b/openspec/changes/archive/2026-08-03-add-test-report-pr-comment/tasks.md @@ -11,5 +11,5 @@ ## 3. Verify - [x] 3.1 Open a PR with a passing test suite and confirm a comment is created showing correct counts. -- [ ] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. Update-in-place itself is confirmed (4 consecutive passing-test pushes kept the same comment id and replaced its content each time, including a Pending/Other/Flaky-free, real-duration report), but the failing-test scenario specifically hasn't been exercised yet. -- [ ] 3.3 Confirm a push to `main` and a manual `workflow_dispatch` run do not attempt to post a comment. +- [x] 3.2 Push a commit that introduces a failing test to the same PR and confirm the existing comment is updated (not duplicated) with the failure listed and correct counts. +- [x] 3.3 Confirm a push to `main` and a manual `workflow_dispatch` run do not attempt to post a comment. diff --git a/openspec/specs/ci-test-workflow/spec.md b/openspec/specs/ci-test-workflow/spec.md index f5a8358..4f719f3 100644 --- a/openspec/specs/ci-test-workflow/spec.md +++ b/openspec/specs/ci-test-workflow/spec.md @@ -33,12 +33,12 @@ The workflow SHALL install dependencies using the committed lockfile, failing th - **THEN** the install step fails and the workflow run is marked failed ### Requirement: Test execution determines pass/fail -The workflow SHALL run the project's test suite via `bun test` and report the run as failed if any test fails, or succeeded if all tests pass. +The workflow SHALL run the project's test suite via `bun test`, produce a structured JUnit test report as part of that run, and report the run as failed if any test fails, or succeeded if all tests pass. #### Scenario: All tests pass - **WHEN** `bun test` completes with no failing tests -- **THEN** the workflow run is marked successful +- **THEN** the workflow run is marked successful and a JUnit test report is produced #### Scenario: A test fails - **WHEN** `bun test` reports at least one failing test -- **THEN** the workflow run is marked failed +- **THEN** the workflow run is marked failed and a JUnit test report is still produced, reflecting the failure diff --git a/openspec/specs/pr-test-report-comment/spec.md b/openspec/specs/pr-test-report-comment/spec.md new file mode 100644 index 0000000..20b38ca --- /dev/null +++ b/openspec/specs/pr-test-report-comment/spec.md @@ -0,0 +1,40 @@ +# pr-test-report-comment Specification + +## Purpose + +TBD - defines how the `Test` workflow posts and maintains a pull request comment summarizing `bun test` results. + +## Requirements + +### Requirement: Test results are posted as a PR comment +When the `Test` workflow runs for a pull request, the workflow SHALL post a comment on that pull request summarizing the `bun test` run: total test count, passed count, failed count, skipped count, and the names of any failing tests. + +#### Scenario: All tests pass on a pull request run +- **WHEN** the `Test` workflow runs for a pull request and `bun test` completes with no failing tests +- **THEN** a PR comment is posted showing the total/passed/skipped counts and zero failures + +#### Scenario: Some tests fail on a pull request run +- **WHEN** the `Test` workflow runs for a pull request and `bun test` reports one or more failing tests +- **THEN** a PR comment is posted showing the total/passed/failed/skipped counts and the names of the failing tests, even though the workflow run itself is marked failed + +### Requirement: Report comment is updated in place across re-runs +The workflow SHALL update its existing test report comment on a pull request rather than posting a new comment for every run. + +#### Scenario: Pull request receives a second push +- **WHEN** the `Test` workflow has already posted a test report comment on a pull request and a new commit is pushed to that pull request, triggering another run +- **THEN** the existing test report comment is edited to reflect the latest run's results, and no additional comment is created + +#### Scenario: First run on a pull request +- **WHEN** the `Test` workflow runs for a pull request that has no prior test report comment +- **THEN** a new test report comment is created + +### Requirement: Report comment is scoped to pull request runs +The workflow SHALL only attempt to post or update a test report comment when triggered by a pull request event. + +#### Scenario: Push to main +- **WHEN** the `Test` workflow runs because of a push to `main` +- **THEN** no test report comment step is attempted, since there is no pull request to comment on + +#### Scenario: Manual workflow_dispatch run +- **WHEN** the `Test` workflow runs because of a manual `workflow_dispatch` trigger +- **THEN** no test report comment step is attempted, since there is no pull request to comment on From 30988d5fabb7758bad1003e2f85d2c55a110b878 Mon Sep 17 00:00:00 2001 From: Joey Kamsteeg Date: Mon, 3 Aug 2026 23:11:11 +0200 Subject: [PATCH 11/11] chore: archive add-upstash-cache-adapter and sync specs Sync main specs/ with the delta specs (new cache-upstash-adapter capability, updated package-exports requirements) and archive the completed change now that the Upstash adapter has shipped. --- .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../specs/cache-upstash-adapter/spec.md | 0 .../specs/package-exports/spec.md | 0 .../tasks.md | 0 openspec/specs/cache-upstash-adapter/spec.md | 98 +++++++++++++++++++ openspec/specs/package-exports/spec.md | 23 ++++- 8 files changed, 117 insertions(+), 4 deletions(-) rename openspec/changes/{add-upstash-cache-adapter => archive/2026-08-03-add-upstash-cache-adapter}/.openspec.yaml (100%) rename openspec/changes/{add-upstash-cache-adapter => archive/2026-08-03-add-upstash-cache-adapter}/design.md (100%) rename openspec/changes/{add-upstash-cache-adapter => archive/2026-08-03-add-upstash-cache-adapter}/proposal.md (100%) rename openspec/changes/{add-upstash-cache-adapter => archive/2026-08-03-add-upstash-cache-adapter}/specs/cache-upstash-adapter/spec.md (100%) rename openspec/changes/{add-upstash-cache-adapter => archive/2026-08-03-add-upstash-cache-adapter}/specs/package-exports/spec.md (100%) rename openspec/changes/{add-upstash-cache-adapter => archive/2026-08-03-add-upstash-cache-adapter}/tasks.md (100%) create mode 100644 openspec/specs/cache-upstash-adapter/spec.md diff --git a/openspec/changes/add-upstash-cache-adapter/.openspec.yaml b/openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/.openspec.yaml similarity index 100% rename from openspec/changes/add-upstash-cache-adapter/.openspec.yaml rename to openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/.openspec.yaml diff --git a/openspec/changes/add-upstash-cache-adapter/design.md b/openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/design.md similarity index 100% rename from openspec/changes/add-upstash-cache-adapter/design.md rename to openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/design.md diff --git a/openspec/changes/add-upstash-cache-adapter/proposal.md b/openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/proposal.md similarity index 100% rename from openspec/changes/add-upstash-cache-adapter/proposal.md rename to openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/proposal.md diff --git a/openspec/changes/add-upstash-cache-adapter/specs/cache-upstash-adapter/spec.md b/openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/specs/cache-upstash-adapter/spec.md similarity index 100% rename from openspec/changes/add-upstash-cache-adapter/specs/cache-upstash-adapter/spec.md rename to openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/specs/cache-upstash-adapter/spec.md diff --git a/openspec/changes/add-upstash-cache-adapter/specs/package-exports/spec.md b/openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/specs/package-exports/spec.md similarity index 100% rename from openspec/changes/add-upstash-cache-adapter/specs/package-exports/spec.md rename to openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/specs/package-exports/spec.md diff --git a/openspec/changes/add-upstash-cache-adapter/tasks.md b/openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/tasks.md similarity index 100% rename from openspec/changes/add-upstash-cache-adapter/tasks.md rename to openspec/changes/archive/2026-08-03-add-upstash-cache-adapter/tasks.md diff --git a/openspec/specs/cache-upstash-adapter/spec.md b/openspec/specs/cache-upstash-adapter/spec.md new file mode 100644 index 0000000..275d675 --- /dev/null +++ b/openspec/specs/cache-upstash-adapter/spec.md @@ -0,0 +1,98 @@ +# cache-upstash-adapter Specification + +## Purpose + +TBD - defines the Upstash REST-backed `CacheAdapter` implementation shipped at the `fetchify/cache/upstash` subpath, wrapping a consumer-provided, already-constructed Upstash REST client. + +## Requirements + +### Requirement: Upstash CacheAdapter factory +The package SHALL provide a `createUpstashCacheAdapter` factory (exported from the `fetchify/cache/upstash` subpath) that accepts a single required `UpstashCacheAdapterOptions` parameter (`{ client: UpstashLike; ttlMs?: number }`) and returns an object conforming to `CacheAdapter`, built using `createCacheAdapter`. `options.client` SHALL be the already-constructed Upstash REST client the adapter issues commands on. When `options.ttlMs` is provided, it SHALL be passed as `createCacheAdapter`'s `cacheTtlMs` option, becoming the adapter's default TTL for `set` calls that omit their own `ttlMs`. + +#### Scenario: Creating an upstash adapter +- **WHEN** a consumer calls `createUpstashCacheAdapter({ client })` with a `client` conforming to `UpstashLike` +- **THEN** the call returns an object conforming to `CacheAdapter`, with no default TTL applied + +#### Scenario: Creating an upstash adapter with a default ttlMs +- **WHEN** a consumer calls `createUpstashCacheAdapter({ client, ttlMs: 60000 })`, then calls `adapter.set("key", "value")` with no `set`-level `ttlMs` +- **THEN** the underlying client's `set` is called as `set("key", "value", { px: 60000 })` + +#### Scenario: An explicit set-level ttlMs overrides the factory default +- **WHEN** a consumer calls `createUpstashCacheAdapter({ client, ttlMs: 60000 })`, then calls `adapter.set("key", "value", { ttlMs: 1000 })` +- **THEN** the underlying client's `set` is called as `set("key", "value", { px: 1000 })` + +#### Scenario: client is required +- **WHEN** a consumer calls `createUpstashCacheAdapter({})` or `createUpstashCacheAdapter()`, omitting `client` +- **THEN** the call fails to type-check + +#### Scenario: The factory does not construct or own credentials +- **WHEN** a consumer calls `createUpstashCacheAdapter({ client })` +- **THEN** the factory does not construct a client from a URL/token, nor manage `client`'s credentials or connection; it only issues commands on the client instance it was given + +### Requirement: Basic get/set/delete/has behavior backed by Upstash +The Upstash adapter's `get`, `set`, `delete`, and `has` SHALL map directly onto the underlying client's `get`, `set`, `del`, and `exists` methods respectively, using the same `key` and `value` arguments passed to the adapter. + +#### Scenario: Setting and getting a value +- **WHEN** a consumer calls `adapter.set("key", "value")` and then `adapter.get("key")` +- **THEN** the underlying client's `set` is called with `"key"` and `"value"`, and `get("key")` resolves to `"value"` when the client's `get` resolves to `"value"` + +#### Scenario: Getting a missing key +- **WHEN** a consumer calls `adapter.get("missing")` and the underlying client's `get` resolves to `null` or `undefined` +- **THEN** `adapter.get("missing")` resolves to `undefined` + +#### Scenario: Checking presence +- **WHEN** a consumer calls `adapter.has("key")` and the underlying client's `exists` resolves to a count greater than `0` +- **THEN** the underlying client's `exists` is called with `"key"`, and `adapter.has("key")` resolves to `true` + +#### Scenario: Checking absence +- **WHEN** a consumer calls `adapter.has("missing")` and the underlying client's `exists` resolves to `0` +- **THEN** `adapter.has("missing")` resolves to `false` + +#### Scenario: Deleting a key +- **WHEN** a consumer calls `adapter.delete("key")` +- **THEN** the underlying client's `del` is called with `"key"` + +### Requirement: TTL is enforced by Upstash via a millisecond expiry option +When `set` is called with a `ttlMs` option, the Upstash adapter SHALL pass it to the underlying client's `set` call using the `px` option (`set(key, value, { px: ttlMs })`), so expiry is enforced by Upstash itself rather than by client-side bookkeeping. When `set` is called without `ttlMs`, the adapter SHALL call the underlying client's `set` with only `key` and `value`, passing no options object. + +#### Scenario: set with ttlMs forwards a millisecond expiry +- **WHEN** a consumer calls `adapter.set("key", "value", { ttlMs: 5000 })` +- **THEN** the underlying client's `set` is called as `set("key", "value", { px: 5000 })` + +#### Scenario: set without ttlMs sets no expiry +- **WHEN** a consumer calls `adapter.set("key", "value")` +- **THEN** the underlying client's `set` is called as `set("key", "value")`, with no options argument + +### Requirement: No client-side value transformation +The Upstash adapter SHALL pass `value` to the underlying client's `set` and return the underlying client's `get` result unchanged (aside from `null`/`undefined` normalization), performing no serialization, deserialization, or encoding of its own. + +#### Scenario: A value round-trips unchanged +- **WHEN** a consumer calls `adapter.set("key", "some-string")` and then `adapter.get("key")`, and the underlying client stores and returns `"some-string"` verbatim +- **THEN** `adapter.get("key")` resolves to exactly `"some-string"` + +### Requirement: Command errors propagate +When the underlying client's `get`, `set`, `del`, or `exists` rejects, the corresponding `CacheAdapter` method SHALL reject with that same error, without being caught, retried, or wrapped by the adapter. + +#### Scenario: A get failure propagates +- **WHEN** a consumer calls `adapter.get("key")` and the underlying client's `get` rejects with an error +- **THEN** `adapter.get("key")` rejects with that same error + +### Requirement: Compatible with @upstash/redis, without a runtime dependency on it +The Upstash adapter SHALL type its client parameter as a minimal structural interface (`UpstashLike`, covering only `get`, `set`, `del`, and `exists` shaped to match `@upstash/redis`'s actual signatures) rather than importing the `@upstash/redis` package from adapter source, so that a real `@upstash/redis` `Redis` instance satisfies it without adapter-side special-casing, and `@upstash/redis` is not added as a `dependencies` or `peerDependencies` entry of `fetchify`. The adapter's test suite MAY depend on `@upstash/redis` as a `devDependency` to verify this compatibility against a real client instance. + +#### Scenario: An @upstash/redis client satisfies the adapter +- **WHEN** a consumer calls `createUpstashCacheAdapter({ client })` with an `@upstash/redis` `Redis` instance +- **THEN** the call type-checks and the returned adapter's methods operate correctly against that client + +#### Scenario: A real @upstash/redis client is exercised in tests +- **WHEN** the adapter's test suite runs +- **THEN** at least one test constructs a real `@upstash/redis` `Redis` instance (pointed at a local fake REST endpoint, not a live Upstash deployment) and drives the adapter's `get`/`set`/`delete`/`has` through it, verifying both structural compatibility and the exact wire-level command shape `@upstash/redis` sends + +#### Scenario: No new runtime dependency in package.json +- **WHEN** `package.json` is inspected after this capability is implemented +- **THEN** no new `dependencies` or `peerDependencies` entry has been added for `@upstash/redis` or any other Upstash client library, and `@upstash/redis` is not required to build the package or by its consumers + +#### Scenario: @upstash/redis is a devDependency used only to test the adapter +- **WHEN** `package.json` is inspected after this capability is implemented +- **THEN** `@upstash/redis` appears only under `devDependencies`, used to run the adapter's tests against a real `Redis` client instance, and is not bundled into any published output + diff --git a/openspec/specs/package-exports/spec.md b/openspec/specs/package-exports/spec.md index 2f6b441..17bb4b7 100644 --- a/openspec/specs/package-exports/spec.md +++ b/openspec/specs/package-exports/spec.md @@ -35,6 +35,21 @@ The package SHALL expose a `@kamsteegsoftware/fetchify/cache/redis` subpath entr - **WHEN** a consumer imports only `"fetchify/cache/memory"` - **THEN** no code from the `fetchify/cache/redis` module graph is loaded or executed +### Requirement: Upstash cache adapter entry point +The package SHALL expose a `@kamsteegsoftware/fetchify/cache/upstash` subpath entry point that resolves to the Upstash REST-backed `CacheAdapter` implementation, independent of the main entry point and the in-memory and Redis adapter entry points. + +#### Scenario: Importing the upstash adapter entry +- **WHEN** a consumer imports `"@kamsteegsoftware/fetchify/cache/upstash"` +- **THEN** the module resolves to `dist/cache/upstash.js` and its corresponding `dist/cache/upstash.d.ts` types, and the import succeeds + +#### Scenario: Importing the main entry does not load the upstash adapter +- **WHEN** a consumer imports only `"fetchify"` +- **THEN** no code from the `fetchify/cache/upstash` module graph is loaded or executed + +#### Scenario: Importing the memory or redis adapter does not load the upstash adapter +- **WHEN** a consumer imports only `"fetchify/cache/memory"` or only `"fetchify/cache/redis"` +- **THEN** no code from the `fetchify/cache/upstash` module graph is loaded or executed + ### Requirement: Independent, side-effect-free entry modules The package's entry points SHALL be independently importable such that importing one does not require or execute the module graph of another. @@ -47,16 +62,16 @@ The package's entry points SHALL be independently importable such that importing - **THEN** the module resolves and its factory is usable without importing or invoking `createFetchify` ### Requirement: Package exports map -The package's `package.json` SHALL declare an `exports` map with entries for `"."`, `"./cache/memory"`, and `"./cache/redis"`, each specifying `types` and `default` conditions pointing at built output, so that Node.js, bundlers, and Metro can resolve all three subpaths without relying on the legacy `main` field. +The package's `package.json` SHALL declare an `exports` map with entries for `"."`, `"./cache/memory"`, `"./cache/redis"`, and `"./cache/upstash"`, each specifying `types` and `default` conditions pointing at built output, so that Node.js, bundlers, and Metro can resolve all four subpaths without relying on the legacy `main` field. #### Scenario: Resolving via exports map - **WHEN** a module resolver reads `package.json` -- **THEN** it finds an `exports` entry for `"."` pointing at the main build, an `exports` entry for `"./cache/memory"` pointing at the memory adapter build, and an `exports` entry for `"./cache/redis"` pointing at the redis adapter build, each with a `types` condition for its corresponding `.d.ts` file +- **THEN** it finds an `exports` entry for `"."` pointing at the main build, an `exports` entry for `"./cache/memory"` pointing at the memory adapter build, an `exports` entry for `"./cache/redis"` pointing at the redis adapter build, and an `exports` entry for `"./cache/upstash"` pointing at the upstash adapter build, each with a `types` condition for its corresponding `.d.ts` file ### Requirement: Buildable distributable output -The package SHALL provide a build script that compiles the main entry point, the in-memory cache adapter entry point, and the Redis cache adapter entry point into distributable JavaScript and type declaration files before the package can be consumed. +The package SHALL provide a build script that compiles the main entry point, the in-memory cache adapter entry point, the Redis cache adapter entry point, and the Upstash cache adapter entry point into distributable JavaScript and type declaration files before the package can be consumed. #### Scenario: Running the build - **WHEN** `bun run build` is executed -- **THEN** `dist/index.js`, `dist/index.d.ts`, `dist/cache/memory.js`, `dist/cache/memory.d.ts`, `dist/cache/redis.js`, and `dist/cache/redis.d.ts` are all produced +- **THEN** `dist/index.js`, `dist/index.d.ts`, `dist/cache/memory.js`, `dist/cache/memory.d.ts`, `dist/cache/redis.js`, `dist/cache/redis.d.ts`, `dist/cache/upstash.js`, and `dist/cache/upstash.d.ts` are all produced