Repository navigation
feat(cli): a feedback report names why the last check failed - #6717
miguel-heygen wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Verdict: REQUEST CHANGES · 1 blocker · 1 important · 0 nits · head adc5142
A check that throws never updates lastCheck, so feedback attaches the result of an older check. ↓ inline
Scope
Read all 6 files, plus check.ts (run/reportCheckFailure), writeConfig/withConfigLock, and the test-home setup. The test is isolated: the config cache means readConfig() reads back the in-process write. Codes are fixed rule ids, so no content leaks. The fails without the fix: test exists and fails on main. CI (CI, Windows, CodeQL) was still running at review time, so this verdict does not depend on it.
| exitCode: checkExitCode(report), | ||
| runId: getRunId(), | ||
| }); | ||
| recordLastCheck({ exitCode: checkExitCode(report), codes: failureCodes(report) }); |
There was a problem hiding this comment.
blocker: this only runs when buildReport is reached. A check that throws (CheckDeadlineError, CheckStoppedError, project lock, a pipeline crash) goes through reportCheckFailure in commands/check.ts and exits 1, but lastCheck keeps the previous run. Example: check passes on project A, then check on project B times out, then feedback sends check=exit0. The report says the newest check passed, which is the opposite of the truth, in the same empty-error case this PR targets.
Fix: in reportCheckFailure, record { exitCode: 1, codes: { [usage ? error.code : "runtime_error"]: 1 } }, or clear lastCheck when run starts. Add a test where runPipeline throws.
important: also store an at timestamp, the way RecentRenderRecord does. Without one, triage can't tell a check from a minute ago from one last week on another project.
Edit accuracy: accurate 271 (base branch 271), smooth 129 of thoseThe gate passes. Quarantined, measured but not gated (0) |
What changed
hyperframes feedbackalready attaches join keys to a report (fid,tid, recent render ids). It now also attachescheck=exit<code>:<error code>*<count>,...for the newest localcheck: the exit code and up to three error codes with how often each fired. Agents sometimes file a check failure with the error left as a placeholder, so the report named no reason and could not be looked up.checkrecords that summary in the local config next to the recent-renders ring after every run. Codes and counts only: no messages, no paths, no project content. Each code is cut to 24 characters so the keys stay inside the report's 500-character env cap.What I measured
feedback.telemetryJoinKeys.test.ts,feedback.test.ts,check.test.tsandcheckBrowser.test.tspass (233 tests). Withfeedback.ts,checkPipeline.tsandconfig.tsat main's versions the three new tests fail (156 pass, 3 fail). A full ring of render ids plus three long error codes stays under the cap.What I did NOT exercise
A real
checkthen a realfeedbackon one machine; the pipeline test reads the saved record and the key builder is tested on its own.Why under 100 lines
A 25-line summariser, a small config record, a 4-line join key and one pipeline call, plus the tests.
fails without the fix: remembers the failed check's error codes so a later feedback can name them