Repository navigation
Harden import, validation, storage, outbound fetches, and deploy manifests (SC-06/07/13/14/15/16/19/22/23/24) - #82
Conversation
An export could carry any positive id, and an alert id of MaxInt64 parked the shared alerts sequence where SQLite has no id left to mint, breaking alert creation in every region. Document.Validate now holds alert, study, survey, and question ids (and a survey response's survey_id) below the floor the cutover runbook bumps the sequences to (1,000,000, OBACloud's ID_HEADROOM); `import --id-floor` matches a raised floor.
The alert url was stored and published verbatim, so javascript:, data:, protocol-relative, and relative values reached the feed and the link hrefs that render it. alerts.ValidateURL (empty, or absolute http(s) with a host, at most 2048 bytes) now guards admin create and patch (400), CLI alert create and edit, and import, where an invalid legacy url fails with an error naming the alert to fix in OBACloud.
Import re-validated enum names, question content, answers, and windows, but not the HTTP layer's limits, and the CLI read the document unbounded. Imported survey responses, push registrations, and ghost bus reports now meet the same caps the rider API applies (identifier and token lengths, coordinate ranges, wait-duration choices, comment length, a URL-safe public id, a captured snapshot that is a bounded JSON object), each failure naming the row; a non-test device's description is normalized away as the registration API does. The caps move into the domain packages, with a test pinning the HTTP layer's copies to them, and the CLI reads at most 256 MiB.
Import treated any existing row at an exported id in the same region as the row being imported, so a locally authored alert at a colliding id was skipped and the export's translations were attached to it. Alerts, studies, and surveys now record imported_at (migration 00022), and a skip requires provenance: a row an earlier import wrote is skipped whatever its content (identical re-imports are no-ops; OBACloud-side edits after the bulk run keep the documented insert-if-absent behavior), a row imported before the column existed is recognized by its verbatim created_at and stamped, and anything else is ErrImportConflict with the whole document rolled back.
The SQLite file and its -wal/-shm siblings were created 0644 under the default umask though they hold password hashes, key digests, OBA keys, push tokens, and rider data. Open now creates a missing database at 0600 (SQLite gives -wal and -shm the database's own mode) and narrows existing files, and the container entrypoint sets umask 077 before Litestream restores or replicates, covering its page copies too. Litestream runs as the same user, so replication is unaffected.
The regions directory's obaBaseUrl and sidecarBaseUrl were stored without a scheme or host check, and OBA response bodies were read unbounded. A directory entry whose obaBaseUrl (or non-empty sidecarBaseUrl) is not an absolute http(s) URL with a host is now skipped like any other invalid entry -- plain http and loopback/private hosts stay allowed for local development and the OBACloud contract harness -- and the OBA client's transport caps every response body at 32 MiB (httpx.CapResponseBodies), surfacing ErrResponseTooLarge through the key-redacting error path.
csvsafe.Cell checked only the first byte, so "\n=1+1", " =1+1", a no-break space before '=', and a full-width = all passed through, though spreadsheets trim or skip those before deciding a cell is a formula. The guard now skips leading whitespace, control, and format characters and matches the ASCII triggers plus their full-width and small-form variants.
golang.org/x/crypto v0.55.0 carried GO-2026-6355/6354/5932 (unreachable from this code). Bump to v0.59.0 (go mod tidy also lifts x/sync, x/sys, x/term, x/text and writes the go directive as 1.26.0). Add `make vulncheck` with a pinned govulncheck and a scheduled workflow that runs it weekly and on go.mod changes to main, SHA-pinning its actions like the existing workflows. With Go 1.26.11 it reports no vulnerabilities; with an older local 1.26 patch it flags the standard library, which is the point.
gorush 1.22 defaults log.hide_token to true (config.go setDefaults) and builds both its inline sync-mode logs and its feedback webhook payloads from the masked log entries (logx.GetLogPushEntry, notify.SendNotification), so the sidecar's exact token match never pruned anything. render.yaml, render.staging.yaml, and compose.yaml now set GORUSH_LOG_HIDE_TOKEN=false (viper env prefix GORUSH, '.' -> '_'), pinned by a manifest test. A token containing '*' is treated as masked: the feedback webhook and SendBatch each log one ERROR naming the setting, and the webhook skips the delete.
…nload The Dockerfile's base images and the gorush service in render.yaml, render.staging.yaml, and compose.yaml used mutable tags, and the build piped the Litestream tarball from GitHub straight into tar. Every FROM is now tag@sha256 (multi-arch index digests resolved with docker buildx imagetools inspect; golang:1.26-alpine is Go 1.26.11), gorush is pinned by digest (digest-only form for Render), and the Litestream tarball is checked against the per-architecture SHA-256 from the v0.5.16 release's checksums.txt (cross-checked against the downloaded files) before it is unpacked. deploy/manifests_test.go is the CI lint: it fails on an unpinned FROM or third-party image or an unverified download. The repo's own ghcr.io/onebusaway/sidecar:main stays a tag; it is the deploy path.
|
Caution Review failedThe pull request is closed. Walkthrough
Merge Risk:
|
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. 🤖 Generated with Claude Code |
|
Maintainer sweep: this is ready on the merits, but I'm not merging it in this pass because it's authored by the account running the sweep, and I don't approve or merge my own PRs. It needs an approval and merge from another maintainer. What I checked myself:
Non-blocking, worth knowing before rollout:
|
The alerts feed published whatever url was stored, so a javascript: URL written straight into the database went out in alerts.pb and alerts.pbtext. BuildFeed now emits url only when alerts.ValidateURL accepts it (empty, or an absolute http/https URL with a host, at most 2048 bytes); otherwise the field is omitted and the alert still ships. internal/alerts/url.go is a byte-identical copy of the validator from PR #82 (harden-import-validation), which wires it into the admin API, the CLI and import. Identical additions on both branches merge cleanly in either order.
|
PR sweep disposition: blocked — not merged. I authored this PR, so the sweep can't approve or merge it. The code review above covers the current head To unblock: another maintainer reviews and approves, then merges it. |
# Conflicts: # internal/obaapi/obaapi_test.go
|



Hardens the import path, content validation, storage permissions, outbound-fetch bounds, and deploy manifests per the pre-launch security audit. One commit per finding, each test-first.
Findings
Document.Validaterefuses alert, study, survey, and question ids, plus a survey response'ssurvey_id, at or above the sequence floor. The floor isexport.DefaultIDFloor= 1,000,000, which is the runbook'ssequence bump --minand OBACloud'sSidecar::Cutover::ID_HEADROOM. A deployment that raised it passesimport --id-floor N. Translations, responses, push registrations, and ghost bus reports are keyed by language, public id, or token, so no carried id applies to them. Alarms and Live Activities are not in the document. Before this, an alert id of MaxInt64 bricked alert creation in every region.alerts.ValidateURL, which accepts an empty url or an absolute http(s) URL with a host, at most 2048 bytes. Admin create and patch (400), CLIalert createandalert edit, and import all use it. An invalid legacy url fails import with an error naming the alert.pushreg,ghostbus,surveys), and a test pins httpapi's copies to them. The CLI reads at most 256 MiB.ErrImportConflict. Semantics are below.sqlite.Opencreates a missing database at 0600 (SQLite gives-wal/-shmthe database's mode) and narrows existing files.entrypoint.shsetsumask 077before Litestream restores or replicates. Checked with the built image against an S3 stub: replication, wipe, restore all work, and every file is 0600 with.sidecar.db-litestreamat 0700.obaBaseUrl, or non-emptysidecarBaseUrl, is not an absolute http(s) URL with a host is skipped like any other invalid entry. Plain http and loopback or private hosts stay allowed on purpose: local dev and the OBACloud contract harness usehttp://127.0.0.1, and the directory URL is chosen by the operator. The OBA client's transport caps every response body at 32 MiB (httpx.CapResponseBodies), andErrResponseTooLargesurvives the key-redacting error path.csvsafe.Cellskips leading whitespace, control, and format characters (NBSP, BOM, ZWSP) and matches= + - @plus their full-width and small-form variants.golang.org/x/cryptov0.55.0 → v0.59.0.go mod tidyalso bumps x/sync, x/sys, x/term, and x/text, and writes the go directive as1.26.0. Addedmake vulncheck(govulncheck pinned at v1.9.0) and.github/workflows/govulncheck.yml, which runs weekly and on go.mod changes to main, with SHA-pinned actions. With Go 1.26.11: "No vulnerabilities found".log.hide_tokendefaults to true (config/config.go), and both the inline logs and the feedback webhook payload are built from the masked log entry (logx.GetLogPushEntry→notify.SendNotification→DispatchFeedback).render.yaml,render.staging.yaml, andcompose.yamlnow setGORUSH_LOG_HIDE_TOKEN=false(viper prefixGORUSH,.→_). A token containing*counts as masked: the webhook andSendBatcheach log one ERROR naming the setting, and the webhook skips the delete. Downside: gorush's own logs now carry full tokens.FROMis nowtag@sha256(multi-arch index digests;golang:1.26-alpineis Go 1.26.11). gorush is pinned by digest, using the digest-only form for Render. The Litestream tarball is checked withsha256sum -cagainst per-arch digests from the v0.5.16checksums.txt, cross-checked against downloads.deploy/manifests_test.gois the CI lint (make test). It skips only our ownghcr.io/onebusaway/sidecar:main, because that tag is how deploys happen. Built and verified on arm64; the litestream stage was also built for amd64.Import semantics (SC-14)
Migration
00022addsimported_attoalerts,studies, andsurveys, set to the instant an import first wrote the row. When an import finds a row in this region (and under the same parent) already holding an exported id:imported_atset → same row → skipped, whatever its content now. An identical re-import does nothing. An edit made in OBACloud after the bulk run is not carried. This is the delta's existing insert-if-absent behavior, and OBACloud's cutover preflight (Sidecar::ContentComparison#import_conflicts) already reports that drift before any flip. Translations added since still land.imported_atNULL butcreated_atequals the export's → a row imported by a binary older than 00022 → skipped, andimported_atis stamped. OBACloud never changescreated_at, and import carries it verbatim, so a bulk import on the old binary followed by a delta on this one keeps working with no backfill guess.ErrImportConflict. The transaction rolls back, so nothing from the document is written and no translations, surveys, or questions get attached to the unrelated row.Questions need no column. A question is only reached under a survey that already passed this check, and its
survey_idownership is compared as before. Rows keyed by public id or token (responses, ghost bus reports, push registrations) keep their existing checks: their keys are random, and a device re-registering directly is meant to be skipped.Validation
make checkgreen.SIDECAR_CONTRACT=1 SIDECAR_SRC=<this branch> bundle exec rspec spec/contract, obacloudsidecar-migration@ 0544ae06): 15 examples, 0 failures, 1 pending. The full-cutover example skips on this branch because main has noalerts.pbalias yet (Serve alerts.pb, weather.json and vehicles.json (production cutover blocker) #80). Against a scratch merge of this branch + Serve alerts.pb, weather.json and vehicles.json (production cutover blocker) #80 it runs and passes: preflights, the real deltasidecar-admin import, and step 5. With that merge, only the "lacks the alias" example skips.--dry-runthen for real, and re-runs were no-ops (0 added).created_atand stamped them, and left the sidecar-authored alert 1000003 unmarked. An export claiming id 1000003 was refused, first by the floor, then (with--id-floor 2000000) as a conflict, with no translations attached.Summary by CodeRabbit