Skip to content

Sync: end an older release line where the next vulnerable range starts - #1330

Merged
simi merged 3 commits into
rubysec:masterfrom
Paul-Bob:fix/sync-patched-versions-ceiling
Nov 21, 2026
Merged

simi merged 3 commits into
rubysec:masterfrom
Paul-Bob:fix/sync-patched-versions-ceiling

Conversation

@Paul-Bob

@Paul-Bob Paul-Bob commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

The sync stops flagging a later minor of an older major that has the fix, and it flags 344 vulnerable releases it used to miss.

The script wrote ~> <first patched> for every release line except the newest, so avo 3.34.1 was flagged after a fix in 3.33.1. #1265 and #1329 corrected that by hand. It also sorted versions as text, so 2.10 came before 2.9 and vulnerable releases were marked as patched or unaffected.

I replayed the old and the new script over GitHub's data, 106,526 checks against every released version.

Before After
Vulnerable, not flagged 368 24
Flagged, not vulnerable 1,901 1,874

No vulnerable version loses its flag, on the replay or on 80,000 generated inputs. The rule went through three adversarial review passes. Four-segment fixes such as Rails 8.0.5.1 keep the plain ~> on purpose, because a later release can predate a backport (cgi 0.1.1 and 0.1.0.2).

Some context. A user on avo 3.34.1 was flagged by bundler-audit for two advisories with fixes in 3.33.1 and 3.33.2. I had my agent trace it and write this change. It ran in Claude Code, on Claude Opus 5.5 for the diagnosis and Claude Fable 5.2 for the change, its reviews and the replay.

For AI agents

Where

  • lib/github_advisory_sync.rb, patched_versions_for. Called from create and update. update fills patched_versions only when the saved file has none, so no existing file changes.
  • patched_until_next_range, with starts_after?, ends_by? and minor_line_ending_at. The rule for every release line except the newest.
  • range_order. The sort key of vulnerable_version_ranges_for, which decides what unaffected_versions_for returns.
  • spec/github_advisory_sync_spec.rb. 22 new examples, built with the generated helper.

Why this shape

For an older line with fix p, the code takes the lowest vulnerable range start s above p, from every range of the gem, with or without a fix.

Where s is Example Before After
at the next major, three-segment fix avo 3.33.1, next range >= 4.0.0.beta.1 ~> 3.33.1 ~> 3.33, >= 3.33.1
inside the span ~> p covers bundler 2.2.10, next range >= 2.2.11 ~> 2.2.10 >= 2.2.10, < 2.2.11
anywhere else activerecord 2.3.18, next range >= 3.1.0 ~> 2.3.18 ~> 2.3.18
  • The plain ~> p is the answer whenever the data is unclear: another range overlaps the fix, no range starts above it, or a version does not parse. Gem::Version.new raises on a bad string, and one of those would abort a whole sync run.
  • Ranges without a fix count as starts. The first commit read only lines with a fix, so an unfixed range inside a widened span (= 3.40.0 between avo 3.33.1 and 4.0) lost its flag.
  • Widening four-segment fixes to their patch line (Rails 8.0.5.1 to ~> 8.0.5, >= 8.0.5.1, the shape of Fix activestorage CVE-2026-66066 patched_versions #1191) was tried and taken out. On the replay it clears 536 more releases, and also cgi 0.1.1, released 2021-11-24, a year before the backported fix 0.1.0.2 of 2022-11-22. GitHub's range for that advisory is < 0.1.0.2, so nothing in the data tells the two shapes apart. Release dates would.
  • Writing >= p, < s for every line was tried and taken out. < 4.0.0 matches 4.0.0.rc1, so it cleared 1,158 prereleases of the next line, and it cleared release lines GitHub lists no fix for (activerecord 3.0.x between 2.3.18 and 3.1.12).
  • unaffected_versions starts below the lowest range start by version. A range with no lower bound, or with a start RubyGems cannot parse, means nothing is listed as unaffected. At equal starts > sorts after >= and =, so the start itself stays vulnerable.
  • The newest line stays >= newest, as before.

Invariants

  • A version inside one of GitHub's vulnerable ranges that the old output flagged stays flagged, through patched_versions and unaffected_versions together.
  • A release line GitHub lists no fix for stays flagged.
  • Each entry is one or two comma-separated ranges, as spec/versions_example.rb requires.

Verify

  • bundle exec rspec spec/github_advisory_sync_spec.rb gives 25 examples, 0 failures. On master, 7 of the 22 new examples fail. The other 15 hold the conservative cases.
  • Mutation check: 19 small changes to the rule (each guard removed, each bound operator swapped, each sort reverted) each make at least one example fail.
  • The replay is not in the repo. It loads the script from master and from this branch into separate modules, builds GitHubAdvisory objects from securityVulnerabilities(ecosystem: RUBYGEMS) (1,976 nodes and 1,196 advisory and gem pairs on 2026-11-20), and for the 393 pairs with more than one release line compares what each output flags with GitHub's ranges, for every version in https://rubygems.org/api/v1/versions/<gem>.json.
  • Replay results: 0 vulnerable versions lose a flag, 0 cleared releases predate their fix, 344 vulnerable versions gain a flag, and 33 releases stop being flagged in 7 pairs (avo, puppet, activeadmin, secure_headers). 6 prereleases just below a vulnerable range gain a flag (ember-source, solidus_api, solidus_frontend).
  • Differential fuzz of master against this branch: 60,000 inputs shaped like GitHub's and 20,000 malformed ones. No vulnerable version loses a flag, and nothing raises that did not raise before.
  • With gems/avo/CVE-2026-55518.yml removed, bundle exec rake "sync_github_advisories[avo]" writes ~> 3.32, >= 3.32.1 for it.

Follow-ups

  • 24 vulnerable versions are still not flagged: avo 3.0.0.pre1 to 3.0.0.pre12 in CVE-2023-34102 and CVE-2023-34103, where an unfixed range sits above the newest fix. Unchanged by this PR.
  • A single vulnerable version (= X) inside the span ~> p covers gives >= p, < X, so later releases of that minor are flagged. Today's data has no such case.
  • A one-segment range start (>= 3) now gives < 3 where the text sort gave another range's start. In 21 of the 80,000 fuzz inputs that output failed the lint rules where the old one passed, each time on a one-segment bound (< 3 or <= 3). Today's data has no such start.
  • The four-segment widening above is available as a follow-up if the 536 Rails-style releases matter more than the cgi shape.

Written with coexist

"~> <first patched>" stops at the next minor, so a later minor of an older
major is flagged although it has the fix. Use the start of the next vulnerable
range instead, and sort release lines as versions so 2.10 comes after 2.9.
An adversarial review of the first commit found that it ignored vulnerable
ranges without a fix, widened backported four-segment fixes that later
releases can predate (cgi 0.1.1 and 0.1.0.2), and raised on a version RubyGems
cannot parse. Keep the plain "~>" in each of those cases, and sort the
vulnerable ranges as versions too, so unaffected_versions starts at the
oldest range.
@Paul-Bob
Paul-Bob marked this pull request as draft November 20, 2026 12:53
@jasnow

jasnow commented Nov 20, 2026

Copy link
Copy Markdown
Member

Reviewed above

  1. Download branch locally and ran "rake" (green).
  2. Copied sync script into my master branch and ran my daily script. (okay)

Two more adversarial passes left these on constructed input. A range start
RubyGems cannot parse was printed into unaffected_versions, and two starts
that are equal but spelled differently sorted in node order. Such a start now
means nothing is listed as unaffected, and ">" sorts after ">=" and "=".
Every guard in the rule has an example that fails without it.
@Paul-Bob
Paul-Bob marked this pull request as ready for review November 21, 2026 11:06
@simi
simi merged commit 74723cd into rubysec:master Nov 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants