Repository navigation
bootstrap: push the trunk first and group small branches - #130
Conversation
Batched bootstrap fetched the commit graph of every branch, one request per branch, before it pushed anything. On a repository with thousands of branches that planning outlasted the 30-minute sync budget, so every attempt timed out with nothing on the target (ENT-3617). The trunk is now planned and pushed first. The remaining branches share one commit-graph fetch. Small branches are pushed in shared packs that create their refs directly. A group over budget splits in half, and a branch too big to share a pack keeps its own checkpointed batch. FetchCommitParents takes its wants as a list of hashes. The git-http-backend test server buffers chunked request bodies, which net/http/cgi rejects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FU3noPxGFpQ3TEX4SRyccF
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved pack-limit handling and retry-state issues can fail grouped fetches, replay confirmed ref creations, and stall checkpointed or resumed bootstrap.
4 open issues
What changed in this PR
Improves batched bootstrap for branch-heavy repositories by pushing the trunk before planning other branches and sharing fetches and packs across small branches.
Changes:
- Defers remaining branch planning until after the trunk is pushed, using one multi-tip commit-graph fetch.
- Groups small branches into shared packs, splitting oversized groups and retaining checkpointed fallbacks.
- Adds grouping tests and a many-branch integration test, with chunked-request buffering in the test server.
| File | Description |
|---|---|
internal/syncer/git_http_backend_test.go |
Buffers chunked requests and tests many-branch bootstrap. |
internal/strategy/bootstrap/github_preflight_test.go |
Updates the commit-graph fetch mock signature. |
internal/strategy/bootstrap/branch_group.go |
Adds shared branch planning, grouped pushes, and splitting. |
internal/strategy/bootstrap/branch_group_test.go |
Tests trunk ordering, grouping, splitting, and resume fallbacks. |
internal/strategy/bootstrap/bootstrap.go |
Integrates deferred planning and grouped execution. |
internal/strategy/bootstrap/bootstrap_test.go |
Adapts mocks and graph helpers to multiple wants. |
internal/gitproto/fetch.go |
Supports multiple commit-graph wants in one request. |
internal/gitproto/fetch_test.go |
Updates capability tests for the new fetch signature. |
CHANGELOG.md |
Documents the bootstrap improvements. |
Review effort: Balanced
| if i == len(batches) && deferred != nil { | ||
| more, err := planAfterTrunk(ctx, p, *deferred) | ||
| deferred = nil | ||
| if err != nil { | ||
| return result, err |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b716b71. Configure here.
A group push used a copy of the batch budget. When the target cut or rejected it, the group split but the run kept the old budget, the old calibration and no announced limit, so every later group and every level of splitting re-sent close to the cutoff. A rejected group push now updates the shared budget, its provenance, the calibration and the announced limit by the same rules as a checkpoint push. It splits into as many parts as the observed size needs, not always two. planAfterTrunk sizes groups with the current budget, which the trunk may already have lowered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FU3noPxGFpQ3TEX4SRyccF
- A branch too big to share a pack now runs on its own checkpoints before the groups still pending. When branches share a large history, it lands once and becomes a have, instead of being re-fetched by every smaller split. - A group whose union passes MaxPackBytes splits instead of failing the run. gitproto.ErrSourcePackTooLarge identifies that read error. A source-side limit does not change the target budget. - A failed group push records the refs it still created, and the retry leaves them out. The pusher splits ref updates across requests, so an earlier request can land before a later one fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FU3noPxGFpQ3TEX4SRyccF


Batched bootstrap now plans and pushes the trunk first, plans every other branch in one commit-graph fetch, and pushes small branches in shared packs (ENT-3617).
Problem
planBatchesfetched each branch's commit graph with its own request, one at a time, and pushed nothing until every branch was planned. On a repository with thousands of branches, planning outlasted a 30-minute sync budget, so every attempt timed out with nothing on the target and each retry started over.Change
tree:0commit-graph fetch (all tips as wants, trunk as have) instead of one request each.FetchCommitParentsnow takes its wants as a list of hashes.net/http/cgirejects.Measurements
Against github.com/openclaw/openclaw (3,904 branches), from a laptop:
Tests
go test ./internal/strategy/bootstrap/ ./internal/gitproto/ ./internal/syncer/passes.🤖 Generated with Claude Code
https://claude.ai/code/session_01FU3noNxsCP6TVRKqEmGsqS
Note
Medium Risk
Rewrites batched bootstrap planning and execution for multi-thousand-branch repos; behavior changes are substantial but scoped to bootstrap with extensive new tests and an integration path.
Overview
Batched bootstrap now makes progress on branch-heavy repositories (ENT-3617) by changing when planning and pushing happen, instead of fetching every branch’s commit graph up front before any data reaches the target.
The trunk is planned and pushed first; non-trunk branches are planned only after trunk batches run (
planAfterTrunk), so a timeout mid-sync still leaves the default branch on the target.FetchCommitParentsaccepts multiple want hashes in one protocol v2tree:0fetch, so remaining branches share a single commit-graph round trip.Small branches are batched into shared packs (
branch_group.go): one fetch and one push create many refs directly (no temp ref). Groups over the pack budget split in half; branches with resume markers or over the size budget keep the existing per-branch checkpoint path.Tests add unit coverage for trunk ordering, grouping, splits, and standalone fallbacks, plus a git-http-backend integration test that buffers chunked push bodies so CGI receive-pack works with streamed uploads.
Reviewed by Cursor Bugbot for commit b716b71. Configure here.