Repository navigation
Commit 21388e4
authored
fix(minidb): buffer large RESP requests in linear time (#393)
## Requirement or Bug
`test-minidb` times out on slower CI runners in `RESP: an oversized
request gets -ERR and the connection recovers` (20 s, 0 bytes back). It
failed twice on #392, which does not touch minidb.
## Bug Reproduction Steps
1. Start the minidb RESP server.
2. Send one `SET` whose bulk value is 65 MB (over the 64 MB request
cap), then `PING`.
3. The `-ERR ... too large` reply takes about 2.4 s on an M5 Max and
more than 20 s on a loaded CI runner.
## Root Cause
`RespParser.feed` joined each incoming socket chunk onto the whole
buffered request (`Buffer.concat([this.buf, chunk])`). A 65 MB request
arrives in about a thousand chunks, so the parser copied tens of
gigabytes before it reached the cap. This is a fix at the root, not a
timeout increase.
## Code Changes
```diff
RespParser.feed(chunk)
- buf = concat(buf, chunk) every chunk
+ pending.push(chunk); size += chunk.length
+ size < need -> return bulk string still incomplete
+ buf = concat(buf, ...pending) once
RespParser.tryParse()
+ bulk string incomplete -> need = bytes required
```
The oversized-request check and its buffer reset do not change; they now
count `size`, which includes the pending chunks.
## Behavior Changes and Affected Users
| Behavior | Before | After | Who relies on the old behavior | Escape
hatch |
|---|---|---|---|---|
| RESP request larger than one chunk | Buffer copied on every chunk |
Chunks joined once the bulk string can complete | Nobody | n/a |
Tests: oversized-request test 2381 ms before, 48 ms after (local). New
test `RESP: a bulk value split across many small writes is reassembled`
covers the split path. minidb suite 554 passed; `tsc` clean; `pnpm run
lint` 0 errors. [skip changeset] Users cannot see this change: it only
changes how fast the RESP server buffers large requests.
## Checklist
- [x] I have read the
[CONTRIBUTING](https://github.com/PyModel/pythinker-code/blob/main/CONTRIBUTING.md)
document.
- [ ] I have linked a related issue (external PRs: issue must have a
maintainer's `/approve`).
- [x] I have added tests that prove my feature works.
- [x] The behavior-change table above is complete, and every removed
behavior or flipped default is named in the changeset and either has an
escape hatch or was explicitly approved by a maintainer in this PR.
- [x] Ran `gen-changesets` skill, or this PR needs no changeset.
- [x] Ran `gen-docs` skill, or this PR needs no doc update.1 parent 2c087e3 commit 21388e4
2 files changed
Lines changed: 42 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
36 | 42 | | |
37 | 43 | | |
38 | 44 | | |
39 | 45 | | |
40 | 46 | | |
41 | 47 | | |
42 | 48 | | |
43 | | - | |
44 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
45 | 52 | | |
46 | 53 | | |
47 | 54 | | |
48 | 55 | | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
49 | 59 | | |
50 | 60 | | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
51 | 65 | | |
52 | 66 | | |
| 67 | + | |
53 | 68 | | |
54 | 69 | | |
55 | 70 | | |
| |||
78 | 93 | | |
79 | 94 | | |
80 | 95 | | |
81 | | - | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
82 | 100 | | |
83 | 101 | | |
84 | 102 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
203 | 203 | | |
204 | 204 | | |
205 | 205 | | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
206 | 227 | | |
207 | 228 | | |
208 | 229 | | |
| |||
0 commit comments