Skip to content

macOS grabber: use a persistent SCStream instead of a screenshot per frame - #1637

Open
PiyalAhmed wants to merge 1 commit into
awawa-dev:masterfrom
PiyalAhmed:fix/macos-persistent-scstream
Open

PiyalAhmed wants to merge 1 commit into
awawa-dev:masterfrom
PiyalAhmed:fix/macos-persistent-scstream

Conversation

@PiyalAhmed

Copy link
Copy Markdown

Summary

The macOS system grabber (ScreenCaptureKit build, macOS 15+) took a new ScreenCaptureKit screenshot on every frame. On each timer tick macOsGrabber::grabFrame() did the following:

  1. SCShareableContent getShareableContentWithCompletionHandler: lists all displays and windows, which means TCC and WindowServer queries every frame.
  2. Allocated a new SCContentFilter and SCStreamConfiguration.
  3. SCScreenshotManager captureImageWithFilter: creates and tears down a one-shot capture session in replayd every frame, at full native resolution.

At 60 fps this put heavy load on system daemons (WindowServer, tccd, opendirectoryd, Control Centre, replayd), much more than HyperHDR used itself. replayd's Mach port count also kept growing while HyperHDR ran. The completion handlers also called decodeFrame(), uninit() and QTimer::singleShot(..., this, ...) from a ScreenCaptureKit queue instead of the grabber thread.

This PR switches to a persistent SCStream, following Hyperion.ng's fix for the same bug (hyperion-project/hyperion.ng#2047, issue hyperion-project/hyperion.ng#1921):

  • Stream lifetime: SCContentFilter, SCStreamConfiguration and SCStream are created once when the grabber starts.
  • Frame delivery: an SCStreamOutput delegate on its own serial queue keeps only the latest complete BGRA CVPixelBuffer in a mutex-protected slot. The existing QTimer processes it on the grabber thread and never blocks.
  • Output size: ScreenCaptureKit scales frames to Max stream width on the GPU. Crop values are still in native pixels and are rescaled to the delivered frame.
  • Frame rate: minimumFrameInterval follows the configured FPS.
  • Rebuilds:
    • Display, FPS and width changes rebuild the stream through the existing uninit()/start() path.
    • A display mode change also rebuilds it.
    • Stream errors (display removed, permission revoked) are handed to the grabber thread and use the existing uninit() + 3-second retry.
  • Cleanup: stopping the grabber stops the stream and releases every ObjC/CF object. Completion handlers never touch a destroyed grabber: it is detached under a lock, and a generation counter discards stale notifications.
  • Unchanged: public interface, settings, and the non-ScreenCaptureKit (CGDisplayCreateImage) path.

What kind of change does this PR introduce? (check at least one)

  • Bugfix
  • Feature
  • Code style update
  • Refactor
  • Docs
  • Build-related changes
  • Other, please describe:

If changing the UI of web configuration, please provide the before/after screenshot:

N/A (no UI changes)

Does this PR introduce a breaking change? (check one)

  • Yes
  • No

If yes, please describe the impact and migration path for existing setups:

The PR fulfills these requirements:

  • When resolving a specific issue, it's referenced in the PR's body (e.g. Fixes: #xxx[,#xxx], where "xxx" is the issue number)

To avoid wasting your time, it's best to open a feature request issue first and wait for approval before working on it.

Other information:

Before / After

Test setup: Mac mini M4, 16 GB, macOS 26.6.2, 5K display (5120×2880, looks like 2560×1440), system grabber enabled, 60 fps, Max stream width 1280. CPU values are top percentages of a single core.

Process Before (grabber on) After (grabber on) After (grabber off, baseline)
WindowServer 40–70 % 26–30 % 24–25 %
tccd 25–35 % 0 % 0 %
opendirectoryd 20–24 % ~0 % ~0 %
Control Centre 20–31 % 0–5 % 0–5 %
replayd 13–21 % ~2 % ~0.1 %
HyperHDR 15–22 % 19–21 % ~1 %

For "After", the grabber was switched off and on twice, with 20 s samples in each state. Most of WindowServer's load in this session came from other on-screen activity, since it was there with the grabber off as well. Capture itself adds about 2–5 %.

HyperHDR's own CPU stays about the same. A profile shows it is spent in the regular frame pipeline (FrameDecoder::processSystemImageBGRA, ImageColorAveraging::getMulticolorForLeds) at 60 fps × 1280 px, not in ScreenCaptureKit. On this 5K display the old code copied a 5120×2880 frame per tick before downscaling on the CPU; ScreenCaptureKit now delivers 1280×720 directly.

replayd Mach ports:

  • Before: 27.7k → 32.7k within minutes, still climbing while HyperHDR runs
  • After: ~1539 → 1547 over 30+ minutes of capture, flat for the last 7 minutes of a 10-minute recording. Five grabber off/on cycles stayed at ~1544 off / ~1550 on, so restarts don't leak.

Testing

  • Capture works and LEDs follow the real screen content, not just the wallpaper (60 fps, 1280 px)
  • Changing grabber settings in the UI restarts capture, which keeps working
  • Stopping and starting the grabber releases the stream (replayd ports drop on stop; no growth over 5 cycles)
  • replayd port count stays flat over time (30+ minutes)
  • Cropping still crops the same screen area
  • Changing the display resolution while running recovers
  • Disconnecting the captured display recovers (auto-picks another display, as before)

…frame

The ScreenCaptureKit path called SCShareableContent and
SCScreenshotManager captureImageWithFilter on every timer tick, so each
frame enumerated all displays/windows (TCC + WindowServer queries) and
set up and tore down a one-shot capture session in replayd. On a Mac
mini M4 at 60 fps this kept WindowServer, tccd, opendirectoryd, Control
Centre and replayd busy (together several times HyperHDR's own CPU) and
replayd's Mach port count kept growing while HyperHDR was running.

Now the SCContentFilter / SCStreamConfiguration / SCStream are created
once when the grabber starts. Frames arrive through an SCStreamOutput
delegate on its own serial queue, which keeps only the latest complete
BGRA CVPixelBuffer in a mutex-protected slot. The existing Qt timer
picks up that frame on the grabber thread and never blocks.

- output size is scaled by ScreenCaptureKit (GPU) to "Max stream width"
  instead of capturing at full native resolution; crop values (native
  pixels) are rescaled to the delivered frame
- minimumFrameInterval follows the configured FPS
- the stream is rebuilt on start/restart (display, FPS and width changes
  already go through uninit()/start()) and when the display mode changes
- stream errors (display removed, permission revoked) are marshalled to
  the grabber thread and use the existing uninit() + 3s retry
- the stream is stopped and every ObjC/CF object released when the
  grabber stops; completion handlers never touch a destroyed grabber
- the non-ScreenCaptureKit (CGDisplayCreateImage) path is unchanged

Approach based on Hyperion.ng's fix for the same issue:
hyperion-project/hyperion.ng#2047
(hyperion-project/hyperion.ng#1921)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant