Skip to content

fix(call): resume the mixed call audio after it was suspended - #20113

Open
AbdurRazzaq2004 wants to merge 1 commit into
nextcloud:mainfrom
AbdurRazzaq2004:fix/call-resume-mixed-audio
Open

AbdurRazzaq2004 wants to merge 1 commit into
nextcloud:mainfrom
AbdurRazzaq2004:fix/call-resume-mixed-audio

Conversation

@AbdurRazzaq2004

@AbdurRazzaq2004 AbdurRazzaq2004 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

☑️ Resolves

  • No existing issue: the problem and how it was reproduced are described below.

Problem

In Safari and on iOS, the audio of all the participants is mixed with an
AudioContext and played by a single audio element. The browser can
suspend that context, for example on iOS when the audio is interrupted
by a phone call or Siri, or when the screen is locked. Nothing resumed
it afterwards, so no participant could be heard again until the call was
left and joined again.

Solution

  • Resume the audio context, and play the element again if needed, when
    the state of the context changes, when the page becomes visible again
    and on the next user interaction (in case the browser requires a user
    gesture for it).
  • The listeners are removed when the player is destroyed.
  • A user interaction resumes it also with touch input: "pointerup" and
    "touchend" are listened to as well, as "pointerdown" is a user gesture
    only with a mouse. The element is also played again right after its
    source is replaced, which is done in Safari when a participant is
    muted.

Testing

  • Lint (eslint on the changed files) passes, and the unit tests pass on
    main with all five related fixes applied (vitest src/utils: 29 files,
    616 tests).
  • The change has been running in production since 2026-11-02 on a
    Nextcloud 34.0.5 server with Talk 24.0.6 and the external signaling
    server (2.1.1, Janus 1.4.1), where all media goes through TURN.
  • End-to-end checks were run with headless Chromium against that live
    server, each time with the unmodified Talk 24.0.6 JavaScript
    ("original", served to the browser through request interception) and
    with this change ("patched"), on the same server and under the same
    conditions.
  • The mixed audio path was tested with Chromium identifying itself as
    Safari (Talk then mixes the audio as in Safari). The mixing
    AudioContext was suspended during a call, original against patched:
    • Original: it stayed "suspended", also after a user interaction.
    • Patched: it was "running" again within 2 seconds, without any user
      interaction.
  • Unit tests for the mixed audio were added: the audio is resumed on
    "pointerup", "touchend", "mousedown" and "keydown", not after the
    player was destroyed, and the element is played again after its source
    is replaced.
  • A listen-only guest (no microphone, so the browser does not allow
    autoplay) joined the call without any user activation, with Chromium
    identifying itself as Safari. With the first version of this pull
    request (resuming only on "pointerdown") the mixing AudioContext
    stayed "suspended" after a full tap; with this version it was
    "running" and the element playing right after the tap. A touch-start
    alone did not unlock it in either version, as the HTML specification
    says.
  • This could not be checked on a real iOS device.

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

🖌️ UI Checklist

🏁 Checklist

  • 🌏 Tested with different browsers / clients:
    • Chromium (Chrome / Edge / Opera / Brave)
      (Chromium identifying itself as Safari, so Talk uses the Safari audio path; not checked on a real iOS device)
    • Firefox
    • Safari
    • Talk Desktop
    • Integrations with Files sidebar and other apps
    • Not risky to browser differences / client
  • 🖌️ Design was reviewed, approved or inspired by the design team
  • ⛑️ Tests are included or not possible: unit tests for the mixed audio are included, and the existing unit tests pass
  • 📗 User documentation in https://github.com/nextcloud/documentation/tree/master/user_manual/talk has been updated or is not required

Problem
-------
In Safari and on iOS, the audio of all the participants is mixed with an
AudioContext and played by a single audio element. The browser can
suspend that context, for example on iOS when the audio is interrupted
by a phone call or Siri, or when the screen is locked. Nothing resumed
it afterwards, so no participant could be heard again until the call was
left and joined again.

Solution
--------
- Resume the audio context, and play the element again if needed, when
  the state of the context changes, when the page becomes visible again
  and on the next user interaction (in case the browser requires a user
  gesture for it).
- The listeners are removed when the player is destroyed.
- A user interaction resumes it also with touch input: "pointerup" and
  "touchend" are listened to as well, as "pointerdown" is a user gesture
  only with a mouse. The element is also played again right after its
  source is replaced, which is done in Safari when a participant is
  muted.

Testing
-------
- Lint (eslint on the changed files) passes, and the unit tests pass on
  main with all five related fixes applied (vitest src/utils: 29 files,
  616 tests).
- The change has been running in production since 2026-11-02 on a
  Nextcloud 34.0.5 server with Talk 24.0.6 and the external signaling
  server (2.1.1, Janus 1.4.1), where all media goes through TURN.
- End-to-end checks were run with headless Chromium against that live
  server, each time with the unmodified Talk 24.0.6 JavaScript
  ("original", served to the browser through request interception) and
  with this change ("patched"), on the same server and under the same
  conditions.
- The mixed audio path was tested with Chromium identifying itself as
  Safari (Talk then mixes the audio as in Safari). The mixing
  AudioContext was suspended during a call, original against patched:
  - Original: it stayed "suspended", also after a user interaction.
  - Patched: it was "running" again within 2 seconds, without any user
    interaction.
- Unit tests for the mixed audio were added: the audio is resumed on
  "pointerup", "touchend", "mousedown" and "keydown", not after the
  player was destroyed, and the element is played again after its source
  is replaced.
- A listen-only guest (no microphone, so the browser does not allow
  autoplay) joined the call without any user activation, with Chromium
  identifying itself as Safari. With the first version of this pull
  request (resuming only on "pointerdown") the mixing AudioContext
  stayed "suspended" after a full tap; with this version it was
  "running" and the element playing right after the tap. A touch-start
  alone did not unlock it in either version, as the HTML specification
  says.
- This could not be checked on a real iOS device.

Signed-off-by: Abdur Razzaq <153341210+AbdurRazzaq2004@users.noreply.github.com>
@AbdurRazzaq2004
AbdurRazzaq2004 force-pushed the fix/call-resume-mixed-audio branch from f024c94 to cc6083f Compare November 2, 2026 12:38
@AbdurRazzaq2004

Copy link
Copy Markdown
Contributor Author

Updated: "pointerdown" is not a user gesture for touch input, so on iOS a tap could not resume the audio. It is now also resumed on "pointerup", "touchend" and "mousedown", and the element is played again after Safari's source swap on mute. Unit tests for the mixed audio were added (eslint and vitest src/utils pass on the base commit), and the end-to-end check with touch input is described in the commit message.

@nickvergessen nickvergessen added bug feature: WebRTC 🚡 WebRTC connection between browsers and/or mobile clients feature: frontend 🖌️ "Web UI" client feature: call 📹 Voice and video calls labels Nov 3, 2026
@nickvergessen nickvergessen added this to the ☃️ Next Major (36) milestone Nov 3, 2026

@Antreesy Antreesy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wish for all open PRs:

  • please disclose agent model with Assisted-by: MODEL:VERSION git trailer
  • keep commit body of a reasonable size 🙈 core change, git trailer and DCO are enough, the rest is OK to be documented in the PR description or the code itself
  • same for large code comments, if it's self-explanatory

Otherwise nice! Thank you, waiting for you to rebase and looking forward to merge!

The change has been running in production since 2026-11-02

Let us know, if you noticed it's better now?

// soon as possible; if the browser requires a user gesture for that it
// will be done on the next interaction with the page.
this._resumeAudioBound = this._resumeAudio.bind(this)
this._audioContext.addEventListener?.('statechange', this._resumeAudioBound)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional chaining should not be needed here

Suggested change
this._audioContext.addEventListener?.('statechange', this._resumeAudioBound)
this._audioContext.addEventListener('statechange', this._resumeAudioBound)

Comment on lines +104 to +106
this._audioContext.resume()?.catch(() => {
// It will be tried again on the next state change or interaction.
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same should be here. Promise should reject to go into catch; if it's the real case to throw an error here because of function missing - then it's an accepted risk to learn it the hard way later from another report

Also more compact writing would be appreciated =)

Suggested change
this._audioContext.resume()?.catch(() => {
// It will be tried again on the next state change or interaction.
})
// Rejected Promise will be tried again on the next state change or interaction.
this._audioContext.resume()?.catch(() => {})

vi.restoreAllMocks()
})

test.each(['pointerup', 'touchend', 'mousedown', 'keydown'])('resumes the audio on %s', (eventName) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can include:

Suggested change
test.each(['pointerup', 'touchend', 'mousedown', 'keydown'])('resumes the audio on %s', (eventName) => {
test.each(['pointerup', 'touchend', 'mousedown', 'keydown', 'visibilitychange'])('resumes the audio on %s', (eventName) => {

And from what I see, statechange is not covered with test, could you add it as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

browser: Safari bug feature: call 📹 Voice and video calls feature: frontend 🖌️ "Web UI" client feature: WebRTC 🚡 WebRTC connection between browsers and/or mobile clients

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants