rafraa/TX
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Task #461: Fix notification sounds on iPad (route via media channel)

Browse Source 
Problem: User reported zero sound on iPad — even the "Test sound"
button in notification settings produced silence. YouTube and other
videos worked fine on the same iPad.

Root cause: The app's notification player used the Web Audio API
(AudioContext + BufferSource). On iOS Safari (iPad/iPhone), Web Audio
plays through the **ringer channel**, which is silenced by:
  - The hardware silent switch (older iPads),
  - Control Center's mute icon,
  - Or simply having ringer volume at 0 (independent of media volume).
HTMLAudioElement (the API behind <video> and YouTube) plays through
the **media channel**, which is what users actually have turned up.

Fix in artifacts/tx-os/src/lib/notification-sounds.ts:
- Added an HTMLAudioElement playback engine alongside the existing
  Web Audio engine. Engine selection is one-time per session via the
  existing `detectIos()` helper (handles iPadOS-as-Mac masquerade).
- iOS path: lazy `new Audio(url)` per sound id with preload="auto",
  primed in `unlock()` by play()-then-pause() at volume 0 inside the
  user gesture so each element is permanently allowed to play later
  out-of-gesture (iOS requirement). Real `play()` resets currentTime
  and plays at full volume; rejection of the play promise dispatches
  the existing autoplay-blocked hint.
- Non-iOS path: unchanged Web Audio behavior preserved verbatim.
- All public API preserved: `unlock()`, `play()`, `testPlay()`,
  `isUnlocked()`, `AUTOPLAY_BLOCKED_EVENT`. No changes needed to
  use-audio-unlock, use-autoplay-hint, use-notifications-socket, or
  notification-settings.
- Throttling (3s burst-coalesce), bypass for testPlay, vibration,
  test-only observability globals, and isIos hint flag all preserved.
- `__txosNotifCtxState` test hook now reports "running"/"suspended"
  on iOS based on `unlocked` flag (no AudioContext exists there).

Locale updates (ar.json + en.json):
- `notifSettings.autoplayHintIos` rewritten: now tells users to raise
  the **media** volume (same one that controls YouTube) instead of
  the old "turn off Silent Mute" advice — matches the new playback
  channel.

Verification: tsc --noEmit clean. Pre-existing `test` workflow failure
on unrelated executive-meetings.ts type errors is out of scope.
Out of scope per task: no PWA, no push notifications, no settings UI
redesign, no new sound files. Manual iPad QA needed before merging.
This commit is contained in:
riyadhafraa
2026-05-10 11:40:07 +00:00
parent 72639c5faa
commit ec29657bf5
3 changed files with 242 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files
artifacts/tx-os/src/lib/notification-sounds.ts
+240 -96
View File
@@ -12,8 +12,10 @@ export const ALL_SOUND_IDS: ReadonlyArray<NotificationSoundId> =
* because the user hasn't yet interacted with the page (browser autoplay
* policy) or the AudioContext could not be resumed. The App listens for
* this and shows a one-time toast hint. The event `detail` carries an
* `isIos` flag so the toast can include iPad-specific guidance (check
* Control Center mute / silent switch).
* `isIos` flag so the toast can include iPad-specific guidance (raise
* the *media* volume, since the iOS path now plays through the media
* channel via HTMLAudioElement — see the iOS notes on `class
* NotificationPlayer` below).
*/
export const AUTOPLAY_BLOCKED_EVENT = "txos:notification-autoplay-blocked";
@@ -22,8 +24,11 @@ export type AutoplayBlockedDetail = {
};
// iPad/iPhone/iPod detection used to surface a more actionable hint
// (e.g. "check the silent switch / Control Center"). Modern iPadOS
// reports as Mac in `userAgent`, so we also check for Mac + multi-touch.
// (e.g. "raise the media volume") AND to switch the playback engine to
// HTMLAudioElement, which is the only way to route notification chimes
// through the iOS *media* channel instead of the *ringer* channel.
// Modern iPadOS reports as Mac in `userAgent`, so we also check for
// Mac + multi-touch.
function detectIos(): boolean {
if (typeof navigator === "undefined") return false;
const ua = navigator.userAgent || "";
@@ -43,34 +48,72 @@ type WindowWithWebkitAC = Window & {
webkitAudioContext?: typeof AudioContext;
};
/**
* NotificationPlayer
* --------------------------------------------------------------------
* Two playback engines, picked once per process based on `detectIos()`:
*
* • Non-iOS (desktop, Android Chrome, Firefox, etc.):
* Web Audio API (`AudioContext` + decoded `AudioBuffer`s). Lower
* latency, can pre-decode in the background, and respects the
* user's expectation of "the sound the page plays."
*
* • iOS (iPad / iPhone Safari + iPadOS-as-Mac masquerade):
* HTMLAudioElement (`new Audio(url)`). On iOS, Web Audio is routed
* through the *ringer* channel, which is silenced by the silent
* switch / Control Center mute and uses the *ringer* volume — even
* when the user has the *media* volume turned up (which is the
* volume that controls YouTube and other media). HTMLAudioElement
* plays through the media channel, identical to a `<video>` tag,
* so notifications stay audible whenever YouTube/Safari videos
* would also be audible.
*
* Both engines share the same throttling, autoplay-hint, vibration,
* and test-only observability so callers (`socket`, settings preview)
* see one uniform `play()` / `testPlay()` interface.
*/
class NotificationPlayer {
// Single shared AudioContext for the whole app. Lazily created in
// `unlock()` because constructing it before a user gesture leaves it
// in `suspended` state on every browser, and iOS Safari is even
// stricter — it must be created (or resumed) inside the gesture
// callback or it never starts.
// ---- Engine selection ----
// Frozen at first use: detectIos() reads navigator, which is stable
// for a session. Caching avoids repeated UA sniffing on every play.
private isIosEngine: boolean | null = null;
// ---- Web Audio (non-iOS) state ----
private ctx: AudioContext | null = null;
// Decoded sound buffers, keyed by sound id. Populated on demand and
// pre-warmed from `unlock()` so the first socket-driven chime after
// page load doesn't have to wait on a network round-trip.
private buffers = new Map<NotificationSoundId, AudioBuffer>();
// In-flight decode promises to dedupe concurrent loads of the same
// sound (e.g. unlock pre-warm + an immediate first play).
private loading = new Map<NotificationSoundId, Promise<AudioBuffer | null>>();
// ---- HTMLAudioElement (iOS) state ----
// One element per sound id, reused across plays. iOS requires that
// each element be `.play()`-ed at least once inside a user gesture
// before it can be played later out-of-gesture; we satisfy that in
// `unlock()` by playing-then-pausing every element silently.
private htmlAudios = new Map<NotificationSoundId, HTMLAudioElement>();
private htmlAudioUnlocked = new Set<NotificationSoundId>();
// ---- Shared state ----
private unlocked = false;
private lastPlayAt = 0;
private hintDispatched = false;
private get useIos(): boolean {
if (this.isIosEngine === null) this.isIosEngine = detectIos();
return this.isIosEngine;
}
isUnlocked(): boolean {
// "Unlocked" here means we've at least attempted resume() inside a
// user gesture AND the context is currently running. If the user
// backgrounded the tab the context can drift back to `interrupted`
// on iOS, so we re-check every call instead of trusting the flag
// alone.
if (!this.unlocked) return false;
if (this.useIos) {
// On iOS the "context" concept doesn't apply — being unlocked
// means at least one HTMLAudioElement has cleared its silent
// play+pause inside a gesture. We trust the flag.
return true;
}
return this.ctx?.state === "running";
}
// ---- Web Audio helpers (non-iOS) ----
/**
* Lazily build the shared AudioContext. Returns `null` when the
* Web Audio API is unavailable (very old browsers / SSR) so callers
@@ -90,53 +133,6 @@ class NotificationPlayer {
return this.ctx;
}
/**
* Browsers require a user gesture before audio can play. Call this
* from inside a gesture handler (pointerdown/touchstart/keydown). We
* resume the AudioContext, play a 1-sample silent buffer to fully
* satisfy iOS Safari's gesture requirement, and pre-warm the decoded
* buffers in the background so subsequent socket-driven chimes are
* instant. Safe to call repeatedly — on iOS the context can slip
* back into `suspended` if the tab is backgrounded, so we always
* try to resume() again on each subsequent gesture.
*/
unlock(): void {
const ctx = this.getContext();
if (!ctx) return;
// Always attempt resume() — even if `unlocked` is already true the
// context may have been auto-suspended (or, on iOS, transitioned
// to `interrupted` after a phone call / route change). Resuming
// any non-`running` state from inside the gesture is what gives
// iPad Safari its highest chance of bringing the ctx back.
if (ctx.state !== "running") {
void ctx.resume().catch(() => {});
}
// Play a 1-sample silent buffer through the destination. iOS Safari
// requires an actual `start()` call inside a gesture before it will
// permit later out-of-gesture playback through the same context.
try {
const silent = ctx.createBuffer(1, 1, 22050);
const src = ctx.createBufferSource();
src.buffer = silent;
src.connect(ctx.destination);
src.start(0);
} catch {
/* not fatal — resume() alone may still be enough */
}
if (this.unlocked) return;
this.unlocked = true;
// Pre-decode all sounds in the background. We deliberately do not
// await — playback path falls back to on-demand decode if a
// particular buffer hasn't loaded yet.
for (const entry of NOTIFICATION_SOUND_MANIFEST) {
void this.loadBuffer(entry.id);
}
}
private async loadBuffer(
soundId: NotificationSoundId,
): Promise<AudioBuffer | null> {
@@ -180,11 +176,132 @@ class NotificationPlayer {
return promise;
}
// ---- HTMLAudioElement helpers (iOS) ----
/**
* Lazily build (or fetch the cached) HTMLAudioElement for a sound.
* `preload="auto"` lets Safari start fetching the file early so the
* first play after unlock has its data ready.
*/
private getHtmlAudio(soundId: NotificationSoundId): HTMLAudioElement | null {
if (typeof window === "undefined") return null;
const cached = this.htmlAudios.get(soundId);
if (cached) return cached;
const entry = SOUND_BY_ID.get(soundId);
if (!entry) return null;
try {
const el = new Audio(entry.url);
el.preload = "auto";
// `playsInline` is a video-only attribute on the type, but
// setting it on an Audio element is harmless and silences a
// theoretical fullscreen-takeover path on very old iOS.
(el as HTMLAudioElement & { playsInline?: boolean }).playsInline = true;
el.crossOrigin = "anonymous";
this.htmlAudios.set(soundId, el);
return el;
} catch {
return null;
}
}
// ---- Public API ----
/**
* Browsers require a user gesture before audio can play. Call this
* from inside a gesture handler (pointerdown/touchstart/keydown).
*
* On non-iOS we resume the AudioContext and play a 1-sample silent
* buffer (iOS Safari's own trick, kept for any WebKit-derived
* non-iOS browser still requiring it), then pre-decode all buffers.
*
* On iOS we instead play-then-pause every HTMLAudioElement silently
* inside the gesture so each one is permanently allowed to play
* later (e.g. when a socket event arrives outside any gesture).
*
* Safe to call repeatedly; both engines tolerate it.
*/
unlock(): void {
if (this.useIos) {
// iOS path: walk every sound and prime its <audio> element. We
// mute it for the priming play so the user doesn't hear a
// burst of all 8 sounds at once. After the silent play resolves
// we pause + restore volume + reset currentTime so the next
// real play() starts from the beginning at full volume.
for (const entry of NOTIFICATION_SOUND_MANIFEST) {
if (this.htmlAudioUnlocked.has(entry.id)) continue;
const el = this.getHtmlAudio(entry.id);
if (!el) continue;
const originalVolume = el.volume;
el.volume = 0;
const p = el.play();
if (p && typeof p.then === "function") {
p.then(() => {
try {
el.pause();
el.currentTime = 0;
el.volume = originalVolume;
this.htmlAudioUnlocked.add(entry.id);
} catch {
/* ignore */
}
}).catch(() => {
// Element couldn't start — likely no gesture yet, or the
// user denied it. We'll try again on the next gesture.
try {
el.volume = originalVolume;
} catch {
/* ignore */
}
});
} else {
// Synchronous return (very old WebKit). Treat as success.
try {
el.pause();
el.currentTime = 0;
el.volume = originalVolume;
this.htmlAudioUnlocked.add(entry.id);
} catch {
/* ignore */
}
}
}
this.unlocked = true;
return;
}
// Non-iOS Web Audio path (original behavior).
const ctx = this.getContext();
if (!ctx) return;
// Always attempt resume() — even if `unlocked` is already true the
// context may have been auto-suspended.
if (ctx.state !== "running") {
void ctx.resume().catch(() => {});
}
try {
const silent = ctx.createBuffer(1, 1, 22050);
const src = ctx.createBufferSource();
src.buffer = silent;
src.connect(ctx.destination);
src.start(0);
} catch {
/* not fatal — resume() alone may still be enough */
}
if (this.unlocked) return;
this.unlocked = true;
for (const entry of NOTIFICATION_SOUND_MANIFEST) {
void this.loadBuffer(entry.id);
}
}
private dispatchHint() {
if (this.hintDispatched) return;
this.hintDispatched = true;
if (typeof window !== "undefined") {
const detail: AutoplayBlockedDetail = { isIos: detectIos() };
const detail: AutoplayBlockedDetail = { isIos: this.useIos };
window.dispatchEvent(
new CustomEvent<AutoplayBlockedDetail>(AUTOPLAY_BLOCKED_EVENT, {
detail,
@@ -209,17 +326,12 @@ class NotificationPlayer {
opts: { vibrate?: boolean; bypassThrottle?: boolean } = {},
): void {
// Throttle: at most one sound every 3s to coalesce bursts of
// notifications (e.g. several orders arriving in quick succession)
// into a single chime instead of a string of overlapping beeps.
// The settings "Test sound" affordance opts out so users can
// audition multiple sounds back-to-back.
// notifications into a single chime. Settings "Test sound"
// bypasses this so users can audition multiple sounds quickly.
const now =
typeof performance !== "undefined" ? performance.now() : Date.now();
if (!opts.bypassThrottle) {
if (now - this.lastPlayAt < 3000) return;
// Only the throttled (production socket-driven) path advances
// `lastPlayAt` so that rapid settings previews cannot suppress
// a legitimate socket chime that arrives moments later.
this.lastPlayAt = now;
}
@@ -238,23 +350,56 @@ class NotificationPlayer {
w.__txosNotifLastSound = entry.id;
}
if (this.useIos) {
this.playIos(entry.id);
} else {
this.playWebAudio(entry.id);
}
if (opts.vibrate && typeof navigator !== "undefined" && navigator.vibrate) {
try {
navigator.vibrate([60, 40, 60]);
} catch {
/* ignore */
}
}
}
private playIos(soundId: NotificationSoundId): void {
const el = this.getHtmlAudio(soundId);
if (!el) {
if (!this.unlocked) this.dispatchHint();
return;
}
try {
el.currentTime = 0;
} catch {
/* some browsers throw if not seekable yet — safe to ignore */
}
const p = el.play();
if (p && typeof p.then === "function") {
p.catch(() => {
// Most common rejections: NotAllowedError (autoplay policy
// before any gesture), or a transient AbortError from an
// immediate currentTime reset on the next play. The hint
// toast covers the autoplay case; the abort case is benign
// and we don't want to spam the user.
if (!this.unlocked) this.dispatchHint();
});
}
}
private playWebAudio(soundId: NotificationSoundId): void {
const ctx = this.getContext();
if (!ctx) {
if (!this.unlocked) this.dispatchHint();
return;
}
// The context may have drifted back to `suspended` (iOS background
// tab). Try to resume opportunistically — succeeds silently if a
// gesture has already been observed; otherwise the play attempt
// below will be inaudible and we surface the hint.
if (ctx.state !== "running") {
void ctx.resume().catch(() => {});
}
void this.loadBuffer(entry.id).then((buf) => {
void this.loadBuffer(soundId).then((buf) => {
if (!buf) return;
// Re-check context state right before scheduling — if it's still
// not running we won't actually be heard, so dispatch the hint.
if (ctx.state !== "running") {
this.dispatchHint();
return;
@@ -268,21 +413,13 @@ class NotificationPlayer {
/* ignore one-off node errors */
}
});
if (opts.vibrate && typeof navigator !== "undefined" && navigator.vibrate) {
try {
navigator.vibrate([60, 40, 60]);
} catch {
/* ignore */
}
}
}
/**
* Force-play used by the "Test sound" button in settings. Bypasses
* the 3-second throttle (so users can audition several sounds in a
* row) and unlocks the context first since this is always called
* from inside a click handler.
* row) and unlocks the audio first since this is always called from
* inside a click handler.
*/
testPlay(soundId: NotificationSoundId): void {
this.unlock();
@@ -293,14 +430,21 @@ class NotificationPlayer {
export const notificationPlayer = new NotificationPlayer();
// Test-only accessor so Playwright can assert the singleton player's
// AudioContext state after a user gesture (the actual production
// path on iPad Safari, not a freshly constructed test ctx). Gated on
// dev/test bundles to avoid leaking internals into production.
// AudioContext state after a user gesture (the actual production path
// on non-iOS, not a freshly constructed test ctx). Gated on dev/test
// bundles to avoid leaking internals into production. Returns the
// engine name on iOS (where there is no AudioContext) so tests can
// still distinguish "unlocked" from "not yet unlocked".
if (typeof window !== "undefined" && import.meta.env.MODE !== "production") {
(window as unknown as {
__txosNotifCtxState?: () => string | null;
}).__txosNotifCtxState = () => {
const p = notificationPlayer as unknown as { ctx: AudioContext | null };
const p = notificationPlayer as unknown as {
ctx: AudioContext | null;
isIosEngine: boolean | null;
unlocked: boolean;
};
if (p.isIosEngine) return p.unlocked ? "running" : "suspended";
return p.ctx?.state ?? null;
};
}
artifacts/tx-os/src/locales/ar.json
+1 -1
View File
@@ -322,7 +322,7 @@
"unmutedToast": "تم إلغاء كتم الإشعارات",
"autoplayHintTitle": "تحتاج الأصوات إلى نقرة",
"autoplayHint": "انقر في أي مكان لتفعيل أصوات الإشعارات.",
"autoplayHintIos": "اضغط في أي مكان لتفعيل الأصوات. إذا ما زلت لا تسمعها، افتح مركز التحكم وعطّل الوضع الصامت.",
"autoplayHintIos": "اضغط في أي مكان لتفعيل الأصوات. إذا ما زلت لا تسمعها، تأكد إن مستوى صوت الوسائط مرفوع (نفس الصوت اللي يشغّل اليوتيوب).",
"slot": {
"order": "الطلبات",
"meeting": "الاجتماعات",
artifacts/tx-os/src/locales/en.json
+1 -1
View File
@@ -325,7 +325,7 @@
"unmutedToast": "Notifications unmuted",
"autoplayHintTitle": "Sounds need a click",
"autoplayHint": "Click anywhere on the page to enable notification sounds.",
"autoplayHintIos": "Tap anywhere to enable sounds. If you still don't hear them, open Control Center and turn off Silent / Mute.",
"autoplayHintIos": "Tap anywhere to enable sounds. If you still don't hear them, raise the *media* volume (the same one that controls YouTube).",
"slot": {
"order": "Orders",
"meeting": "Meetings",