1861869 - Videos over 60 minutes or those with closed captions support can cause the volume slider in Firefox's media player to disappear after entering PiP mode

Status: VERIFIED FIXED

(Toolkit :: Video/Audio Controls, defect, P3)

Description

[aoia7rz7l]

Tested on 115.4.0esr but also reproducible on the latest Nightly (121.0a1).

STR:

1. Disable javascript and open any proxied video served via any Invidious instance.
2. Confirm that Firefox's media player comes with a volume slider.
3. Play the video.
4. Enter PiP mode.
5. Look for the volume slider in the media player (not the one in the PiP window).
6. Click on the Back to tab button in the PiP window.
7. Look for the volume slider in the media player again.

Expected Behavior:
The volume slider in Firefox's media player does not disappear upon entering or exiting PiP mode.

Actual Behavior:
The volume slider disappears after entering PiP mode and does not return even after exiting PiP mode. The volume slider reappears when the user entered full-screen mode using the media player's full screen button while the PiP window is closed. If the PiP window is still on, the media player's volume slider will only return after exiting the media player's full screen mode. Entering and exiting the PiP window's full screen mode has no effect on the media player's volume slider.

Issue is not reproducible using the example in https://www.w3schools.com/html/html5_video.asp, for example.

Mozregression returned

Last good revision: b8732a1c1fa931fc3cc5369e31a64db459f60c83 (2022-04-07)
First bad revision: 0671f5ff7249d3d7c458a29a4099a093c01f0ac1 (2022-04-08)
Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=b8732a1c1fa931fc3cc5369e31a64db459f60c83&tochange=0671f5ff7249d3d7c458a29a4099a093c01f0ac1
Switching bisection method to taskcluster
Getting mozilla-central builds between b8732a1c1fa931fc3cc5369e31a64db459f60c83 and 0671f5ff7249d3d7c458a29a4099a093c01f0ac1
...
There are no build artifacts for these changesets (these are probably too old).

I am guessing bug 1617283?

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Toolkit::Picture-in-Picture' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Molly Howell (she/her) (mostly inactive)]

Thanks for reporting this and for the regression range. I'm not able to reproduce on my Windows machine unfortunately; the volume slider on the player in the tab never disappears for me. So I'm not sure how to tell what's going on.

I will say I doubt that bug 1617283 is related, because it only touches the audio backend (i.e., the code responsible for communicating with the operating system). From your regression range, it seems plausible that bug 1733232 could be related, but I think that probably isn't the actual cause of the issue, for the simple reason that Invidious does not use our built-in video controls, they use Video.js. The W3Schools example you linked does use our built-in controls. So it would appear Video.js itself may be involved somehow.

I wonder if other similarly-configured Video.js players would have the same issue. I found a good example on Video.js's web site, would you mind trying that and seeing if you get different behavior?

Also, it would be helpful if you could copy and attach your support information here, or at least some basic system/configuration info if you don't want to provide all of that; it might give me some idea of why I'm not seeing this bug myself.

Thanks again.

Comment 3

[aoia7rz7l]

(In reply to Molly Howell (she/her) [:mhowell] from comment #2)

the simple reason that Invidious does not use our built-in video controls, they use Video.js.

If you disable javascript then Invidious will serve the video via media instead of xhr. You shouldn't be able to see the Video.js player when JS is disabled.

I found a good example on Video.js's web site

You should see a 0:00 "video" with Firefox's media player on this page when JS is disabled (not that you can force PiP on it though).

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:mhowell, could you have a look please?

For more information, please visit BugBot documentation.

Comment 5

[aoia7rz7l]

FWIW I don't think this has anything to do with JS disabled per se. I originally noticed this when playing a local mp4 file that I won't be able to share here, so I used Invidious as an example because I was also able to reproduce the issue there.

However, I have noticed since then that this does not necessarily apply to any video on Invidious, but only those that came with closed captions support (e.g. I can reproduce the issue on this first video but not on this second video).

The second thing is that I don't necessarily need to enter Firefox's media player's full screen mode to force the volume slider to reappear. Resizing/maximizing/un-maximizing the browser window will also cause the volume slider to reappear.

Comment 6

[aoia7rz7l]

Okay I might now have a better STR without having to depend on random Invidious instance's uptime.

STR:

1. Install uBlock Origin.
2. Disable javascript and then visit https://media.ccc.de/v/37c3-12004-please_identify_yourself.
3. Confirm that the video is using Firefox's media player and comes with a closed captions button.
4. Play the video.
5. Enter PiP mode.
6. Look for the volume slider in the media player (not the one in the PiP window).
7. Add either media.ccc.de##^track[kind="subtitles"][src] or media.ccc.de##track[kind="subtitles"][src]:remove() to uBO's My filters and refresh the video page.
8. Confirm that Firefox's media player no longer comes with a closed captions button.
9. Repeat steps 4-6.

Expected behavior:

Volume slider in Firefox's media player should not disappear under any circumstances when entering PiP mode.

Actual Behavior:

Volume slider in Firefox's media player consistently disappears when entering PiP mode, at least when closed captions were available.

So this might actually be regressed by bug 1733232 (instead of bug 1617283).

If you also wanted to test the example links posted in comment 5, add %DOMAIN_OF_INVIDIOUS_INSTANCE%##^track[kind="captions"][src] or %DOMAIN_OF_INVIDIOUS_INSTANCE%##track[kind="captions"][src]:remove() to uBO's My filters instead.

As for the local mp4 file that I also described in comment 5, its file headers are probably corrupt because the duration is shown as 0:00:00 on Windows' File Explorer, although both VLC and Firefox's media player correctly showed the actual duration when playing the video.

Comment 7

[aoia7rz7l]

I think I have finally narrowed down why that local mp4 file of mine is causing this issue. It's because the video is over a hour long.

When I used https://media.ccc.de/v/37c3-12142-breaking_drm_in_polish_trains to reproduce the STR in comment 6, the volume slider would still disappear even when closed captions were made unavailable by uBO. The only difference for this example and the others was that it was just over 60 minutes. I was also able to confirm this with other >60-minutes videos served via Invidious.

If for whatever reason you weren't able to disable JS in order to force Firefox to use its own media player on media.ccc.de, clicking on one of the download buttons below the video should force it to load the mp4/webm video file in a new page. This should prove that this issue has nothing to do with JS per se.

Comment 8

[aoia7rz7l]

One final caveat: Total video duration shown in the built-in media player will also disappear when entering PiP mode, but only when the video is simultaneously over an hour long and comes with closed captions support.

So comment 2 is correct, this couldn't have been caused by bug 1617283. Moving this bug to Video/Audio Controls and marking bug 1733232 as the regressor because this no longer seemed to be an audio-related issue as I initially thought.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

:emilio, since you are the author of the regressor, bug 1733232, could you take a look?

For more information, please visit BugBot documentation.

Comment 10

[Emilio Cobos Álvarez [:emilio]]

So, to be clear, what I'm doing and seeing is:

• Go to https://mirror.selfnet.de/CCC//congress/2023/h264-hd/37c3-12142-eng-Breaking_DRM_in_Polish_trains.mp4
• See the volume slider.
• Go into PiP
• See that the volume slider disappears.
• Go back (close pip window).
• See the volume slider is back.

Does that match what you're reporting? Sorry, just wanted to clear up since there's a lot of stuff up-thread.

Comment 11

[Emilio Cobos Álvarez [:emilio]]

Keeping the ni?, as I don't think comment 10 is intended.

Comment 12

[Emilio Cobos Álvarez [:emilio]]

I see what's going on.

Comment 13

[Emilio Cobos Álvarez [:emilio]]

Attachment: Bug 1861869 - Minor videocontrols.js clean-ups. r=#pip-reviewers,mconley

Comment 14

[Emilio Cobos Álvarez [:emilio]]

Attachment: Bug 1861869 - Remove some unused events in videocontrols.js. r=mconley,#pip-reviewers

Comment 15

[Emilio Cobos Álvarez [:emilio]]

Attachment: Bug 1861869 - Don't re-create the UA widget when entering PiP. r=mconley,#pip-reviewers

This is the actual fix. The issue is that, after the regressing bug, the
video element will only dispatch resized* events when:

• The frame is recreated.
• The video actually resizes.

And when going into PiP neither is happening, so we create a new impl,
but never update the "reflowed" dimensions.

Long term we should probably rewrite this to use ResizeObserver rather
than special code, but this should be a good perf improvement, and fix
the correctness issue, so seems worth doing.

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1733232

Comment 17

[aoia7rz7l]

(In reply to Emilio Cobos Álvarez (:emilio) from comment #10)

• See the volume slider is back.

Does that match what you're reporting? Sorry, just wanted to clear up since there's a lot of stuff up-thread.

On my end, the volume slider does reappear for a split second when I close the PiP window, but then very quickly disappears again. It will only come back when I click on the fullscreen button or attempt to resize the browser window. Otherwise your STR do match.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/46b0246ee0e8
Minor videocontrols.js clean-ups. r=pip-reviewers,reusable-components-reviewers,mstriemer,niklas
https://hg.mozilla.org/integration/autoland/rev/b0effc731d7c
Remove some unused events in videocontrols.js. r=pip-reviewers,niklas

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/459b7510116f
Fix test_videocontrols.html.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b4a12b771cd8
Don't re-create the UA widget when entering PiP. r=mconley,pip-reviewers,reusable-components-reviewers

Comment 21

[Sandor Molnar[:smolnar]]

Backed out for causing bc failures @ toolkit/components/pictureinpicture/tests/browser_reversePiP.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/16317fbb0f58115cb7b525f6fb49d8ab2bec00b0

Push with failures

Failure log -> TEST-UNEXPECTED-FAIL | toolkit/components/pictureinpicture/tests/browser_reversePiP.js | "true" == null

Comment 22

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/46b0246ee0e8
https://hg.mozilla.org/mozilla-central/rev/b0effc731d7c
https://hg.mozilla.org/mozilla-central/rev/459b7510116f

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/edf70c42e48d
Don't re-create the UA widget when entering PiP. r=mconley,pip-reviewers,reusable-components-reviewers

Comment 24

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/edf70c42e48d

Comment 25

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:emilio, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox124 to wontfix.

For more information, please visit BugBot documentation.

Comment 26

[Emilio Cobos Álvarez [:emilio]]

Rather old regression, can probably ride the trains.

Comment 27

[Ina Popescu, Desktop QA]

I've replicated this issue using on Nightly 125.0a1 (2024-03-04) Windows 10 x64 following the STR from Comment 10.
Verified as fixed in the latest Firefox 125.0b5 version on Windows 10 x64, macOS 13, and Ubuntu 22.04, as the issue no longer occurs.