1557994 - Picture-in-Picture flickering on animelab.com

Status: VERIFIED FIXED

(Toolkit :: Video/Audio Controls, defect, P2)

Description

[Winson Wen]

Attachment: flicker.mp4

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0

Steps to reproduce:

Sign in to animelab.com
Start playing a show
Switch to picture-in-picture

Animelab might be geoblocked to Australia

Actual results:

Video flickers between currently playing point and the beginning of the episode

Expected results:

Video plays without flicker

Comment 1

[Jared Wein [:jaws] (please needinfo? me)]

ni mconley for PiP triage

Comment 2

[Mike Conley (:mconley) (:⚙️)]

Hi jya,

I suspect something's going wrong in the visual cloning we're doing here... does the video capture from the reporter give you any insight into what we might be doing wrong here?

Comment 3

[Jean-Yves Avenard [:jya]]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #2)

Hi jya,

I suspect something's going wrong in the visual cloning we're doing here... does the video capture from the reporter give you any insight into what we might be doing wrong here?

That's frames being displayed out of order and the destination frame being re-used by the decoder before it got painted.

Sounds like a refcount being cleared too early.

When in the case of PiP, how do we make sure frames aren't being recycled before they are rendered?

We had a similar case occurring on mac in the past because we didn't increase the refcount on the underlying IOSurface.

Matt, how is this handled on Windows?

Comment 4

[Matt Woodrow (:mattwoodrow)]

This sounds like it could be a bug in the recycling code we use for D3D11 video textures. I'm not sure if we correctly handle the case where there are two consumers that might need to hold references.

Any ideas Sotaro?

Comment 5

[Sotaro Ikeda [:sotaro]]

Winson Wen, can you upload about:support after the problem happened?
Open about:support, click on "Copy text to clipboard", paste it into a text file and attach it to the report. Thanks!

Comment 6

[Sotaro Ikeda [:sotaro]]

I live in Japan, then it seems that I could not watch the video.
Winson Wen, is there another video service that cause the same problem?

Comment 7

[Sotaro Ikeda [:sotaro]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #4)

This sounds like it could be a bug in the recycling code we use for D3D11 video textures. I'm not sure if we correctly handle the case where there are two consumers that might need to hold references.

Any ideas Sotaro?

Recycling should be handled correctly even in this use case. If recycling happened too early, a bit future frame is shown like flickering. But in this case, the old same frame was shown...

Comment 8

[Winson Wen]

Attachment: about:support

I've attached the about:support text.
Haven't encountered this problem anywhere else so I don't know what other websites you could try, sorry.
The weird thing about the bug is though, the flickering always shows frames from the beginning of the video, even if I start playing the video from the middle. And every time I go into picture-in-picture, the flickering frames reset to the beginning.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:drno, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 10

[Mike Conley (:mconley) (:⚙️)]

Hi jya,

We were talking about this at Whistler, and I recall you mentioning that you were interested in whether or not the user experienced the flickering after the PiP player window was closed. Looking at the posted recording at https://bugzilla.mozilla.org/attachment.cgi?id=9070817, it does appear as if the flicker goes away once the player window is closed.

Does that help box the problem in a bit?

Comment 11

[deistyny]

I confirm the same issue on seasovar.ru
Appears that picture-in-picture mode begins to autoplay video without sound and without any synchronisation with the main video frame. This causes flickering - two video streams being played at the same time of the same video but on different time position.

Comment 12

[Jean-Yves Avenard [:jya]]

(In reply to Mike Conley (:mconley) (:⚙️) from comment #10)

Hi jya,

We were talking about this at Whistler, and I recall you mentioning that you were interested in whether or not the user experienced the flickering after the PiP player window was closed. Looking at the posted recording at https://bugzilla.mozilla.org/attachment.cgi?id=9070817, it does appear as if the flicker goes away once the player window is closed.

Does that help box the problem in a bit?

Nope.

In the other bug, the issue was resolved by enabling webrender.
But here webrender is enabled.

I have no explanation on why this is happening here.
The only explanation is that one of the recycled surface is incorrectly used, or used at the wrong time.

Comment 13

[Winson Wen]

Attachment: weirdness.mp4

So I've experimented a bit more and found that with hardware acceleration turned off in about:preferences, the flickering reduces dramatically, only flickering when first entering picture-in-picture, and around the control buttons when hovering in and out or the window.

Also, pausing the video while in picture-in-picture only pauses the video playing the current time, the parts playing from earlier continue and also stop flickering. Attached a video showing what I mean.

Comment 14

[Winson Wen]

So the reduced flickering with hardware acceleration disabled feels very temperamental, having obs open seems to break it. Pausing and playing while in pip with h/w acceleration off also seems to randomly switch between 3 states, the flicker-y mess from the bug description, playing the current time of the video with little flicker, or playing the earlier part with audio from the current part.

Comment 15

[Winson Wen]

So testing a bit more, even with hardware acceleration it appears to switch randomly between 3 states when pausing and playing in pip, flickering evenly between the early and current part of the video, playing mostly the current part with flickers from the early part, and playing mostly the early part with flickers of the current part.

Also, sorry for all the comment spam, I'm kinda writing things as I find them. I will try batching stuff up more before commenting next If I find anything else.

Comment 16

[Nils Ohlmeier [:drno]]

Matt does any of the new information provided by Winson in comments #13-15 provide any more insight into what could be going on here?

Comment 17

[Nils Ohlmeier [:drno]]

Forgot to set NI: see above ^

Comment 18

[igor.zuk]

I can confirm identical problem with all tvn24.pl videos.
e.g. https://tvnmeteo.tvn24.pl/informacje-pogoda/prognoza,45/pogoda-na-16-dni-niedlugo-ochlodzenie-przerwie-upal,296088,1,0.html
Tested on Windows 10 with Geforce 1070 and Ubuntu 19.04 and Intel HD 520.

Comment 19

[Sotaro Ikeda [:sotaro]]

(In reply to igor.zuk from comment #18)

I can confirm identical problem with all tvn24.pl videos.
e.g. https://tvnmeteo.tvn24.pl/informacje-pogoda/prognoza,45/pogoda-na-16-dni-niedlugo-ochlodzenie-przerwie-upal,296088,1,0.html
Tested on Windows 10 with Geforce 1070 and Ubuntu 19.04 and Intel HD 520.

Thanks for the report! I also confirmed the problem.

Comment 20

[Sotaro Ikeda [:sotaro]]

(In reply to igor.zuk from comment #18)

I can confirm identical problem with all tvn24.pl videos.
e.g. https://tvnmeteo.tvn24.pl/informacje-pogoda/prognoza,45/pogoda-na-16-dni-niedlugo-ochlodzenie-przerwie-upal,296088,1,0.html
Tested on Windows 10 with Geforce 1070 and Ubuntu 19.04 and Intel HD 520.

When the problem happened, VisualCloneTarget HTMLVideoElement also created MediaDecoder and playbacked a video. Then, VideoSink of VisualCloneTarget also rendered video. Then VideoFrameContainer of SecondaryContainer received video frames from two VideoSinks.

When I disabled to create MediaDecoder at VisualCloneTarget HTMLVideoElement , the problem did not happen.

:mconley, can you comment to it?

Comment 21

[Mike Conley (:mconley) (:⚙️)]

Attachment: Bug 1557994 - Picture-in-Picture player video element should not try decoding its own version of the originating video. r?JSON_Voorhees

Comment 22

[Mike Conley (:mconley) (:⚙️)]

Thanks, sotaro! That helps a lot.

The original <video> HTML node is cloned, and inserted into the player video, and then becomes the "visual clone target". The reason that we clone the original node is to get the same MediaInfo of the <video>, which includes things like its dimensions.

We can halt the decoder on the cloned video by pausing it after it's cloned.

Comment 23

[Mike Conley (:mconley) (:⚙️)]

Here are some try builds with the patch to try:

Windows 32-bit: https://queue.taskcluster.net/v1/task/b9nf-9qxROG9qWd19p-9tw/runs/0/artifacts/public/build/target.zip
Windows 64-bit: https://queue.taskcluster.net/v1/task/cp5N4XrQTA6mq07xvmrVOg/runs/0/artifacts/public/build/target.zip

Hey Winson, if you have a moment, can you confirm that the patch fixes the issue with one of these builds?

Comment 24

[Winson Wen]

Everything's working fine, looks like the problem's fixed!

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dbb8e4a9609d
Picture-in-Picture player video element should not try decoding its own version of the originating video. r=JSON_voorhees

Comment 26

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/dbb8e4a9609d

Comment 27

[Cristian Craciun]

Build ID 20190902191027
User Agent Mozilla/5.0 (Windows NT 10.0; rv:70.0) Gecko/20100101 Firefox/70.0

Verified as fixed on the latest Beta version (v70b3) and on the latest Nightly version.