1467013 - Intermittent PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::MediaSourceTrackDemuxer::DoGetSamples(int)]

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nerli [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=182000953&repo=autoland

https://queue.taskcluster.net/v1/task/EDmRcZHiSpKBFutckkW_gA/runs/0/artifacts/public/logs/live_backing.log

[task 2018-06-06T01:57:29.659Z] 01:57:29     INFO - TEST-START | dom/media/mediasource/test/test_ChangeWhileWaitingOnMissingData_mp4.html
[task 2018-06-06T01:57:30.109Z] 01:57:30     INFO - GECKO(1028) | [Parent 1028, Gecko_IOThread] WARNING: pipe error (73): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 353
[task 2018-06-06T01:57:30.109Z] 01:57:30     INFO - GECKO(1028) | ###!!! [Parent][MessageChannel] Error: (msgtype=0x160080,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv
[task 2018-06-06T01:57:30.198Z] 01:57:30     INFO - GECKO(1028) | JavaScript error: resource:///modules/PageStyleHandler.jsm, line 55: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDocShell.tabChild]
[task 2018-06-06T01:57:30.206Z] 01:57:30     INFO - GECKO(1028) | A content process crashed and MOZ_CRASHREPORTER_SHUTDOWN is set, shutting down
[task 2018-06-06T01:57:30.370Z] 01:57:30     INFO - GECKO(1028) | ###!!! [Parent][MessageChannel] Error: (msgtype=0x2D0020,name=PContent::Msg_PreferenceUpdate) Closed channel: cannot send/recv
[task 2018-06-06T01:57:30.489Z] 01:57:30     INFO - GECKO(1028) | 1528250250478	Marionette	DEBUG	Received observer notification xpcom-will-shutdown
[task 2018-06-06T01:57:30.489Z] 01:57:30     INFO - GECKO(1028) | 1528250250479	Marionette	INFO	Stopped listening on port 2828
[task 2018-06-06T01:57:30.489Z] 01:57:30     INFO - GECKO(1028) | 1528250250479	Marionette	DEBUG	Remote service is inactive
[task 2018-06-06T01:57:30.752Z] 01:57:30     INFO - TEST-INFO | Main app process: exit 0
[task 2018-06-06T01:57:30.752Z] 01:57:30     INFO - runtests.py | Application ran for: 0:00:38.420142
[task 2018-06-06T01:57:30.753Z] 01:57:30     INFO - zombiecheck | Reading PID log: /tmp/tmpdCLV8ypidlog
[task 2018-06-06T01:57:30.753Z] 01:57:30     INFO - ==> process 1028 launched child process 1051
[task 2018-06-06T01:57:30.753Z] 01:57:30     INFO - ==> process 1028 launched child process 1079
[task 2018-06-06T01:57:30.754Z] 01:57:30     INFO - ==> process 1028 launched child process 1145
[task 2018-06-06T01:57:30.754Z] 01:57:30     INFO - ==> process 1028 launched child process 1241
[task 2018-06-06T01:57:30.755Z] 01:57:30     INFO - zombiecheck | Checking for orphan process with PID: 1145
[task 2018-06-06T01:57:30.756Z] 01:57:30     INFO - zombiecheck | Checking for orphan process with PID: 1241
[task 2018-06-06T01:57:30.756Z] 01:57:30     INFO - zombiecheck | Checking for orphan process with PID: 1051
[task 2018-06-06T01:57:30.759Z] 01:57:30     INFO - zombiecheck | Checking for orphan process with PID: 1079
[task 2018-06-06T01:57:30.759Z] 01:57:30     INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/d9OoVBXYRtGZo4gzIvaZFQ/artifacts/public/build/target.crashreporter-symbols.zip
[task 2018-06-06T01:57:36.979Z] 01:57:36     INFO - mozcrash Copy/paste: /usr/local/bin/linux64-minidump_stackwalk /tmp/tmpSkz7Do.mozrunner/minidumps/1d344b06-08a4-d4af-71d6-c34d1462564b.dmp /tmp/tmpWTv6xt
[task 2018-06-06T01:57:44.839Z] 01:57:44     INFO - mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/1d344b06-08a4-d4af-71d6-c34d1462564b.dmp
[task 2018-06-06T01:57:44.840Z] 01:57:44     INFO - mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/1d344b06-08a4-d4af-71d6-c34d1462564b.extra
[task 2018-06-06T01:57:44.961Z] 01:57:44     INFO - PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::MediaSourceTrackDemuxer::DoGetSamples(int)]
[task 2018-06-06T01:57:44.962Z] 01:57:44     INFO - Crash dump filename: /tmp/tmpSkz7Do.mozrunner/minidumps/1d344b06-08a4-d4af-71d6-c34d1462564b.dmp
[task 2018-06-06T01:57:44.964Z] 01:57:44     INFO - Operating system: Linux
[task 2018-06-06T01:57:44.965Z] 01:57:44     INFO -                   0.0.0 Linux 4.4.0-1014-aws #14taskcluster1-Ubuntu SMP Tue Apr 3 10:27:00 UTC 2018 x86_64
[task 2018-06-06T01:57:44.966Z] 01:57:44     INFO - CPU: amd64
[task 2018-06-06T01:57:44.967Z] 01:57:44     INFO -      family 6 model 62 stepping 4
[task 2018-06-06T01:57:44.967Z] 01:57:44     INFO -      2 CPUs
[task 2018-06-06T01:57:44.968Z] 01:57:44     INFO - 
[task 2018-06-06T01:57:44.968Z] 01:57:44     INFO - GPU: UNKNOWN
[task 2018-06-06T01:57:44.969Z] 01:57:44     INFO - 
[task 2018-06-06T01:57:44.969Z] 01:57:44     INFO - Crash reason:  SIGSEGV
[task 2018-06-06T01:57:44.970Z] 01:57:44     INFO - Crash address: 0x0
[task 2018-06-06T01:57:44.971Z] 01:57:44     INFO - Process uptime: not available
[task 2018-06-06T01:57:44.971Z] 01:57:44     INFO -

Comment 1

[Jean-Yves Avenard [:jya]]

the backtrace isn't very useful here...

hopefully it will occur again on a debug build...

Comment 2

[Intermittent Failures Robot]

16 failures in 126 pushes (0.127 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 10
* try: 2
* mozilla-inbound: 2
* mozilla-central: 2

Platform breakdown:
* linux64: 7
* windows10-64-qr: 3
* windows10-64: 2
* linux32: 2
* osx-10-10: 1
* linux64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1467013&startday=2018-06-06&endday=2018-06-06&tree=trunk

Comment 3

[Tiberius Oros[:tiberius_oros]]

Retriggered jobs for the mochitest-media-e10s test on Linux x64 opt & Windows 10 x64 debug for this failure:
PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::MediaSourceTrackDemuxer::DoGetSamples(int)]

https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=mochitest-media-e10s&tochange=2c78b7574bc1d1b9f5797bf0016438f6121449ec&fromchange=6d7899c2a5eafb705ee898d110fe373aae5fed7c&selectedJob=182050942

on the following pushes:
https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=mochitest-media-e10s&tochange=2c78b7574bc1d1b9f5797bf0016438f6121449ec&fromchange=6d7899c2a5eafb705ee898d110fe373aae5fed7c

At the moment with a few tests still running, i believe this is the push where the failure first appeared:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=4c1f90791492fc441cc600f66ff2d36c7910e905&filter-searchStr=mochitest-media-e10s&selectedJob=182454274

:jmaher, could you please have another look when you have some time? thank you!

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Thanks Tibi for working on this.  I agree that the first time we see this after retriggering is with Bug 1466606:
https://hg.mozilla.org/integration/autoland/rev/4c1f90791492fc441cc600f66ff2d36c7910e905


I did 20 extra retriggers for that revision and the previous one:
https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=mochitest-media-e10s%20linux&fromchange=7f52320924050d171fbd61fbb0ad1b88fa343dfd&tochange=4c1f90791492fc441cc600f66ff2d36c7910e905

:jya- we are gaining more confidence that your code from bug 1466606 is the trigger for this failure- I know you are looking for more debug specific failures.  Is there a chance you could backout your code or take a stab at this without debug failures?

Comment 5

[Jean-Yves Avenard [:jya]]

The change pushed (now backed-out) was designed to make it crash in the failure condition.

the crash can no longer happen

Comment 6

[Intermittent Failures Robot]

27 failures in 669 pushes (0.04 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 12
* mozilla-inbound: 8
* try: 4
* mozilla-central: 3

Platform breakdown:
* linux64: 12
* windows10-64: 6
* windows10-64-qr: 3
* linux32: 3
* osx-10-10: 2
* linux64-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1467013&startday=2018-06-04&endday=2018-06-10&tree=trunk