1401481 - [Crash in shutdownhang | js::Thread::join] MOZ_CRASH(Shutdown too long, probably frozen, causing a crash.)

Status: RESOLVED WORKSFORME

(Core :: JavaScript Engine, defect, P2)

Description

[Trevor Skywalker]

This bug was filed from the Socorro interface and is 
report bp-1382c272-98eb-44bc-802d-978690170920.
=============================================================

UUID 	82518226-751a-4c8a-8b83-69c960170920
Date Processed 	2017-09-20 00:32:13
Uptime 	53,483 seconds (14 hours, 51 minutes and 23 seconds)
Last Crash 	101,084 seconds before submission (1 day, 4 hours and 4 minutes)

UUID 	1382c272-98eb-44bc-802d-978690170920
Date Processed 	2017-09-20 00:36:32
Uptime 	53,483 seconds (14 hours, 51 minutes and 23 seconds)
Last Crash 	101,084 seconds before submission (1 day, 4 hours and 4 minutes)

Release Channel 	beta
Version 	56.0b12
MOZ_CRASH Reason 	MOZ_CRASH(Shutdown too long, probably frozen, causing a crash.)
Crash Reason 	EXCEPTION_BREAKPOINT

Total Virtual Memory 	8,796,092,891,136 bytes (8 TB)
Available Virtual Memory 	8,791,910,772,736 bytes (8 TB)
Available Page File 	10,978,947,072 bytes (10.22 GB)
Available Physical Memory 	479,772,672 bytes (457.55 MB)
System Memory Use Percentage 	88 

Crashing Thread (0)
Frame 	Module 	Signature 	Source
0 	ntdll.dll 	ZwWaitForSingleObject 	
1 	kernelbase.dll 	WaitForSingleObjectEx 	
2 	xul.dll 	js::Thread::join() 	js/src/threading/windows/Thread.cpp:98
3 	xul.dll 	js::GlobalHelperThreadState::finishThreads() 	js/src/vm/HelperThreads.cpp:899
4 	xul.dll 	JS_ShutDown() 	js/src/vm/Initialization.cpp:160
5 	xul.dll 	mozilla::ShutdownXPCOM(nsIServiceManager*) 	xpcom/build/XPCOMInit.cpp:1025
6 	xul.dll 	ScopedXPCOMStartup::~ScopedXPCOMStartup() 	toolkit/xre/nsAppRunner.cpp:1465
7 	xul.dll 	mozilla::UniquePtr<ScopedXPCOMStartup, mozilla::DefaultDelete<ScopedXPCOMStartup> >::reset(ScopedXPCOMStartup*) 	obj-firefox/dist/include/mozilla/UniquePtr.h:345
8 	xul.dll 	XREMain::XRE_main(int, char** const, mozilla::BootstrapConfig const&) 	toolkit/xre/nsAppRunner.cpp:4802
9 	xul.dll 	XRE_main(int, char** const, mozilla::BootstrapConfig const&) 	toolkit/xre/nsAppRunner.cpp:4869
10 	firefox.exe 	NS_internal_main(int, char**, char**) 	browser/app/nsBrowserApp.cpp:309
11 	firefox.exe 	wmain 	toolkit/xre/nsWindowsWMain.cpp:115
12 	firefox.exe 	__scrt_common_main_seh 	f:/dd/vctools/crt/vcstartup/src/startup/exe_common.inl:253
13 	kernel32.dll 	BaseThreadInitThunk 	
14 	ntdll.dll 	RtlUserThreadStart

Comment 1

[Trevor Skywalker]

Signature report for shutdownhang | js::Thread::join
Showing results from 6 months ago to 6 minutes ago.

Windows 7 	2189 	47.5%
Windows 10 	676 	14.7%
Windows Vista 	645 	14.0%
Windows XP 	560 	12.2%
Windows 8.1 	473 	10.3%
Windows 8 	50 	1.1%
Windows Sr 2003	11 	0.2%

Firefox 	55.0.3 		177 	56.7% 	113
Firefox 	52.3.0esr 	60 	19.2% 	62
Firefox 	56.0b12 	20 	6.4% 	26
Firefox 	56.0b11 	18 	5.8% 	19
Firefox 	54.0.1 		10 	3.2% 	8
Firefox 	56.0b3 		5 	1.6% 	6
Thunderbird 	52.3.0 		5 	1.6% 	5
Firefox 	56.0b10 	4 	1.3% 	5
Firefox 	56.0b6 		2 	0.6% 	2
Firefox 	56.0b8 		2 	0.6% 	2
Firefox 	56.0b9 		2 	0.6% 	2
Firefox 	51.0b6 		1 	0.3% 	1
Firefox 	52.2.0esr 	1 	0.3% 	1

Uptime Range
> 1 hour 	2372 	51.5%
15-60 min 	794 	17.2%
1-5 min 	777 	16.9%
5-15 min 	656 	14.2%
< 1 min 	5 	0.1%

Comment 2

[Trevor Skywalker]

Signature report for shutdownhang | js::Thread::join
Showing results from 6 months ago to 6 minutes ago.


297094e3-c149-45c6-8e66-e762f0170918 	who the help knows; Firefox has been so slow, and then if I open a page and leave it open Firefox is labeled Not Responding by Windows; 	2017-09-18 17:32:14 	en-US 

a88b62a7-f0c9-42c4-b074-9210a0170915 	I stopped FF manually because it had been running for a couple days and I have learned I must occasionally stop it due to memory leaks and hung scripts. 	2017-09-15 15:49:35 	en-US

f01e9519-9c37-480a-92c5-7e18c0170913 	FAILURES CONTINUES 	2017-09-13 14:13:40 	en-US

b3d2be8d-0559-4bc0-a6cd-691e50170906 	Ctrl+Shift+Q crashes systematically Firefox (hence this report). Ctrl+Shift+M does not work. Whereas Ctrl+Shift+A opens about:addons as expected. 	2017-09-06 10:39:34 	fr

2e40d300-cb47-4cc8-bff6-b467d0170905 	I was trying to access the game Simon's Cat on Facebook. 	2017-09-05 01:27:07 	en-US

d7fd799d-c7a2-4e7a-bd39-19e180170816 	Browser ran away with 5G of memory, probably caused by Yahoo, but still not acceptable. 	2017-08-16 17:16:47 	en-US

1decfdb2-49f8-41f4-884e-2619a0170813 	just tell me why it keeps crashing, firefox was once my favourite browser, slowly getting fed up with it now. 	2017-08-13 20:31:15 	en-GB

14c60a70-c8a7-408c-8171-8f0ee0170810 	just keeps crashing every time i get on facebook to play candy crush or just look on my facebook page 	2017-08-10 02:00:16 	en-US

f0eb37a1-058d-4b6f-a5f6-1839a0170807 	attempting to convert LinkedIn document to Adobe PDF 	2017-08-07 20:40:53 	en-US

86f27c40-8e19-4dc3-bfad-1cc060170803 	mozilla crashes too frequently!! 	2017-08-03 00:04:32 	en-US 

82dc21bb-fd7e-4718-904a-c4e3f0170726 	fed up with your pluggins and flash players. I am disabled person and you are giving me damn hard times by your solutions and problems.. Fed up with you.,,, 	2017-07-26 18:12:35 	en-GB

3e120c95-05fd-48a3-8f7b-7ca5a0170726 	OMG - the program is very unstable on Windows 7 	2017-07-26 15:20:12 	en-US

0a324c6f-2d92-40d9-abc2-ab1e90170725 	I didn't know it happened. 	2017-07-25 16:25:37 	en-US

49cec95b-353e-40e0-9b2f-df5820170724 	Firefox developed a virus, Please fix it 	2017-07-24 18:55:48 	en-US

4be96f1f-4018-40d6-9da3-ea8880170720 	This is not the first time I have experienced a problem with Mozilla. Sometimes, I have to closedown the PC and swith on again 	2017-07-20 06:12:22 	en-GB

cf5d9747-bf9c-45af-94c9-ac4550170718 	a programe called ( myway) from google chrome arrived on my computer two days ago. I tried to remove it using the tools on my computer but was unable to do so. I down loaded an anti virus programe called BULGUARD but I have been unable to register this programe and MYWAY is still present 	2017-07-18 16:23:20 	en-US

10e400fc-de04-4587-8a7d-1ffb10170717 	FYI - lately when we close multiple tabs (4 or more) a crash report is being generated... but actually there was no crash. thanks for your attention in this matter 	2017-07-17 21:37:27 	en-U

689ce0c6-9dea-4793-9f85-c8daa0170713 	Didn't crash. Supposed to be an update and I shut it down. 	2017-07-13 02:00:57 	en-US

e070c44d-a24e-4e75-aee4-125330170712 	mozilla is getting slower than usual 	2017-07-12 12:23:22 	id 

aeb8b4e1-f6fc-4aab-87dd-20fc50170710 	I was just Looking at a video and firefox decided to crash sorry about this 	2017-07-10 15:21:10 	en-US

233b936d-03da-43bc-b703-8f7870170709 	The browser could not allocate enough memory for the WebGL content. If you are the developer of this content, try allocating less memory to your WebGL build in the WebGL player settings. 	2017-07-09 00:08:24 	zh-CN 

42a988d7-ec5d-4c3c-a1df-dfdd70170706 	Don't really know what to tell you. Just every time I tried to do anything in any tab Firefox would do part of the step and then freeze up and I'd get the message up on the top that it wasn't responding. After a few minutes it would come back for just a minute and then stop again. 	2017-07-06 19:28:09 	en-US

0a4e57cb-086f-4fc2-bf66-c0c5a0170617 	I get" firefox is not responding" every day for the past few weeks 	2017-06-17 22:05:44 	en-US

3b30ebed-61ef-4e9b-8390-3b1e10170615 	This is getting beyond a joke. How can we carry out our work responsibilities useing tools that simple don't respond??? 	2017-06-15 12:47:33 	en-GB

05f1e669-0386-4174-a91c-fb9700170512 	Firefox FOR MANY YEARS gains too much memory. It cannot free memory used by itself and gains more and more until PC crashes or until FF crashe :( 	2017-05-12 05:33:43 	pl 

762d4bd0-b708-498b-8c0f-2d2bb0170506 	I always get that Mozzilla is not responding (it is so slow) 	2017-05-06 11:14:35 	en-GB

af0b911e-b400-455a-98b4-5b72e0170428 	it is very very boring matter .. user want browser never crush.. if next time it will crush i do not use this browser. 	2017-04-28 21:09:50 	en-US

Comment 3

[Naveed Ihsanullah [:naveed]]

Brian, can you please evaluate this crash? It appears to have spiked on 55 with no presence currently on 57. Perhaps we could get some guards or asserts into beta to help us collect data.

Comment 4

[Brian Hackett [Laid off!]]

OK, I looked at 20 of these crashes and see several different potential problems.  This hang happens when the main thread is shutting down and is waiting for all the JS helper threads to finish; if one of them is deadlocked or is doing something that is taking too long, we end up crashing.  It's generally easy to see what the problematic helper thread is doing by looking at all the thread stacks in the crash report.  Anyways:

110622a5-5c68-45c2-a549-7a2360170921
101d6659-6e67-4d64-b005-b843c0170921
a7d7941e-3d05-4686-9968-76af30170921
5a2430e5-34d4-4d1b-aa6d-abeca0170921

In these reports the helper thread does not seem to be blocked but is freeing the contents of its context's name collection pool.  Before the multithreaded runtime stuff (bug 1325050) a separate pool was created for each parse task and was destroyed afterwards, but now there is a single pool per thread and it doesn't look like we ever purge the pools for helper threads, causing both this hang and wasted memory.  I'll write a patch for this and file a bug for it in a bit.

97c882e3-bbdb-450f-843e-8c2f50170921
007102f4-7707-40e0-8347-34e060170921

In these two reports a helper thread seems to be blocked in a parse task while waiting for an associated source compression task to complete.  This code was removed by bug 1348134; maybe it would be worth uplifting those changes, but I don't know.

c7095e84-af4a-4005-86f3-110580170921
99da3d03-7620-4ab2-b0fc-d55900170921
b34f9b63-03ef-472e-bd87-ea1130170921
17599028-d82f-42be-89fa-546060170921
bce8fcf8-5167-4b18-ac2f-6de080170921
dd4a32fb-d783-43a1-b355-416180170921
9ce10385-4eed-4012-b61a-94e550170921
d7d5ee1c-acbc-4ca1-9831-fa24b0170921

In these reports a helper thread seems to be blocked while resolving a promise task.  This code was rewritten in bug 1347644, I don't know whether those changes should be uplifted.

6488e981-4c34-48a3-8a40-726800170921

In this report a helper thread seems to be blocked somewhere in wasm compilation (ModuleGenerator::finishOutstandingTask).  This code changed in bug 1395587, I don't know whether those changes should be uplifted.

e3a1b6b6-b62c-46ca-8cdb-a5d560170921
b96807e4-ecc8-47b6-9b2a-3ea4c0170921
b2cd12b6-ec9f-48dc-8fc5-824400170921
04dead44-ba69-44df-8e80-255230170921
8ec744a1-d554-45f3-a44e-36fa60170921

The stack information seemed unhelpful/incorrect in these reports, I don't know what's going on in these.

Comment 5

[Andrew McCreight [:mccr8]]

I don't see any stacks with js::Thread::join in them with the signature [@ shutdownhang | ZwWaitForKeyedEvent | RtlSleepConditionVariableCS | SleepConditionVariableCS], at least for beta, so I'm going to remove it to reduce confusion.

Comment 6

[Sylvestre Ledru [:Sylvestre]]

My bad, sorry, I was confused with the other bug :)

Comment 7

[Intermittent Failures Robot]

3 failures in 3712 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• windows10-64-mingwclang: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1401481&startday=2020-11-30&endday=2020-12-06&tree=all

Comment 8

[Intermittent Failures Robot]

3 failures in 4060 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• windows10-64-mingwclang: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1401481&startday=2020-12-07&endday=2020-12-13&tree=all

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

Since the crash volume is low (less than 5 per week), the severity is downgraded to S3. Feel free to change it back if you think the bug is still critical.

For more information, please visit auto_nag documentation.

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

Closing because no crashes reported for 12 weeks.