1436256 - view-source on Mozilla IoT Gateway triggers "Corrupted Content Error", w/ error console showing "Service worker event waitUntil() was passed a promise that rejected with 'TypeError: NetworkError when attempting to fetch resource.'. service-worker.js:14"

Status: VERIFIED FIXED

(Core :: DOM: Service Workers, defect, P2)

Description

[Daniel Holbert [:dholbert]]

NOTE: "yourdomain" is a placeholder below, for whatever subdomain you choose for your own IoT gateway. https://yourdomain.mozilla-iot.org/ does not actually exist (right now) -- it's just an example.


STR:
 1. Set up a Mozilla Things Gateway as described at
https://hacks.mozilla.org/2018/02/how-to-build-your-own-private-smart-home-with-a-raspberry-pi-and-mozillas-things-gateway/

(Alternately, I have a gateway that I've configured which is publicly available on the internet, and I can create an account for you on it if you'd like to test this.)

 2. Create a user account, and log in.  Let's say you chose "yourdomain" as your custom subdomain (I'll avoid posting my own subdomain publicly).

 3. In the web interface, click the menu and choose the "rules" option, which takes you to https://yourdomain.mozilla-iot.org/rules

 4. Ctrl+U to view source.

ACTUAL RSULTS: You get an error page:
> Corrupted Content Error
> 
> The site at view-source:https://yourdomain.mozilla-iot.org/rules
> has experienced a network protocol violation that cannot be repaired.
> 
> The page you are trying to view cannot be shown because an error
> in the data transmission was detected.
> 
> Please contact the website owners to inform them of this problem.

And the error console shows:
> Service worker event waitUntil() was passed a promise that
> rejected with 'TypeError: NetworkError when attempting to
> fetch resource.'.     service-worker.js:14

EXPECTED RESULTS:
Useful page source.


Firefox Nightly 60 gives me "actual results" -- though if I reload the Corrupted Content Error page, then it gets replaced with some useful-looking page source instead of the error page.  So, an explicit reload of the "view-source:" URL seems to fix this, basically.

Chrome gives "expected results" up front.

Ben, would this be in your wheelhouse? I'm happy to create a user account for you on my gateway, if that makes things easier.  It's entirely possible this is a bug in the gateway software which we should fix over there, too (though it is interesting that Chrome has no trouble).

Comment 1

[Daniel Holbert [:dholbert]]

Attachment: the gateway's service-worker.js file

BTW here's the service-worker.js file that's linked to from the error console.

It lives at https://[yourdomain].mozilla-iot.org/service-worker.js

The error is pointing to line 14, which is this line at the end of the 'fetch' listener:
>   event.waitUntil(update(event.request));

Comment 2

[Marion Daly [:mdaly]]

Confirm issue and note implications, will then up priority.

Comment 3

[Ben Kelly [:bkelly, not reviewing]]

Daniel, is there any chance you can get this information:

1. FetchEvent.request contents (url, mode, redirect, etc)
2. Is the fromCache() function falling back to network?
3. What resources are in the cache?  You can see this in the devtools storage panel.

Note we do have some known issues with view-source: links already.  See bug 1317036.

My guess is that this is not finding the resource in Cache API, trying to go to network, but failing.  It would be helpful to know what the URL is.  Does the device have network connectivity to that server, etc?

The service worker script is also a bit weird. It does a fetch() potentially twice.  It also races a cache.match() with a cache.put().

Comment 4

[Ben Kelly [:bkelly, not reviewing]]

Daniel gave me access to the his device.  I'll investigate once I have a working build.  I think this might be something I broke (more) in recent service worker changes only in FF60.

Comment 5

[Daniel Holbert [:dholbert]]

I ran mozregression and got this regression range:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=ba13ada04e711ac26412f2d176a0fa80151455dd&tochange=470a289fb31c86aa989127c3d27221305e703244

--> regression from bug 1231211

(Also, I'm restoring the status from "unconfirmed" back to "new" - I've reproduced this on 2 separate client machines in 2 different locations, so I think we can call this confirmed. And I'm guessing the "unconfirmed" bump in comment 2 was accidental, unless that status has different semantics in this component or something.)

Comment 6

[Ben Kelly [:bkelly, not reviewing]]

I'll investigate.

Comment 7

[Ben Kelly [:bkelly, not reviewing]]

So I have some idea what is happening here, but not quite the full picture yet.

1. When you click on the entry in the menu on the site it uses fetch() to request "https://[yourdomain].mozilla-iot.org/rules" with an Accept value of application/json.  The response has http cache no-store, so its not http cached.  It also has a VARY header that includes 'Accept'.  It seems to use the json to populate the view as a SPA.
2. When you hit ctrl-u to open view-source a FetchEvent is dispatched to the service worker.  It has a URL *without* the view-source: nested scheme, so that is good.  It does, however, have a FetchEvent.request.cache value of 'only-if-cached'.  It also has an Accept header of "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8". This is because of how view-source resources are loaded through docshell.
3. When the service worker does a fetch(FetchEvent.request) the resulting promise rejects because the underlying nsIChannel gets NS_ERROR_DOCUMENT_NOT_CACHED.  This is because either the resource is not in http cache at all, or its only in the cache with an Accept header of */* and the response Vary's on Accept.
4. The fetch() code converts the NS_ERROR_DOCUMENT_NOT_CACHED to NS_ERROR_DOM_TYPE_ERR and then FetchEvent.respondWith() treats it as NS_ERROR_INTERCEPTION_FAILED.  As a result you get a corrupted content page.

What I don't understand yet, is how this works without the service worker.  In that case we should also get the NS_ERROR_DOCUMENT_NOT_CACHED, but the view-source seems to fall back to the network in that case.  Either we are missing some load flag to do that or its not triggering the fallback logic because we are hiding the NS_ERROR_DOCUMENT_NOT_CACHED result.

We have a mechanism for propagating the actual internal error type.  I will investigate trying to make that work here.

Note, I'm not 100% sure this should be called a "regression".  Something changed with bug 1231211 in that we now actually fire a FetchEvent for view-source requests.  This is largely a fix for bug 1317036.  This even works for most sites with a service worker.  We only have a problem here because this site is doing some combination of:

a. Serving the URL as application/json with no-store in some cases.
b. VARY'ing the response on Accept
c. Doesn't store the resource in Cache API so it doesn't find the view-source in cache.match() and instead tries to do the only-if-cache fetch().

For example, if you go to https://blog.wanderview.com, hit ctrl-U, open network panel, and then ctrl-r.  You will see it goes to the service worker and then the SW's fetch() hits http cache.  If you do the same thing on one of the posts it won't do the fetch() because its hitting Cache API.

We should still try to fix it, but some of this is weirdness in the site and weirdness in our view-source code.

Comment 8

[Ben Kelly [:bkelly, not reviewing]]

Actually, the view-source weirdness is a stupid bug in service workers.  See bug 1437080.

Comment 9

[Daniel Holbert [:dholbert]]

I can't reproduce this issue anymore in latest Nightly, version 60.0a1 (2018-02-11) -- seems like that might be expected, per comment 8.

bkelly, should we dupe this to bug 1437080? (Or semi-equivalently, resolve as fixed by that bug?)

And let me know if I should keep your account on my IOT gateway for further testing, or if you're done with it. :)

Comment 10

[Ben Kelly [:bkelly, not reviewing]]

Yea, I believe bug 1437080 fixed this.  Thanks for the test account.  I should be done with it now.

Comment 11

[Daniel Holbert [:dholbert]]

Marking "VERIFIED" based on comment 9.

Retested to be sure. "mozregression --launch 2018-02-09" does indeed trigger this view-source issue, whereas mozregression --launch 2018-02-11 does not.

Comment 12

[Daniel Holbert [:dholbert]]

[and clearing qe-verify:+ because I think that was a request for verification)