1338017 - [meta] Minimize the safebrowsing DB update blocking time

Status: RESOLVED FIXED

(Toolkit :: Safe Browsing, defect, P1)

Description

[Henry Chang [:hchang]]

Long story short:

The URI lookup will be blocked by background DB update. We want to improve this.

Long story longer:

nsIURIClassifier.classify is an async call to do the DB lookup and hash
completion. It doesn't block the caller. Perfect!....? No. Even it doesn't
block the caller, it blocks opening a channel. It doesn't block any thread
but the channel will take longer time to open than it's supposed to be. 

We have no choice if DB is not present: either let the channel go or wait.
However, if we have a outdated DB already, we can try to minimize the 
waiting time by looking up the outdated DB.

Comment 1

[Henry Chang [:hchang]]

This is a major change so I'd like to do it progressively:

1) Change the DB update scheme. 

   The reason we block DB lookup while updating is because we directly update 
   the DB which is used for lookup. We can improve this by copying and updating 
   DB in a temporary folder then "swapping" the updated and outdated ones.

2) Update DB on a off-worker-thread.
   
   All DB operations occurs on a single worker thread for now. If (1) is implemented,
   we can do 99% of the DB update off the worker thread. The only 1% exception is
   "swapping". We have to block DB lookup while swapping updated and outdated DBs.
   This subtask is much harder than (1) so I'd further like to split (2) to even smaller tasks:

   (a) Take DB update codes out of nsURLClassifierDBService.
   (b) Run the codes we took out in (a) off the worker thread. This might be split again:
       (i) Firstly we run the update code off the worker thread BUT leave it still
           block the worker thread.
       (ii) Afterward, we carefully let the off-worker-thread unblock the worker thread
            whenever possible. (Swapping DBs must either block worker thread or just occurs
            on worker thread.)


Only 2.b.(ii) MUST be done lastly. 1, 2.a, 2.b.i can be done independently.

Comment 2

[Henry Chang [:hchang]]

All the dependent bugs have been fixed so I am closing this one.