503469 - Support packaging WinCE Firefox builds as zip, complete.mar

Status: VERIFIED FIXED

(Firefox Build System :: General, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Need to be able to create dist/firefox from dist/bin, then use that to create the cab, zip and complete.mar files for nightly builds.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: WIP - gets as far as a zip

This expands some ifdef conditions from just WINNT to WINNT and WINCE, and includes the WinCE specific file in windows/packages-static. 

There are several warnings about missing files (eg mozcrt19.dll, crashreporter.*, softokn3.chk, freebl3.dll) that need looking at and confirming as OK. Others are expected, eg missing updater.*, AccessibleMarshall.dll, from --disable's in the mozconfig. Example build,
http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-wince/1247199251/
on MPT-VPN. That location has a fairly short life, 1 day I think.

This doesn't do anything towards creating a cab file. I had a shot at borrowing http://hg.mozilla.org/mobile-browser/file/tip/installer/Makefile.in#l115
with tweaks but hit issues, mostly because it's tangled in browser/installer/windows/Makefile.in. Ted ! Help !

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: WIP - creates a zip that actually works

There's a couple of extra Makefile changes here, relaxing OS_ARCH tests to include WINCE so that we get xpt files named correctly (and hence getting picked up by the manifest during make package). 

Remaining warnings on 'make package' are:
 bin/defaults/existing-profile-defaults.js - OK, we hit this on win32 too
 bin/uninstall/helper.exe - OK, no installer
 bin/mozcrt19.dll - in libxul, not built ?
 bin/plugins/npnul32.dll - ?
 bin/AccessibleMarshal.dll - OK, accessibility disabled in mozconfig(and the next two)
 bin/components/accessibility.xpt
 bin/components/accessibility-msaa.xpt
 bin/components/layout_printing.xpt - ?
 bin/components/migration.xpt - ?
 bin/components/proxyObject.xpt - ?
 bin/components/browserdirprovider.dll - ?
 bin/softokn3.chk - ?
 bin/freebl3.chk - ?
 bin/nssdbm3.dll - ?
 bin/crashreporter.exe - OK, disabled in mozconfig (and the next two)
 bin/crashreporter.ini
 bin/crashreporter-override.ini

Comment 3

[Phil Ringnalda (:philor)]

mozcrt19.dll is jemalloc, only around #ifdef MOZ_MEMORY

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Fix or suppress a bunch more packaging warnings

Updated manifest checks:

  bin/defaults/existing-profile-defaults.js - OK, we hit this on win32 too
  bin/uninstall/helper.exe - from --disable-installer in mozconfig

  bin/mozcrt19.dll - MOZ_MEMORY is defined, but WinCE folds jemalloc into mozce_shunt.dll - packages-static adjusted in this patch

  bin/plugins/npnul32.dll - Not built because of $(OS_ARCH) == WINNT test at
http://mxr.mozilla.org/mozilla-central/source/modules/plugin/Makefile.in#63
  Do we want the default plugin anyway ?

  bin/AccessibleMarshal.dll
  bin/components/accessibility.xpt
  bin/components/accessibility-msaa.xpt -  --disable-accessibility in mozconfig

  bin/components/layout_printing.xpt - from --disable-printing in mozconfig

  bin/components/migration.xpt - explicitly disabled for WinCE at
http://mxr.mozilla.org/mozilla-central/source/browser/components/Makefile.in#76, packages-static updated here

  bin/components/proxyObject.xpt - from
  http://mxr.mozilla.org/mozilla-central/source/xpcom/proxy/public/Makefile.in#46 - adjusted in this patch

  bin/components/browserdirprovider.dll - from http://mxr.mozilla.org/mozilla-central/source/browser/components/dirprovider/Makefile.in#47 - updated in this patch

  bin/softokn3.chk - 
  bin/freebl3.chk - both from http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/installer/packager.mk#184 - we seem to disable the chk files when cross compiling. Do we need them ?

  bin/nssdbm3.dll - from --disable-dbm in mozconfig

  bin/crashreporter.exe 
  bin/crashreporter.ini
  bin/crashreporter-override.ini - "#define MOZ_CRASHREPORTER 1" is missing from mozilla-config.h, from http://mxr.mozilla.org/mozilla-central/source/configure.in#5716, ie not a supported platform, packages-static updated for this

Also adds .NOTPARALLEL to build/wince/shunt/Makefile.in because I was hitting spurious errors about missing vs90.pdb when using make -j4.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 391048 [details] [diff] [review]
Fix or suppress a bunch more packaging warnings

Oops, I wasn't going to set r? until I had a chance to test the resulting build on a device, and verify that the other platforms weren't regressed with a try build.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Package crashreporter on desktop builds [Checkin: Comment 11]

This is a slight tweak on attachment 391048 [details] [diff] [review], correcting the test for packaging the crashreporter (which the try server caught). Unit tests were all green. The Makefile.in changes to XPIDL_MODULE affect WinMo too.

Comment 7

[(not currently active) Ted Mielczarek]

Comment on attachment 391278 [details] [diff] [review]
Package crashreporter on desktop builds
[Checkin: Comment 11]

+#ifndef WINCE
 ; [Crash Reporter]
 bin\crashreporter.exe
 bin\crashreporter.ini
 bin\crashreporter-override.ini
+#endif

I'd rather this was ifdef MOZ_CRASHREPORTER, since we will probably eventually ship Breakpad on wince.

diff --git a/toolkit/mozapps/installer/packager.mk b/toolkit/mozapps/installer/packager.mk

Have you verified that the changes to packager.mk don't break packaging of Windows Mobile builds? I'm particularly worried about adding the INSTALLER_DIR bits there.

Looks fine otherwise.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #7)
> (From update of attachment 391278 [details] [diff] [review])
>> +#ifndef WINCE
>>  ; [Crash Reporter]
> I'd rather this was ifdef MOZ_CRASHREPORTER, since we will probably eventually
> ship Breakpad on wince.

I'll fix this on checkin.

> diff --git a/toolkit/mozapps/installer/packager.mk
> b/toolkit/mozapps/installer/packager.mk
> 
> Have you verified that the changes to packager.mk don't break packaging of
> Windows Mobile builds? I'm particularly worried about adding the INSTALLER_DIR
> bits there.

I ran the patch + fix from above through the try server and compared the logs from WinMo's 'make package' and 'make installer' to another try server run, and to the main mozilla-central builds. The only difference appears to be the order in which files are copied out of dist/bin. I also checked that the fennec and xulrunner zip files contain the same set of files; can't really do that for the cab since of the semi-random way the files get renamed.  

Anything else I could do to put your mind at rest ?

Comment 9

[(not currently active) Ted Mielczarek]

Might just get someone to try installing the cab you produced on a WinMo device or emulator and ensure that it works, I think that would fully satisfy me.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Aki reports that the try server WinMo build runs fine, and the file list is the same as the regular builds.

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

Patch as landed: http://hg.mozilla.org/mozilla-central/rev/793b4c9e138f

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Disable parallel make in build/wince/tools too [Checkin: Comment 13]

I've hit this bogus vc90.pdb error in the other build/wince sub-directory now:
e:/builds/slave/mozilla-central-wince-nightly/build/build/wince/tools/arm-wince-res.c : fatal error C1033: cannot open program database 'e:\builds\slave\mozilla-central-wince-nightly\build\obj-firefox\build\wince\tools\vc90.pdb'

Google hits imply this is mspdbsrv's fault, and the workaround is to disable parallel builds. Since we build the rest of the tree -j4 normally I'm hoping this will be enough.

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 394251 [details] [diff] [review]
Disable parallel make in build/wince/tools too
[Checkin: Comment 13]

http://hg.mozilla.org/mozilla-central/rev/13bc65e5eeec
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/0107d240e10e

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Package splash.bmp if MOZ_SPLASHSCREEN is defined [Checkin: Comment 15]

The branding dirs copy splash.bmp into dist/bin ifdef on MOZ_SPLASHSCREEN, so this just adds it to the packaging list on the same condition. That var defaults to undefined so this is safe for desktop Firefox. The only thing I don't follow is how the WinMo builds are getting splash.bmp now, as it appears they're using the stage-package from toolkit's packager.mk.

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 395026 [details] [diff] [review]
Package splash.bmp if MOZ_SPLASHSCREEN is defined
[Checkin: Comment 15]

http://hg.mozilla.org/mozilla-central/rev/167d89514f81
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/c7586a60721f

Comment 17

[Mike Beltzner [:beltzner, not reading bugmail]]

Mass change: adding fixed1.9.2 keyword

(This bug was identified as a mozilla1.9.2 blocker which was fixed before the mozilla-1.9.2 repository was branched (August 13th, 2009) as per this query: http://is.gd/2ydcb - if this bug is not actually fixed on mozilla1.9.2, please remove the keyword. Apologies for the bugspam)

Comment 18

[Tony Chung [:tchung]]

Verified fix on 
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/
and 
ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-1.9.2/