Closed Bug 339948 (fuzzjs) Opened 19 years ago Closed 10 years ago

fuzz.js - shared code for JavaScript fuzzers

Categories

(Core Graveyard :: Tracking, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 1172704

People

(Reporter: jruderman, Assigned: jruderman)

References

Details

(Keywords: meta, sec-other, Whiteboard: [sg:nse] meta-ish)

Attachments

(14 obsolete files)

I've converted seven JavaScript fuzzers to share a fuzz.js file, which will live in this bug. fuzz.js should make it easier to maintain the shared code, add features, integrate fuzzers with Spider or even Lithium, etc. The fuzzers converted so far: * Bug 306663 * Bug 306939 * Bug 325861 * Bug 326633 * Bug 331889 * Bug 336383 * Bug 339128
Attached file fuzz.js 1.0 (obsolete) —
Alias: fuzzjs
Whiteboard: [sg:nse] meta-ish
Blocks: fuzz
Attached file fuzz.js 2.0 (obsolete) —
Attachment #224048 - Attachment is obsolete: true
fuzz.js 2.0 * Bumped version numbers of all these fuzzers to 2.0. These will be kept in sync except for subminor numbers (e.g. I could make a Stir DOM 2.0.1 that doesn't need a new version of fuzz.js). * Added support for "hard reductions" where intermediate seeds are recorded. Hopefully this won't be needed too often. * Changed initialization stuff a bit to make Spider happier (hopefully). * Split fuzzers from their test pages. * Renamed some variables to avoid conflicts with Spider and webpage variables. The variable name changes that affect individual fuzzers are: * init -> initFuzzerSpecific * commands -> fuzzCommands * settings -> fuzzSettings
Attached file fuzz-start.js (obsolete) —
I use this when testing a "default" page for each fuzzer.
Attached file fuzz bookmarklet maker (obsolete) —
Based on the Brainjar Crunchinator. Works best in Safari.
Is there a way to run the fuzzer for a given range of seeds, say from 0 to 10 with a limited amount of mods? This would be very useful for me for regression testing.
Attached file fuzz.js 2.0.3 (obsolete) —
Funny you should ask ;) * Fixed a bug where after hitting maxSteps of 1000, it would keep outputting "fuzzer name: 1100", etc. to the console. * Added a scanning mode. For example, you can now scan starting with seed 300 and do 1000 steps with each seed by loading stirdom.xhtml?fuzz=300,0,100,100,1000,0&scan=1. When it gets to 1000, reloads the page with ?fuzz=301,... . It will keep trying new seeds until you stop it, or until Firefox hits a crash/hang/"uncatchable exception" bug ;) Scanning mode has two advantages over running with a single seed for a long time. First, any crashes you hit will be easier to reduce (and will tend to give smaller reduced testcases when reduced only with Lithium). Second, some fuzzers tend not to find bugs after running with a single seed for a while, perhaps because early decisions are important or because the document becomes too small or too large.
Attachment #226743 - Attachment is obsolete: true
Attached file fuzz.js 3.0 (obsolete) —
Changes to fuzz.js: * Move addDOMNodes (used by most of the DOM fuzzers) into fuzz.js. Some of the DOM fuzzers had versions with various bugs; those bugs are no more :) Also, most fuzzers no longer dig into frames, at least until the known crashes involving frames are fixed. * Add simpleSource, which is like uneval for strings and numbers, and use it instead of uneval in many of the fuzzers. This allows more of the fuzzers to work in WebKit in addition to Gecko. * If I cancel the prompt for parameters, run initFuzzerSpecific anyway, so e.g. allNodes[30] still works. This makes by-hand parts of reduction easier. * When a recorded function throws an exception, don't just say "bogus"; display the exception in the status bar. * Better use of gPageCompleted. * Miscellaneous bug fixes.
Attachment #228513 - Attachment is obsolete: true
* Added how-to-use.txt and how-to-reduce.txt. Documentation! * Added some test pages that aren't associated with specific fuzzers (tp subdirectory). * Removed doCommand from all fuzzers except "mutations". It was a source of differences between recorded and non-recorded runs. * Made simpleSource work for strings containing embedded nulls or newlines. (Plus additional changes/tweaks to some of the individual fuzzers.)
Attachment #226747 - Attachment is obsolete: true
Attachment #226756 - Attachment is obsolete: true
Attachment #242966 - Attachment is obsolete: true
Attached file autofuzz (obsolete) —
For running lots of fuzzers on lots of URLs overnight. Only tested on Mac, and probably only useful on Mac (relies on Mac OS X Crash Reporter for crash logging).
Attached file Autofuzz Win32 (obsolete) —
Port of Jesse's Autofuzz for Windows. Also utilizes a method for organizing crashes based on a hash of their information. Written for VS 2005 .NET in C++ with managed extensions.
Also, I forgot to include something about the win32 autofuzz port in the readme. Do not try to run while another instance of Firefox is open. This may confuse autofuzz since new instances of Firefox will be closed and fused with the first (I think), and the fuzzer will think the program terminated completely, opening new instances ad infinitum, causing your system to go berserk.
Added canvas fuzzer
Another thing about autofuzz. The usage string says the timeout argument is "/to", when its actually "/t". Not sure if it makes much of a difference, since Jesse has informed me that for the most part I'm the lone Windows-user :)
Includes major changes and improvements to fuzzer-designMode.js
Also, I don't know if 3.3 was built on top of 3.2 (I might have accidently built it on 3.1), so I just made both the 3.3 changes and the 3.4 changes together to version 3.2 and uploaded that. The recent bugs I've been uploading were found with the new fuzzer-designmode. I have a few more that I have to sort through - I've been experimenting with new ways of reducing and analyzing testcases, but I should have bugs filed within the next day or two.
Manual merging is no fun, even when we're lucky enough to not touch the same files during the same weeks. I wonder how hard it would be to set up a secret repository (CVS, SVN, etc) for these things. * Factored isInDoc out into fuzz.js. * makeCommand can now return an array of function strings instead of a single function string. This makes reduction much easier with the "Element and Attribute" fuzzers. * Included new versions of fuzzer-designMode.js and fuzzer-canvas-api.js from pvnick. I think these were the only files he modified.
Blocks: 382389
Attached file fuzzer-all.js (obsolete) —
Contains many of the fuzz.js fuzzers split into classes in one script file. This allows you to test using all fuzzers at once, potentially finding bugs that wouldn't be found using the individual fuzzers alone. Use fuzzer-all.js the same way you would any of the other fuzzers. Contains stirdom, stirtable, stirattributes, randomstyles, randomjs, randomclasses, htmlattributes, and deepen.
Attached file plan for combining fuzzers (obsolete) —
A lot of the fuzzers need changes in order to work well when running along with other fuzzers. I'm planning to make these changes over the next few days.
Once the fuzzers have been edited to work well with each other, combining them into one fuzzer is quite simple. They can be separated into classes, with one central makeCommand calling different makeCommands at random. Because the individual fuzzers are constantly updated, we should take into consideration an easy way to update the combined fuzzer (eg separate script files?).
I'm just going to remove the individual fuzzer .js files as I combine them into one large .js file. I'll make sure it's easy to run e.g. "just Stir Table" so we don't lose anything by combining them.
For me it would be nice to have separate fuzzer .js files and that it is automagically recognized that multiple fuzzer .js were loaded and that random commands were taken out of those. But it's not a big deal, though.
Changed files from 3.6: fuzzer-designMode.js - updates
Fixed some bugs for my hasty upload of the updated fuzzer-designMode ;)
The following fuzzers have been combined into fuzzer-combined.js: * fuzzer-bindings.js * fuzzer-html-attributes.js * fuzzer-javascript-src.js * fuzzer-mathml-attributes.js * fuzzer-mutations.js * fuzzer-new-script-nodes.js * fuzzer-random-classes.js * fuzzer-random-js.js * fuzzer-random-styles.js * fuzzer-stir-attributes.js * fuzzer-stir-dom.js * fuzzer-stir-table.js * fuzzer-svg-attributes.js * fuzzer-xul-attributes.js Those 14 files had 4963 lines total. fuzzer-combined.js has only 3399 lines (a 31% reduction), mostly thanks to eliminating redundant code. Additional changes: * Made "Random Classes" work around bug 379178 (to test MathML better). * Rewrote "Mutation events", eliminating its special tree-like commands. * Made "New Script Nodes" (etc) do more interesting things when recursing. * Changed how "Stir Table" picks and refers to elements. * Changed how "Stir Table" outputs complex commands. * Fixed all the strict warnings (modulo bug 383574). * Added "Clone node". * Added "Change root". * Removed the "Special recording" stuff from fuzz.js. * Removed the "win" / "doc" stuff from fuzz.js. This zip includes the newest version of pvnick's fuzzer-designMode.js, with "doc" changed to "document" throughout.
Attachment #268201 - Attachment is obsolete: true
Includes... -even more updates to fuzzer-designMode.js, with some trivial, but important, bug fixes. -fuzzer-combined-dm.js, which is basically fuzzer-combined.js with a wrapper for the designmode fuzzer. -some version continuity stuff
minor change in fuzzer-combined-dm.js that allows newscriptnodes to work
i dont know what it is with me and finding bugs in my code right after i release it.
I'm pretty excited about this update. Included are significant changes to fuzz.js that make testcase handling and reduction easier. I included a feature that outputs a script for recreating the DOM. This script is split into one line per element, so automated reducers such as Lithium can reduce the testcase markup without being confused by dynamic allNodes indices. This new serialization technique is superior to DOM Inspector's, since it works with namespaces and was designed for text/html. Once the DOM has been created, the script that fuzz.js produced will output the new DOM's innerHTML, in case you want to refer back to markup. Fuzzer-combined and fuzzer-combined-dm have both been converted to work with the new reduction technique.
Btw, details on how to use the new reduction techniques are in how-to-reduce.txt
Changed in 4.3: - Changed the way that body, head, etc. nodes are added to the DOM. They are now added last so that the whole DOM will be rendered at once. Changed in 4.2 (forgot to make a clear changelog): - Added serialization technique to make reduction easier - Added doNodeWork which calls addDOMNodes when necessary, and also calls buildDOM, which is the function that creates the DOM of the testcase dynamically, if it exists. - Added variable to specify how many fuzzers fuzzer-combined.js chooses from allSubFuzzers - DOM constructor outputs innerHTML of root node that is build - Moved findInAllNodes to fuzz.js - Changed fuzzer-combined.js and fuzzer-combined-dm.js to work with the updated fuzz.js - Updated how-to-reduce.txt - Specify outputDOMConstructor=true to have the serialization feature output DOM construction script - Call doNodeWork instead of addDOMNodes - Added justDump (dumpln without newline)
Broke something last time I uploaded. Fixed in this version
fuzz.js * Separated serializeDOMAsScript from addDOMNodes, so serializeDOMAsScript can be used again after reduction if desired. * Changed serializeDOMAsScript to output each attribute on a separate line, so Lithium can strip out unnecessary attributes. Similarly, put the appendChild calls on their own lines (which also simplifies ordering). * Added serializeHTML, which can serialize an HMTL DOM as either HTML or XHTML. This is convenient if you're almost done reducing the DOM construction/modification script and want to move back to markup. (Based on pvnick's code from bug 382389.) * Added a "FRC1" or "FRC2" prefix before each line of output, so a quick "grep FRC logfile | pbcopy" will include everything you need to start reproducing and reducing, including DDBEGIN/DDEND lines. The serializeDOMAsScript output is tagged with FRC2, so grep for FRC1 if you want to skip it for some reason. * Added ensureIds. (Helpful for manual reduction, especially when you can't use the "serialize DOM as script" trick and have to reduce lots of markup.) * Added fuzzRetry. (Helpful for reducing "sometimes crashes" testcases.) * Improved Safari compatibility by making dumpln() call console.log() instead of dump(), depending on which is available. * Improved Safari compatibility by removing justDump. * fuzz-start.js now removes its event listener, so multiple onloads don't confuse it. fuzzer-combined.js * Added fuzzerContentEditable, which makes subtrees editable and calls focus() a lot. * Random Classes and Random Styles now test *all* CSS properties in addition to the 50 or so it knows useful values for. It gets a list of all CSS properties using "computed style" APIs. It has to make wild guesses for the values for these properties, of course. * Taught it about several new SVG features. * Improved Safari 2 compatibility: don't use named functions. how-to-reduce.txt * Updated to reflect the awesome new tools and the new reduction strategy I've been using now that I have the new tools. I didn't update fuzzer-combined-dm.js, so I didn't include it.
fuzz.js * Make simpleSource use uneval instead of encodeURIComponent for non-ASCII strings. encodeURIComponent throws for some strings, such as invalid UTF-16. This change hurts cross-browser compatibility, but only for non-ASCII strings. * Fix a bug in serializeDOMAsScript that leaves allNodes[0] undefined. fuzzer-combined.js * Incorporate fuzzer-text.js as fuzzerChars, replacing fuzzerTextNodes. Make it operate on text node data rather than element.textContent and element.innerHTML in the hope of making reduction easier.
* Change eaCommandMaker::makeSetAttribute to mix things up more and sometimes set things as properties instead of attributes. * Change fuzzerRandomJS::makeCommand so it doesn't access properties directly, since this was triggering crashes without corresponding fuzz output. * Test pseudo-classes such as :root and :hover in fuzzerRandomClasses. * Rename variable "count" to "fuzzCount" to avoid interfering with web page script. * Make aC and iB act like rM if the first argument is missing. This helps in reduction, especially with serializeDOMAsScript, since a node might disappear entirely. * Change goQuitApplication so each page doesn't need UniversalXPConnect. Smaller changes: * Test percentage padding and margin better. * Test more mime types. * Fix various scattered bugs that caused it to stop.
Attached file Autofuzz (Mac) (obsolete) —
These are the scripts I use for overnight runs. I now usually have it pick a random reftest file and run some random DOM fuzzers on it for ~30 seconds or ~3000 commands. It usually goes through 1000 to 2000 Firefox instances a night. Now with known_assertions.txt (for ignoring assertions I know about bugs on) and a script for telling me about interesting trace-refcnt leaks.
Attachment #256014 - Attachment is obsolete: true
Added the ability to specify a custom interval for specific rest commands. To do so, use something like { origCount: 0, rest: true, timeout: MILLISECONDS }
* fuzzerMutationEvents is now fuzzerDuringEvents and tests a lot more events (e.g. onbeforepaste, onchange). * fuzzerContentEditable now tests execCommand. * Make custom interval work even if the interval is 0ms. * Compatibility fixes for Opera and Safari.
* Attempt to improve IE compatibility
* More IE compat * Un-comment-out fuzzing of -moz-column-*
-Added fuzzerInterconnectedness, which is an attempt at making the fuzzers work with each other. Right now it doesn't have many features, and only fuzzerRandomJS takes advantage of it. -Function to replace objects in fuzzer commands with fuzzed references -Function to replace function arguments in fuzzer commands with fuzzed arguments (with support for function calls used as parameters) -Since RandomJS uses commands from all other fuzzers (thus having a wide attack surface) it is available to load consistently by setting loadRJS to true. -All RandomJS n1 references changed to rjn1 to avoid name collisions
-Added fuzzerInterconnectedness.changeRndScriptVarValue, which edits the values that variables are set to in random fuzzer commands -Changed the way fuzzerInterconnectedness handles commands, allowing for command arrays fuzzerInterconnectedness allows testing specific features across most of the fuzzer modules; it centralizes command-editing
Removed debugging code and backup files
Added a simple vb6 activex control that lets internet explorer communicate with the fuzzer handler (like window.dump) using debug strings. Everything is in the iecompat folder. Runfirst.bat must be executed before the dumper can be used.
Attached file Autofuzz GM4IE file (obsolete) —
This is what you download for GM4IE (Internet Explorer's version of GreaseMonkey). You can get GM4IE at http://www.gm4ie.com/ Use the same way you would with Firefox
Attachment #294512 - Attachment mime type: application/octet-stream → text/plain
I didn't know GM4IE doesn't allow https files. You have to manually save autofuzz.js from above to program files\gm4ie\scripts\autofuzz.js and extension.list to program files\gm4ie\extension.list (replace of the existing file or add the contents to the end)
Attachment #294512 - Attachment is obsolete: true
Attachment #294511 - Attachment description: autofuzz.js for ie greasemonkey → Save this to program files\gm4ie\scripts\autofuzz.js
Attachment #294511 - Attachment mime type: application/x-javascript → text/plain
Comment on attachment 294511 [details] Save this to program files\gm4ie\scripts\autofuzz.js > > >function load(sURL) >{ > var script = document.createElement("script"); > script.src = sURL; > var par = (document.body || document.documentElement); > > par.appendChild(script); > > function yank() { par.removeChild(script); } > > setTimeout(yank, 300); // so this *temporary* script element doesn't confuse the fuzzer >} > >if (location.hash.indexOf("#squarefree-af") > -1) >{ > var prefix = "http://localhost/fuzzers/"; > > load(prefix + "fuzz.js"); > > //some fuzzers want to show some information. we dont need that! > window.alert = function(){void 0;}; > > // Specified fuzzer? > r = location.hash.slice("#squarefree-af".length + 1); > > r = r.split("!"); > > load(prefix + r[0]); > > setTimeout(function() { > eval("fuzzSettings = [" + r[1] + ",suggestedMaxSteps," + (r[2] ? r[2] : 0) + "];"); > eval("setTimeout(fuzzOnload, 400);"); > }, 600); >} >
Woops. I thought that would actually edit the attachment. Fixed a minor bug and reuploaded. Make sure to change http://localhost/fuzzers in the script.
Attachment #294511 - Attachment is obsolete: true
Everything is included in the zip. I hope everyone doesn't mind if updates are applied to this version. Included: -Fuzzers -Autofuzz win32 and mac -Script that works for greasemonkey and gm4ie. -ActiveX control for internet explorer logging Changes: -Added recursive object padding (eval($), [$], etc) to fuzzerRandomJS -Fixed the way rndDecProb picks numbers so we don't freeze up anymore -Added fuzzerDesignMode as a module, which can be optionally disabled -Added freezeBrowser (disabled by default), which causes an infinite loop and leads to the frozen script dialog, which sometimes crashes oddly but can't be reliably reproduced. I'm working on this now
Attachment #264702 - Attachment is obsolete: true
Attachment #290808 - Attachment is obsolete: true
Attachment #294514 - Attachment is obsolete: true
Attachment #294707 - Attachment is obsolete: true
Changed since 4.8: -Fixed a bug in autofuzz.user.js -Added extension that allows privileged code to run from an unprivileged website (for things like mouse control and pagination) -Added fuzzerUserInteraction (disabled by default, set simulateUserInteration=true in fuzzer-combined to enable. You can change the odds of using it in global makeCommand) -Simulate print preview (pagination) -Simulate mouse actions, and I'm pretty sure this is deterministic, too -Can be loaded with simulateUserInteraction = true, and is called less frequently than the other fuzzers (odds set in global makeCommand) -Removed disableDesignModeFuzzer as a variable (redundancy): remove fuzzerDesignMode from allSubFuzzers to disable -Added mouse.exe, which watches a file (controlled by the browser) to simulate mouse actions -Added support for mouse.exe to autofuzz win32 and fixed some bugs in there, too Elevateprivs.xpi must be installed for user simulation to work. Unfortunately mouse simulation is only available on windows, since it uses an external application to control the mouse. However, pagination should work on other OSs; just change paginationDump to the path you wish to dump to. To enable pagination fuzzing, the browser must be a debug build with --enable-extensions=all and --enable-tests added as options to .mozconfig How mouse.exe works (in case someone wants to make a port): Mouse.txt, in the directory of mouse.exe, is watched for one of the following commands: move, click, drag, rightclick. After the command are 2 coordinates (x and y) except for drag, which has 4 coordinates (x1, y1, x2, y2). Examples: click 100 200 drag 100 200 500 400 Once the command is read and executed, the contents of moust.txt are erased to tell the browser to continue.
From what I know pagination is not the same as print preview.
I wasn't quite sure, but bug 351322, where I got the pagination fuzzer code, seemed to use the terms interchangeably, so I guess pagination takes place during printing and print preview? (http://en.wikipedia.org/wiki/Pagination ) It seems like it's pagination that we're actually fuzzing (I got a few bugs from this that I need to process), since bug 351225 is mentioned in the pagination fuzzer bug as being a bug that would be found while pagination fuzzing.
Ah, sorry, that seems to test print preview, indeed. I thought you mentioned pagination mode, that is what bug 374050 is about.
Pagination fuzzing means to take a arbitrary document and to force a print preview on it. This requires layout to break objects at the page boundaries and to create nextInFlow frames. A pagination fuzzer will test this code path which is rather seldom tested. Paul, thanks for picking this up.
The same frame manipulations will happen of course also during ordinary printing, but print preview does the same job without producing wasted paper.
Added testing fullscreen and enlarge/reduce text to the UI simulator Changed the way fuzzerDesignMode is loaded to allow for DOM serialization Enabled some commented code (might start catching old bugs, but must are fixed, I think) Fixed a bug in the execcommand tester Fixed a freeze-up bug in the way commands are read
Keywords: meta
This code now lives in a private mercurial repository. If you want access, email me and I'll file a bug asking to have you added.
No longer blocks: fuzz
I had to make a few attachments here private, but the bulk of the code is now public as part of https://github.com/MozillaSecurity/funfuzz/.
Group: core-security
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → DUPLICATE
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: