Web Site Quality Assurance Testing using Mozilla
Table of Contents
- Mozilla and Web Site Testing
- Example Document
- Mozilla Release/Nightly Builds
- NSPR (Netscape Portable Runtime) Logs
- Debug Mozilla Builds
- Customized Mozilla Builds
- Automated Web Site Testing
- Conclusion
Mozilla and Web Site Testing
Henrik Gemal has written an excellent article Using Mozilla in testing and debugging web sites which details many of the features available in Mozilla which can help you find and correct errors in your web pages. Henrik focuses upon the interactive features such as the JavaScript™ Debugger and DOM Inspector which are available in official Mozilla releases and Mozilla nightly builds. Svend Tofte has also written an excellent tutorial Learning the JavaScript debugger Venkman.
While these interactive features are extremely useful when developing and testing web content, they do not provide the full range of information which Mozilla can provide nor do they lend themselves to use in batch-oriented automated testing of web sites. This article illustrates the use of Netscape Portable Runtime (NSPR), Debug Builds of Mozilla and customized versions of Mozilla to obtain additional information which can be used in Quality Assurance testing of web sites.
Example Document
Throughout this article, the web page
example-html-page.htm
will be
used as an example to illustrate the different methods of recording
information about a web page using Mozilla. This page has
an HTML 4.01 Strict DOCTYPE which invokes Standards
mode in Mozilla and contains the following:
-
Sets and retrieves a cookie.
-
Uses Mozilla's
dump
function, to write the cookie to STDOUT. -
Contains a JavaScript error (a reference to the document.all object which is only defined for Internet Explorer).
-
A reference to an external CSS file which contains CSS errors.
-
A reference to an external CSS file with an incorrect MIME type.
This will cause an error due to the fact that the web page
link
s to a file containing CSS rules but which is of the wrong MIME type. In Mozilla, if a web page is in Standards mode it will not load CSS files unless they are of MIME typetext/css
. If the web page had been in Quirks mode, then the CSS file would have been loaded but a warning message about the incorrect MIME type would have been logged to the JavaScript console. -
a
style
block which contains CSS errors
Mozilla Release/Nightly Builds
When loading
example-html-page.htm
in an official Mozilla release or nightly build, Mozilla will output
the following:
Default Mozilla Output from Example Document
Location | Message |
---|---|
JavaScript Console |
Error: The stylesheet http://bclary.com/log/2004/04/10/example-css.htm was not loaded because its MIME type, "text/html", is not "text/css". Error: document.all has no properties Source File: http://bclary.com/log/2004/04/10/example-html-page.htm Line: 25 |
STDOUT |
JavaScript dump output cookie = cookie=value Document http://bclary.com/log/2004/04/10/example-html-page.htm loaded successfully |
STDERR |
Note how an official or nightly build of Mozilla will only report the JavaScript errors and CSS MIME errors. Other limitations are that the reports are sent to the JavaScript console which can not be easily saved to a file and which has a limited number of messages it can contain at any one time.
NSPR (Netscape Portable Runtime) Logs
NSPR provides the platform independent system level routines for low-level operations such as threading, input and output, timing and other low-level services used in Mozilla. NSPR also provides a logging facility which can be used to log the results of the actions of these low level operations to external text files. This ability can be very useful when testing a web site however it does have some limitations in terms of what is reported.
The environment variables NSPR_LOG_MODULES
and
NSPR_LOG_FILE
are used to control NSPR Logging.
NSPR_LOG_MODULES
contains a comma separated list of
module names and logging levels in the format
module:level
. The module name controls what messages
are written to the log while the logging level controls the amount
and severity of the messages output to the log.
If NSPR_LOG_FILE
is set, Mozilla will output the logs
to the file specified, otherwise Mozilla will write the logs to
STDERR
.
The nsHttp
, JSDiagnostics
and
cookie
modules are of particular importance to web site
developers. The other modules
are better suited for use by developers of the Mozilla browser. The special module name
all
can be used to enable all NSPR Log modules while the
special module name sync
will cause all output to the log
file to be unbuffered. Note that sync
is not useful in general
since it pads the output with special characters which can interfere with
reading the logs.
NSPR Logging Levels
Level | Meaning |
---|---|
0 | no logging output |
1 | informational messages |
2 | error messages |
3 | warning messages |
4 | debug messages |
5 | all messages |
Useful NSPR Log Module settings
Log Module setting | Meaning |
---|---|
JSDiagnostics:2 | display JavaScript errors |
nsHttp:3 | display HTTP request/response headers |
cookie:4 | display cookies accepted/rejected/sent |
Example - NSPR JavaScript Logging in Release builds
In Windows, to log all JavaScript informational, error and informational messages to c:\temp\mozilla-nspr.log, you would open a DOS Window, change directory to the location where the mozilla.exe executable is located, set the appropriate environment variables, and start Mozilla.
set NSPR_LOG_MODULES=JSDiagnostics:2 set NSPR_LOG_FILE=C:\temp\mozilla-nspr.log mozilla > c:\temp\mozilla-stdout.log 2> c:\temp\mozilla-stderr.log
Location | Message |
---|---|
NSPR Log File |
0[274098]: file http://bclary.com/log/2004/04/10/example-html-page.htm, line 25: TypeError: document.all has no properties |
Note that the JavaScript Console, STDOUT and STDERR output are omitted from the above table since they are identical to the previous output.
Example - NSPR Cookie Logging in Release builds
set NSPR_LOG_MODULES=cookie:4 set NSPR_LOG_FILE=C:\temp\mozilla-nspr.log mozilla > c:\temp\mozilla-stdout.log 2> c:\temp\mozilla-stderr.log
Location | Message |
---|---|
NSPR Log File |
0[274098]: ===== COOKIE ACCEPTED ===== 0[274098]: request URL: http://bclary.com/ 0[274098]: cookie string: cookie=value 0[274098]: current time: Mon Jun 07 03:18:49 2004 GMT 0[274098]: ---------------- 0[274098]: name: cookie 0[274098]: value: value 0[274098]: host: bclary.com 0[274098]: path: / 0[274098]: expires: at end of session 0[274098]: is secure: false 0[274098]: 0[274098]: ===== COOKIE SENT ===== 0[274098]: request URL: http://bclary.com/ 0[274098]: cookie string: cookie=value 0[274098]: current time: Mon Jun 07 03:18:49 2004 GMT 0[274098]: 0[274098]: ===== COOKIE SENT ===== 0[274098]: request URL: http://bclary.com/ 0[274098]: cookie string: cookie=value 0[274098]: current time: Mon Jun 07 03:18:49 2004 GMT 0[274098]: |
Example - NSPR HTTP Logging in Release builds
set NSPR_LOG_MODULES=nsHttp:3 set NSPR_LOG_FILE=C:\temp\mozilla-nspr.log mozilla > c:\temp\mozilla-stdout.log 2> c:\temp\mozilla-stderr.log
Location | Message |
---|---|
NSPR Log File |
0[274098]: http request [ 0[274098]: GET /log/2004/04/10/example-html-page.htm HTTP/1.1 0[274098]: Host: bclary.com 0[274098]: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040604 0[274098]: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 0[274098]: Accept-Language: en-us,en;q=0.5 0[274098]: Accept-Encoding: gzip,deflate 0[274098]: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 0[274098]: Keep-Alive: 300 0[274098]: Connection: keep-alive 0[274098]: ] 2348[9b1488]: http response [ 2348[9b1488]: HTTP/1.1 200 OK 2348[9b1488]: Date: Mon, 07 Jun 2004 03:26:17 GMT 2348[9b1488]: Server: Apache/1.3.31 (Win32) 2348[9b1488]: Keep-Alive: timeout=15, max=100 2348[9b1488]: Connection: Keep-Alive 2348[9b1488]: Transfer-Encoding: chunked 2348[9b1488]: Content-Type: text/html 2348[9b1488]: ] 0[274098]: http request [ 0[274098]: GET /log/2004/04/10/example-css.htm HTTP/1.1 0[274098]: Host: bclary.com 0[274098]: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040604 0[274098]: Accept: text/css,*/*;q=0.1 0[274098]: Accept-Language: en-us,en;q=0.5 0[274098]: Accept-Encoding: gzip,deflate 0[274098]: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 0[274098]: Keep-Alive: 300 0[274098]: Connection: keep-alive 0[274098]: Referer: http://bclary.com/log/2004/04/10/example-html-page.htm 0[274098]: ] 2348[9b1488]: http response [ 2348[9b1488]: HTTP/1.1 200 OK 2348[9b1488]: Date: Mon, 07 Jun 2004 03:26:17 GMT 2348[9b1488]: Server: Apache/1.3.31 (Win32) 2348[9b1488]: Keep-Alive: timeout=15, max=99 2348[9b1488]: Connection: Keep-Alive 2348[9b1488]: Transfer-Encoding: chunked 2348[9b1488]: Content-Type: text/html 2348[9b1488]: ] |
This ability to log both the exact HTTP requests and responses can be very useful in analysing issues related to content negotiation, content types, and server configuration.
Example - NSPR Combined Logging in Release builds
It is also possible to combine the output to STDOUT with the output to the NSPR Log file by not specifying NSPR_LOG_FILE and by redirecting the STDOUT and STDERR to the desired file.
set NSPR_LOG_FILE= set NSPR_LOG_MODULES=JSDiagnostics:2,cookie:4,nsHttp:3 mozilla > c:\temp\mozilla.log 2>&1
Location | Message |
---|---|
STDOUT |
0[274098]: http request [ 0[274098]: GET /log/2004/04/10/example-html-page.htm HTTP/1.1 0[274098]: Host: bclary.com 0[274098]: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040604 0[274098]: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 0[274098]: Accept-Language: en-us,en;q=0.5 0[274098]: Accept-Encoding: gzip,deflate 0[274098]: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 0[274098]: Keep-Alive: 300 0[274098]: Connection: keep-alive 0[274098]: ] 2196[9b1438]: http response [ 2196[9b1438]: HTTP/1.1 200 OK 2196[9b1438]: Date: Mon, 07 Jun 2004 03:46:31 GMT 2196[9b1438]: Server: Apache/1.3.31 (Win32) 2196[9b1438]: Keep-Alive: timeout=15, max=100 2196[9b1438]: Connection: Keep-Alive 2196[9b1438]: Transfer-Encoding: chunked 2196[9b1438]: Content-Type: text/html 2196[9b1438]: ] 0[274098]: http request [ 0[274098]: GET /log/2004/04/10/example-css.htm HTTP/1.1 0[274098]: Host: bclary.com 0[274098]: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040604 0[274098]: Accept: text/css,*/*;q=0.1 0[274098]: Accept-Language: en-us,en;q=0.5 0[274098]: Accept-Encoding: gzip,deflate 0[274098]: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 0[274098]: Keep-Alive: 300 0[274098]: Connection: keep-alive 0[274098]: Referer: http://bclary.com/log/2004/04/10/example-html-page.htm 0[274098]: ] 2196[9b1438]: http response [ 2196[9b1438]: HTTP/1.1 200 OK 2196[9b1438]: Date: Mon, 07 Jun 2004 03:46:31 GMT 2196[9b1438]: Server: Apache/1.3.31 (Win32) 2196[9b1438]: Keep-Alive: timeout=15, max=99 2196[9b1438]: Connection: Keep-Alive 2196[9b1438]: Transfer-Encoding: chunked 2196[9b1438]: Content-Type: text/html 2196[9b1438]: ] 0[274098]: ===== COOKIE ACCEPTED ===== 0[274098]: request URL: http://bclary.com/ 0[274098]: cookie string: cookie=value 0[274098]: current time: Mon Jun 07 03:46:31 2004 GMT 0[274098]: ---------------- 0[274098]: name: cookie 0[274098]: value: value 0[274098]: host: bclary.com 0[274098]: path: / 0[274098]: expires: at end of session 0[274098]: is secure: false 0[274098]: 0[274098]: ===== COOKIE SENT ===== 0[274098]: request URL: http://bclary.com/ 0[274098]: cookie string: cookie=value 0[274098]: current time: Mon Jun 07 03:46:31 2004 GMT 0[274098]: 0[274098]: ===== COOKIE SENT ===== 0[274098]: request URL: http://bclary.com/ 0[274098]: cookie string: cookie=value 0[274098]: current time: Mon Jun 07 03:46:31 2004 GMT 0[274098]: JavaScript dump output cookie = cookie=value 0[274098]: file http://bclary.com/log/2004/04/10/example-html-page.htm, line 25: TypeError: document.all has no properties Document http://bclary.com/log/2004/04/10/example-html-page.htm loaded successfully |
Debug Mozilla Builds
Using NSPR Logging with Mozilla releases is useful for logging JavaScript errors and warnings, JavaScript dump messages as well as Cookie processing however it does not provide any information regarding CSS MIME errors and warnings which are reported to the JavaScript console nor does it allow reporting of other CSS errors. Debug builds of Mozilla do provide this information at the cost of generating somewhat more complicated log files.
For more information on how to build Mozilla on your platform see Mozilla build documentation.
Example - Logging using a Mozilla Debug build
This example illustrates the output from a Debug Mozilla build. Note that the NSPR JSDiagnostics log is not included since Debug builds automatically report JavaScript errors. Note that you could also use the techniques outlined above to include additional information such as HTTP headers or Cookies from the NSPR logs.
mozilla > c:\temp\mozilla.log 2>&1
The output of the DEBUG log may be confusing since it includes a great deal of information which is of interest only to developers of the Mozilla browser however with a little practice, it is easy to find the important information. In the following, parts of the log which are not important for web site quality assurance are marked as ignorable.
Type Manifest File: C:\work\mozilla_source\1.7\mozilla\obj-i586-pc-msvc-debug\dist\bin\components\xpti.dat +++ JavaScript debugging hooks installed. nsNativeComponentLoader: autoregistering begins. nsNativeComponentLoader: autoregistering succeeded nNCL: registering deferred (0) nsNativeComponentLoader: autoregistering begins. nsNativeComponentLoader: autoregistering succeeded nNCL: registering deferred (0) nsNativeComponentLoader: autoregistering begins. nsNativeComponentLoader: autoregistering succeeded nNCL: registering deferred (0) WARNING: dependent window created without a parent, file c:/work/mozilla_source/1.7/mozilla/xpfe/bootstrap/nsWindowCreator.cpp, line 128 ++WEBSHELL == 1 ++DOMWINDOW == 1 For application/x-java-vm found plugin C:\Program Files\Java\jre1.5.0\bin\NPOJI610.dll WARNING: NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) failed, file c:/work/mozilla_source/1.7/mozilla/extensions/cookie/nsPermissionManager.cpp, line 635 Note: verifyreflow is disabled Note: styleverifytree is disabled Note: frameverifytree is disabled WARNING: NS_ENSURE_TRUE(globalObject) failed, file c:/work/mozilla_source/1.7/mozilla/xpfe/appshell/src/nsWebShellWindow.cpp, line 1081 WARNING: NS_ENSURE_TRUE(globalObject) failed, file c:/work/mozilla_source/1.7/mozilla/xpfe/appshell/src/nsWebShellWindow.cpp, line 1081 WARNING: NS_ENSURE_TRUE(globalObject) failed, file c:/work/mozilla_source/1.7/mozilla/xpfe/appshell/src/nsWebShellWindow.cpp, line 1081 --WEBSHELL == 0 ++WEBSHELL == 1 ++DOMWINDOW == 2 ++WEBSHELL == 2 ++DOMWINDOW == 3 ++WEBSHELL == 3 ++DOMWINDOW == 4 The stylesheet http://bclary.com/log/2004/04/10/example-css.htm was not loaded because its MIME type, text/html, is not "text/css" CSS Error (http://bclary.com/log/2004/04/10/example-css.css :1.2): Selector expected. Ruleset ignored due to bad selector. CSS Error (http://bclary.com/log/2004/04/10/example-css.css :5.2): Selector expected. Ruleset ignored due to bad selector. CSS Error (http://bclary.com/log/2004/04/10/example-css.css :6.0): Unexpected end of file while searching for closing } of invalid rule set. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :2.22): Expected color but found 'wite'. Error in parsing value for property 'color'. Declaration dropped. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :2.41): Unknown property 'background-cokor'. Declaration dropped. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :3.26): Expected color but found 'ccc'. Error in parsing value for property 'color'. Declaration dropped. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :3.54): Expected color but found '#FFFFFFF'. Error in parsing value for property 'background-color'. Declaration dropped. ###!!! ASSERTION: URIs don't match: 'uri && NS_SUCCEEDED(uri->Equals(aURI, &equals)) && equals', file c:/work/mozilla_source/1.7/mozilla/extensions/p3p/src/nsP3PService.cpp, line 148 ###!!! ASSERTION: URIs don't match: 'uri && NS_SUCCEEDED(uri->Equals(aURI, &equals)) && equals', file c:/work/mozilla_source/1.7/mozilla/extensions/p3p/src/nsP3PService.cpp, line 148 ###!!! ASSERTION: URIs don't match: 'uri && NS_SUCCEEDED(uri->Equals(aURI, &equals)) && equals', file c:/work/mozilla_source/1.7/mozilla/extensions/p3p/src/nsP3PService.cpp, line 148 JavaScript dump output cookie = cookie=value JavaScript error: http://bclary.com/log/2004/04/10/example-html-page.htm line 25: document.all has no properties Document http://bclary.com/log/2004/04/10/example-html-page.htm loaded successfully WARNING: getting z level of unregistered window, file c:/work/mozilla_source/1.7/mozilla/xpfe/appshell/src/nsWindowMediator.cpp, line 636 WARNING: getting z level of unregistered window, file c:/work/mozilla_source/1.7/mozilla/xpfe/appshell/src/nsWindowMediator.cpp, line 636 nsPluginHostImpl::Observe "quit-application" WARNING: requested removal of nonexistent window , file c:/work/mozilla_source/1.7/mozilla/embedding/components/windowwatcher/src/nsWindowWatcher.cpp, line 936 --WEBSHELL == 2 --WEBSHELL == 1 --WEBSHELL == 0 nsPluginHostImpl::Observe "xpcom-shutdown" WARNING: nsExceptionService ignoring thread destruction after shutdown, file c:/work/mozilla_source/1.7/mozilla/xpcom/base/nsExceptionService.cpp, line 191 --DOMWINDOW == 3 --DOMWINDOW == 2 --DOMWINDOW == 1 +++ JavaScript debugging hooks removed. ###!!! ASSERTION: Potential deadlock between Monitor@2ef0d8 and Lock@26fe890: 'Error', file c:/work/mozilla_source/1.7/mozilla/xpcom/threads/nsAutoLock.cpp, line 299 ###!!! ASSERTION: Potential deadlock between Monitor@2ef0d8 and Lock@26fe890: 'Error', file c:/work/mozilla_source/1.7/mozilla/xpcom/threads/nsAutoLock.cpp, line 299 ###!!! ASSERTION: Potential deadlock between Monitor@2ef0d8 and Lock@26fe890: 'Error', file c:/work/mozilla_source/1.7/mozilla/xpcom/threads/nsAutoLock.cpp, line 299 ###!!! ASSERTION: Potential deadlock between Monitor@2ef0d8 and Lock@26fe890: 'Error', file c:/work/mozilla_source/1.7/mozilla/xpcom/threads/nsAutoLock.cpp, line 299 ###!!! ASSERTION: Potential deadlock between Monitor@2ef0d8 and Lock@26fe890: 'Error', file c:/work/mozilla_source/1.7/mozilla/xpcom/threads/nsAutoLock.cpp, line 299 --DOMWINDOW == 0 nsStringStats => mAllocCount: 1 => mReallocCount: 0 => mFreeCount: 1 => mShareCount: 0 => mAdoptCount: 0 => mAdoptFreeCount: 0 nsStringStats => mAllocCount: 23672 => mReallocCount: 2110 => mFreeCount: 23671 => mShareCount: 23843 => mAdoptCount: 3332 => mAdoptFreeCount: 3331
Note that the output from the Debug build of Mozilla provided much more information such as CSS Parsing errors and CSS MIME errors. It should be reasonably easy to create a filter program which extracts only those log messages which are of interest.
Custom Mozilla Builds
While Debug builds are useful, the extra browser-developer oriented messages can be confusing if not intimidating. Another approach which removes the internal debug messages while retaining the ability to report JavaScript and CSS Errors is to create a customized Mozilla build.
A custom Mozilla build created using the following patch against Mozilla 1.7 sends CSS Parser and MIME type errors to STDERR. Note that if you build this customized version, you will need to regenerate the configure script using the stable version of autoconf and set the environment variables CSS_REPORT_PARSE_ERRORS=1, CSS_REPORT_MIME_ERRORS=1 and FORCE_PR_LOG=1 when building.
Compare the output from visiting example-html-page.htm using this customized version of Mozilla to that produced by a DEBUG build.
Example - Logging using a Mozilla Custom build
set NSPR_LOG_MODULES=JSDiagnostics:2 mozilla > c:\temp\mozilla.log 2>&1
The output of the log created using this customized version of Mozilla now contains all of the JavaScript and CSS errors without any of the developer oriented debug messages.
The stylesheet http://bclary.com/log/2004/04/10/example-css.htm was not loaded because its MIME type, text/html, is not "text/css" CSS Error (http://bclary.com/log/2004/04/10/example-css.css :1.2): Selector expected. Ruleset ignored due to bad selector. CSS Error (http://bclary.com/log/2004/04/10/example-css.css :5.2): Selector expected. Ruleset ignored due to bad selector. CSS Error (http://bclary.com/log/2004/04/10/example-css.css :6.0): Unexpected end of file while searching for closing } of invalid rule set. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :2.22): Expected color but found 'wite'. Error in parsing value for property 'color'. Declaration dropped. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :2.41): Unknown property 'background-cokor'. Declaration dropped. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :3.26): Expected color but found 'ccc'. Error in parsing value for property 'color'. Declaration dropped. CSS Error (http://bclary.com/log/2004/04/10/example-html-page.htm :3.54): Expected color but found '#FFFFFFF'. Error in parsing value for property 'background-color'. Declaration dropped. JavaScript dump output cookie = cookie=value 0[284040]: file http://bclary.com/log/2004/04/10/example-html-page.htm, line 25: TypeError: document.all has no properties Document http://bclary.com/log/2004/04/10/example-html-page.htm loaded successfully
Depending upon your needs, either a Debug or Custom version of Mozilla should provide the level of detail which you need to test your web site using Mozilla.
Automated Web Site Testing
To really make Mozilla's ability to log the result of loading web pages useful for web site testing, you need the ability to automatically spider a set of pages and generate a log for the entire set.
Please note that the older version of CSpider has been replaced with an updated version which contains several bug fixes and improvements. Please see Mozilla Spiders.
CSpider is a web application which
visits each page in a web site up to a specified depth by
loading each page in a separate window, loading and parsing each CSS
and JavaScript file, executing any JavaScript then waiting for the
page's
load
event to fire before optionally navigating
to linked pages. By setting up a standard script or batch file which
invokes Mozilla with the desired NSPR logging features, then spidering
your site using CSpider, you can automatically obtain a full log of
Mozilla's loading of your HTML, CSS and JavaScript along with any
CSS or JavaScript errors it finds.
CSpider is a web application which
CSpider's easy customizability provides a platform for adding additional
functionality for diagnosing problems with a web site.
For example, this link
will automatically spider
www.mozilla.org
one-click deep for the pages linked from the home page.
Example - Automated Logging using a Mozilla Custom build and CSpider
Mozilla can also be invoked from the command-line or batch/shell script.
set NSPR_LOG_MODULES=JSDiagnostics:2,cookie:4,nsHttp:3 mozilla "http://bclary.com/log/2004/04/10/CSpider?url=http://www.mozilla.org/&domain=&depth=1&restrict=on&timeout=120&autostart=on" > mozilla.log 2>&1
Conclusion
Mozilla makes a good addition to any Quality Assurance team's toolkit.