It seems like the document is missing some frames because at the time userOnAfterPage() is called not all the content has been loaded.

This can be solved by increasing the delay in doGrab() or adding a delay in userOnAfterPage() before retrieving the document using gSpider.mDocument.

Thank you for your help.

I added the following line in userOnAfterPage() to see what happens.

window.open(“chrome://browser/content/pageinfo/pageInfo.xul”);

Interestingly the popped out pageInfo window did not have the iframe-embedded image that I could see the in the parent spider window! So it means what we get is not what we see.

It happens on many sites including http://www.aol.com.

May be it is happening for elements inside DIV elements?

Thank for your response. Yes, you are right. It is an wrappedJSObject. But it appears still I do not get all the frames.

What my hook is trying to do is download all the images in each page the spider visits. Since this is identical to the function of firefox pageInfo, I am trying to use the flow of pageInfo.js (I cannot think of a way of directly using pageInfo.js itself).

I can see my hook is missing out some embedded flash images which pageInfo is able to capture. I did some debugging and the difference I notice is pageInfo.js gets more frames out of the window. The way pageInfo receives the window object is
====
gWindow = window.opener.gBrowser.contentWindow;
gDocument = gWindow.document;
====
The way my hook obtains them is
====
var aDocument = gSpider.mDocument;
var aWindow = gSpider.mDocument.defaultView;
if (aWindow.wrappedJSObject)
{aWindow = aWindow.wrappedJSObject;
}
====

The processes after that are identical: go through all the frames and use tree-walker.

To cite an example, if I use spider to visit http://home.live.com after signing in to hotmail, the hook says there are 2 frames and it misses out the embedded flash image that appears at the right bottom corner. If I use pageInfo, it says there are four frames and it captures the said flash.

try this:

if (aWindow.wrappedJSObject)
{
aWindow = aWindow.wrappedJSObject;
}