Start Here

The Internet Researcher is a powerful tool that can be used to

  • Download to your PC and organize webpages and/or entire websites. Content of websites changes often. You can record a website and view it later as it was at the time of recording.
  • Browse pages and websites later when Internet connection is unavailable.
  • Surf the Internet faster. Use this program instead of the standard browser if you want a faster Internet experience. Just launch the built-in browser and use it like a conventional browser. The difference starts when you want to visit several links on a page. Drag and drop the links or entire text selections containing several links to the Drop Box. By doing this you tell the browser that you wish to visit the links later. You can continue to browse to another page while the links are downloaded in the background. When you subsequently click on any of the links you previously dropped to the Drop Box, they will be loaded immediately.
  • Record every page you browse with the built-in browser.
  • Search downloaded websites and pages for a particular word, phrase, or any combination of them. The program gives you very powerful search capabilities.
  • Create an exact duplicate of a website on disk including its directory structure.
  • View directory and web structure of a website. You can easily find how pages are linked to each other and what are the relations between pages. For instance, you can find all pages that link to a particular page.
  • Relink a website to convert absolute links to relative links.

Features How to... User Interface

Click anywhere within the picture to get help about each element.

Search Results Task Structure Task Context Menu Browser Tab Create Task bar Task Toolbar Task Files Main Menu Progress Window Project Toolbar Folders Window List of Tasks Statistics Window


Contacts

The Internet Researcher is a product from Zylox, Inc.

Visit our website www.zylox.com:

  • to get latest information about this and other products developed by Zylox.
  • to get updates of Internet Researcher

We provide technical support for registered users by email. If you have questions concerning this software product, send an email to support@zylox.com. Be sure to include the version of Internet Researcher and your licence number both of which can be found in the About box available from the Help menu.


Features

  • Multithreaded webspider
  • The program resumes broken downloads for HTTP and FTP-via-HTTP connections (direct FTP resume will be added in subsequent versions).
  • HTTP and FTP-via-HTTP proxy servers are supported.
  • You can import favorites from Internet Explorer.
  • The program has a built-in ping utility to detect if Internet connection is available. The program will stop download if the Internet connection is lost and will resume automatically once it becomes available again.
  • You can define such connection options as maximum number of concurrent connections, maximum number of concurrent connections per server, and pause between connections to the same server. This prevents the servers from overloading while maintaining high speed downloads when you download from multiple servers.
  • Global filters are available in addition to task filters. For instance, you can define a global filter to exclude banners. Global filters are applied to every task in the project file.
  • You can search in downloaded files by keywords, phrases, complex boolean expressions, URLs, webpage titles, file size ranges, file types, file modification dates, and download dates.
  • The program uses standard MS Access database as a project file. As this is a very widely used database format, this gives you direct access to the project data. You can also access the database programmatically.
  • You can duplicate websites on disk including directory structure.
  • The program extensively supports drag-and-drop. You can drag and drop links, files, and HTML selections to the Tasks Window or to the Drop Box. You can also drag and drop tasks and resource files to file system or between two projects.

How this program differs from other offline browsers

  • The most distinctive feature of this program is that it can download virtually everything that you can view with a standard browser, be it Flash, JavaScript links (no matter how complex the scripts are), content available through web forms, HTTPS websites, or FTP sites. If the program failed to download a task or a number of pages of a task, open the task with the built-in browser and browse to the failed webpage. The page will be retrieved with the help of the browser and the download will continue. In subsequent versions of the program you will have an option to do it automatically.
  • Only this program provides security against viruses and malicious scripts you can download along with the webpages. When you browse the offline content with the built-in browser you can be sure that you have the same security as when you browse the Internet with the standard browser. Other offline utilities store downloaded websites on your local hard drive as linked files or run a webserver on your PC. When you browse your local hard drive the browser turns off security as it assumes you browse trusted content. This program notifies the browser that the pages you browse must be treated the same way as pages from the Internet.
  • This program gives you a true ability to download only the pages you need and nothing else. And you do not have to use filters for this purpose (though the program provides you with very powerful automated filtering tools). Do not set levels and filters on a task. Just open a page with the built-in browser and drag-and-drop the links you want to retrieve to a special topmost window called Drop Box. The program will download these links. You can also drag and drop web page text selections. Every link in the selected text fragment will be downloaded.
  • You can browse tasks with the built-in browser while the program downloads them. Retrieved pages will be loaded immediately. If you click on a link that is not yet retrieved, it will be scheduled for immediate download.
  • You can search the downloaded websites and pages for a particular word, phrase, or a combination of them. The program gives you very powerful search capabilities. This is like having a search engine to search only the offline content.
  • No matter how, when, and in what order you download webpages, and regardless of what tasks they belong to, if they link to each other you will be able to click on the link and get to the linked page even though it might have been retrieved by another task. This gives you the ability to download different parts of a site by different tasks with different parameters and still have the parts linked to each other as if they were downloaded by a single task; or the ability to have shortcuts in the Tasks List to any page of a website.


License

End-User License Agreement (EULA) for Internet Researcher

This End-User License Agreement ("EULA") is a legal agreement between Zylox, Inc. and an individual or a single entity who agrees and accepts the following terms and conditions by installing, copying, or otherwise using the software product identified above ("SOFTWARE PRODUCT").

The SOFTWARE PRODUCT is protected by copyright laws and international copyright treaties. The SOFTWARE PRODUCT is licensed, not sold. This EULA does not give you any license to the copyrights or other intellectual property rights. All rights not expressly granted by this EULA are reserved to Zylox, Inc.

DISTRIBUTION

You may distribute this program in unmodified form via electronic means or on CD compilations. You may not sell this program. You must explicitly state that this is shareware and that the product is not free.

GRANT OF LICENSE

You may install and use one (1) copy of the SOFTWARE PRODUCT on a single computer. You may also make copies of the SOFTWARE PRODUCT for backup and archival purposes.

LIMITATIONS

You may not alter, reverse engineer, decompile, or disassemble the SOFTWARE PRODUCT except and only to the extent that such activity is expressly permitted by applicable law notwithstanding this limitation.

You may not rent, lease, lend, sell, or sublicense the SOFTWARE PRODUCT.

DISCLAIMER OF WARRANTY

This SOFTWARE PRODUCT is provided as is without warranty of any kind, either express or implied, including, but not limited to, warranties of merchantability or fitness for a particular purpose. The Zylox, Inc. assumes no liability for damages, either direct or consequential, which may result from the use of the software product.

TERMINATION

Without prejudice to any other rights, Zylox, Inc. may terminate this EULA if you fail to comply with the terms and conditions of this EULA. In such event, you must destroy all copies of the SOFTWARE PRODUCT.


User Interface

  • Main Menu
  • Project Toolbar
  • Browser
  • Task Files
  • Task Structure
  • Search Results
  • Create Task Bar
  • Task List
  • Task Toolbar
  • Progress Window
  • Statistics Window
  • Folders Window
  • Program Options
  • Project Properties
  • Task Properties
  • Drop Box


Registration

The evaluation version of Internet Researcher will download only up to 500 files per project and is limited to 30 uses. If you like this program and want to remove this limitation, you should pay for Internet Researcher by registering it. Registration costs only $59.95 US.

Registration is handled by our reseller at regsoft.com. They accept Visa, Mastercard, American Express, Discover, Eurocard as well as checks and money orders.

Immediately after paying the registeration fee, you will get a registration code which will unlock the software. Unlocking the software removes all limitations.

Registered users receive free technical support and free upgrades to Internet Researcher.

Note that a single license gives you permission to install Internet Researcher on a single machine for use by a single user. A separate license should be bought for each additional user and/or machine.

Visit our website or email us for information about volume discounts and site licenses.

Register Internet Researcher


What is a Task

A task is a request to retrieve web pages or files (resources) from the Internet and make them available offline. The downloaded resources are permanently stored in the project database until you explicitly delete them. You can view the downloaded pages and browse the downloaded websites using the built-in Browser.

You can think of each task as a separate spider. The spider starts by retrieving the starting webpage of a task along with the embedded resources (graphics, scripts, stylesheets, sounds, movies, frames, applets). It then searches the page for links and retrieves each linked page and its embedded resources, then searches the page for new links, and so on. The spider remembers each link it has retrieved and never retrieves the same link again.

Note that every time you use the built-in browser you browse the entire project file, not just a single task. Resources (webpages, images, and other files downloaded from the Internet) you browse with the built-in browser remain to be linked even if they are downloaded with different tasks. When you click on a link, the browser first searches for the page in the list of files retrieved by the task you browse. If the page is not found among the files of the task the browser searches the entire project database. If there are different versions of the page associated with the URL the version that has a timestamp nearest to the browsed task is retrieved.

You can save a single task or entire project file to the file system to create a duplicate of a website on your hard-drive. Alternatively, you can export a task or entire project file to a HTML Help file (.CHM).


How to Browse the Downloaded Content When You Are Not Connected to the Internet

To browse websites in offline mode you should first [image] download them. You can browse while you download.

When you finish your downloads, it is strongly recommended that you switch the program to [image] offline mode. Even if you have configured the built-in Ping utility which automatically stops downloading when the Internet connection disapears, the manually set offline mode still prevents the program from downloading queued files without your notice when the Internet connection becomes available again.

To browse a task, first select the task and then click on the [image] Open button on the Task Toolbar (or just double-click the task). A browser will be launched with the starting page of the task. You can click on any link on the page. If the link was retrieved, it will be immediately loaded into the browser. If the link was not retrieved it will be [image] scheduled for next download. This way you can retrieve links that are not retrieved by the standard parser, such as links generated by client scripts, links from Flash sites, or pages that can be reached only by filling out web forms. You can also drag and drop links to the [image] Drop Box or directly to the Tasks Window.

Alternatively, you can click on [image] Launch Browser button on the Project Toolbar to launch a new browser window. You can then type in a URL into the Address Bar. If the page is available in the project file, it will be shown immediately, otherwise you will get a message explaining why the browser was not able to display the page.

By default, browser windows open in tabs. If you want to open browsers in separate windows (as standard Internet Explorer does) you should uncheck the "Open browsers in tabs" option in the Browser Options tab in the Options dialog box. You can anytime undock a browser or dock it to tabs again by clicking on the [image] Dock/Undock button in the upper-right corner of the browser window.

While working offline you can enqueue the links not available in the project database by either:

  • clicking on a link. You will get a page explaining that the page is queued and will be downloaded next time you go online. Click on Back to go to the previous page.
  • right-clicking on a link or a text selection containing several links and choosing Enqueue from the context menu. This way you will stay on the current page without the need to press Back button.
  • dragging and dropping links to the [image] Drop Box, the list of Tasks, to a [image] Folder, or right on the Tasks tab. New task will be created for each dropped link.

You can also drag and drop entire webpage text selections. All links in the selected text will be enqueued.


How to Clear the Entire Project File

To clear the project file, select Clear Project command from the Project menu. This will delete all files in the project database. Actually, this command deletes the old project and creates an empty one with the same name.


How to Download a Website or a List of URLs for Offline Use

The quickest method to create a task is through the Create Task bar in the Tasks tab:

  • This is a perfect method for getting just webpages. You only need to enter the address of a webpage and press Enter key (or click on the Get button). You can also drag a URL from external or the built-in browser to the Tasks window or to the Drop Box.
  • If you want to retrieve a webpage and all links from the page, you should also set the Depth of link exploration to 1. To download a webpage, all pages linked to the page, and all pages linked to all those pages, set the Depth to 2. Depth can be set as deep as 255. You can raise or lower the Depth for a task at any time. It is not recommended though to set depths to a value more than 2 without specifying a filter. When you create a task through Create Task bar the task is created with the default filter "Retrieve pages within this site only".
  • If you want to download entire website you should set the Depth to a large value (e.g. 20 will be sufficient in most but very rare cases). The program retrieves websites by levels so you can [image] stop a task when a website is retrieved down to a level you think is sufficient.
  • If you want more options (such as to specify filters or get new version of pages and websites) you should create tasks via the New Task dialog box.

See [image] New Task dialog box for more options. You can also import a list of URLs from a text file.

The tasks you create are added to the list of tasks. You can create any number of tasks with same or different parameters including tasks with the same starting URL. If the number of tasks becomes too large you have an option to organize them by folders.

If you are working in [image] Online mode, the pages and websites start downloading immediately after you create a task. If you are in [image] Offline mode the tasks will be waiting until you Go Online. You can watch the progress in the Progress window and the Statistics window. When the number of queued files shown in the Statistics window will become zero, all tasks will be finished.


How to Duplicate a Website on Disk

To create duplicates of websites on disk, you should first [image] download them. Once you have downloaded a website (or several websites), you can export either the entire project or just a single task. To export the entire project file to the disk, click on the [image] Export Project button on the Project Toolbar. To export a single task, select the task and click on the [image] Save As button on the Task Toolbar. You can also export a task by dragging and dropping a task to a file system folder.

See the Export Dialog Box for export options.


How to Find All Pages that Link to a Particular Page

Click on [image] Locate button in the built-in browser window. A search results window will appear. The pages that link to this page will be listed in the Link Explorer Window in the Referenced By tab.


How to Get New Versions of Pages and Websites

If you want to get a new version of a page or a website, you should select the "Download new version of task" in the New Task dialog box (or in the Task Properties dialog box for existing tasks).

Modifying this parameter for existing tasks will affect only new URLs generated by the task. In most cases there is no sense in changing this parameter for completed or partially completed tasks.


How to...

How to:

  • Download a Website or a List of URLs for Offline Use
  • Organize the Downloaded Websites and Webpages
  • Browse the Downloaded Content When You Are Not Connected to the Internet
  • Surf the Internet Faster
  • Record Every Page You Browse in a Project File
  • Search in the Downloaded Files
  • Duplicate a Website on Disk
  • View Directory and Web Structure of Websites
  • Find All Pages that Link to a Particular Page
  • Use Filters
  • Import Favorites
  • Use the Built-In Ping Utility
  • Clear the Entire Project File
  • Get New Versions of Pages and Websites
  • Import a List of URLs from a Text File
  • use multiple selection operations
  • Stop or Run a Task
  • Use Online and Offline Modes
  • Modify Existing Task
  • Not to Overload Websites


How to Import Favorites

To import Internet Explorer Favorites, select Import Favorites command from the Project menu.


How to Import a List of URLs from a Text File

You can import a list of URLs from a text file. Each URL should be on a separate line. Select Import Tasks from the Task menu. Specify a file name to import URLs from in the Open File dialog box. A task will be created for each URL in the imported file.


How to Modify Existing Task

To bring the Task Properties dialog box, select a task and click on the Properties button on the Task Toolbar or choose Properties from the Task Context Menu.

You can simultaneously change some properties of a set of tasks if you select multiple tasks and launch Task Properties – Multiple Selection dialog box.


Multiple Selection Operations

How to Select Multiple Items

To select multiple items, click on each item while holding the Ctrl key. To select several items in a row, click on the first and then on the last item while holding the Shift key.

Multiple Selection Operations in the Tasks Window

The following two commands perform differently in the multiple selection mode:

  • Rename: Shows Rename - Multiple Selection dialog. Deletes a user-provided substring from the names of all selected tasks.
  • Properties: Shows Properties - Multiple Selection dialog. Provides ability to change levels or set the same filters for all selected tasks.

The following commands perform the same operation on several tasks as they do on a single task.

  • Delete
  • Cut
  • Copy
  • Run Task
  • Stop Task
  • Refresh
  • Drag and Drop
  • Save As..

Multiple Selection Operations in the Files List Window

The following commands are available in the multiple selection mode:

  • Create Task
  • Refresh
  • Delete
  • Save As..


How not to Overload Websites

If you have a high-speed Internet connection you may easily overload some websites so that other users will experience a slow-down or will not be able to connect to the server at all.

You have two options to protect websites from overloading:

  • Set the number of simultaneous connections per server to 1
  • Set a pause between sequential connections to the same server to several seconds.
Both options are located in the General Options tab in the Options Dialog Box.


How to Organize the Downloaded Websites and Webpages

You can organize tasks by folders. A folder can have subfolders. Click on a folder to bring up the list of tasks in that folder

If you do not see the Folders Window, click on the [image] Folder button on the Create Task bar or choose Windows->Folders from the View Menu.

To create a new folder, right-click on a folder under which you want a new folder to be created and select New Folder from the context menu (or select a folder and press Insert key). A new name will be automatically assigned to the newly created folder. You can rename the folder if you wish. To rename a folder, click on it with the right button of the mouse and select Rename from the context menu (or just select a folder and press F2).

You can move a task to another folder using either drag-and-drop or Cut/Paste commands in the Edit Menu. You can move a folder to another location by dragging and dropping it to another folder.


How to Record Every Page You Browse in a Project File

You can launch the built-in browser without creating tasks and browse with it like you do with a conventional browser. To launch the built-in browser, click on the [image] Launch Browser button on the Project Toolbar.

The program saves all pages you view in the browser in the project database. A new task is created in the list of Tasks for each URL you manually type in the address bar of the browser. When you click on the links, the linked pages are appended to the task you browse.

You can view the list of files for each task by selecting a task and clicking on the [image] List button on the Task Toolbar (this command is also available from the Task menu and from the context menu). You can also [image] search in retrieved files.

You should explicitly delete the tasks you do not want to keep. You can also clear the entire project which will delete all files in the project.


How to Search in the Downloaded Files

Press the [image] Search button on the Project Toolbar to launch the Search Dialog Box.

You can search for files by URL, title (for webpages), file type, file state (queued, retrieved, error), or search for keywords in the text of webpages. To use the full-text search capability, you first have to create the full-text index. The first time you run a full-text search, the program asks you if you want to create the full-text index. This operation can take a while depending on how large is your project database. For small projects the index is created in a flash. If you have already created the index before, the program has to update the index if the contents of the database has changed since the last time the index was created or updated. Update of the index takes the time proportional to the number of new text files added to the project since the last time the index was updated.

You can use wildcard characters in URL, Title, and Text boxes. When searching in titles and text of webpages, you can also join the search terms with boolean operators AND, OR, NOT (not case sensitive). If operator is not specified, AND is assumed. You can use parentheses to nest expressions within a query. The expressions in parentheses are evaluated before the rest of the query. You should enclose words in quotation marks if you want to search for phrases.

Search results are listed in the Search Results Window. You can change parameters of your search and rerun the search by pressing the Search button on the Search Results Toolbar. The new results will be placed in the same window. If you run a new search by clicking the Search button on the Project Toolbar, the results of the search will be placed in a new window.


How to Stop or Run a Task

In most cases you do not have to stop individual tasks. If you want to stop all tasks in the project, you should switch the program to [image] offline mode. This is the preferred method even if you have only one task in your project.

To stop a single task without stopping other tasks, select the task and click on the [image] Stop Task button on the Task Toolbar. A stopped task will not download queued files until you run it again.

To run a task again, select the task and click on the [image] Run Task button. The task will not start to download until you switch the program to [image] online mode. If the program is in the online mode the task will begin to download immediately.


How to Surf the Internet Faster

You can use this program to surf the Internet faster. You can launch the built-in browser without creating tasks and browse with it like you do with a conventional browser. To launch the built-in browser, click on the [image] Launch Browser button on the Project Toolbar.

With a conventional browser, when you want to visit several links on a page (which happens very frequently), you have to click on the first link, wait until the page is loaded, view the page (often you wait for a page just to find that this is not what you expected to see), click on the Back button (and sometimes wait again while the browser downloads the previous page because it thinks that the page might have changed between your clicks), then click on the second link, and so on. When you find an interesting page, you stop on the page and your Internet connection becomes idle (when it could download other links you wanted to visit from the previous page).

With Internet Researcher, when you want to visit several links on a webpage, you can drag-and-drop them to the [image] Drop Box, to the [image] Tasks Window, or to a [image] Folder, by which you tell the program that you want to view them later. The program will download the links in the background while you continue to surf or view the current page.

For instance, you can type in a search phrase in a search engine (e.g. Altavista) and get many results. You can then drag and drop selected links (or a text selection containing several links) to the Tasks Window. The links will be downloaded in the background and will be available even if you disconnect from the Internet. Thus you can download hundreds or even thousands of pages for viewing them later.

When a page contains many links and you want to visit them all, you can select the text containing all those links and drag and drop the selected text to any of the drop targets mentioned above.

You can also right-click on a link or on a text selection containing several links and choose Enqueue from the context menu. This will have the same effect as if you clicked on the links except you stay on the current page and the links are downloaded in the background.

Clicking on links or choosing Enqueue from the context menu differs from dropping links to the Drop Box, to the Tasks Window, or to a Folder. Clicking on links or enqueuing them does not create new tasks. Instead the links are appended to the same task you currently browse. If you want the links to be listed in the list of tasks you have to explicitly drop them to the Tasks window, to the Drop Box, or to a folder. If the Tasks tab is not active, you can drop directly on the tab or wait a second until the Tasks tab becomes active.

If you do not want to retrieve banners you can set Project Filters to filter out advertisements. This will also allow you to surf faster.

The program saves all pages it downloads from the Internet in the project database as tasks. You should explicitly delete the tasks you do not want to keep. You can also clear the entire project which will delete all files in the project.


How to Use Filters

You can use filters to limit the addresses and types of files that will be downloaded. For instance, you can specify that the OC should:
- not download images embedded on Web pages
- download only from a particular Web site
- not download from a certain directory on a Web site
- exclude particular Web sites.

The scope of filters can be project-wide or be limited to a single task.

  • Project-wide filters are set in the Project Properties dialog box. To launch Project Properties window, click on the Properties button on the Project Toolbar.
  • Task filters are set in the Task Properties dialog box. To launch Task Properties window for an existing task, select a task and click on the Properties button on the Task Toolbar. Task Properties for a new task are set in the New Task dialog box.

Task filters are combined with current project filters using AND logical operation every time the program needs to decide whether to retrieve a given URL or not. Thus, project filters affect every task in a project. Task filters affect only the task for which they were specified.

There are four types of filters:

  • link filters are used to prevent the spider from following certain links on Web pages. The most common link filter you will use is "Retrieve only from" which is used to limit the downloads to a particular Web site. This is the default option when you download Web sites several levels deep. If you want more control over the link filters, you should use advanced pattern matching filters (see below).
  • image filters are used to exclude embedded images on all or some of the pages. The most common image filters you will use are Retrieve only from which can be used to retrieve images only from a particular Web site or a directory, and Do not retrieve which can be set if you are interested only in text of Web pages and do not want to download embedded graphics. If you want more control over the image filters, you should use advanced pattern matching filters.
  • embedded object filters are used to exclude embedded objects other than images such as sounds, Flash movies, or Java Applets.
  • dynamic object filters are project-wide filters used to exclude images, pages, and other objects dynamically generated by scripts when you browse them in the built-in browser. This filter is available only in Project Properties window and has effect only when you browse Web pages using the built-in browser.

You can also filter files by size. You can specify that the program should retrieve pages or files that are no more than a certain maximum size and/or no less than a certain minimum size. Type in the maximum size or a range of sizes in the Max Size or Min Size - Max Size text box for each type of filter. This is an optional parameter. If you do not provide the sizes, Internet Researcher will download files regardless of their sizes.

Pattern Matching Filters:

This is a logical expression that is evaluated for each URL of a task to determine whether the program should retrieve the URL or not. Logical expression can evaluate either to true or to false. If a filter applied to a URL evaluates to true, the URL is accepted and put into the queue to be downloaded. If a filter evaluates to false, a URL is not accepted and not downloaded.

The operands of the logical expression are URLs, URL patterns, and other expressions. The operators are AND, OR, and NOT (the case is not significant: operators can be written either in lowercase or uppercase). Expressions can include subexpressions enclosed in parenthesis. URL patterns can include two kinds of wildcard characters: an asterisk (*) and a question mark (?).

Character | Usage | Example
* | Matches zero or more characters. In most cases it is used as the first or last character in a URL pattern. | http//domain.com/* matches http://domain.com/, http://domain.com/index.com, http://domain.com/images/logo.jpg
? | Matches any single alphabetic character. | img??.gif matches img01.gif, img35.gif but does not match img112.gif

  • In almost every case you will use only the asterisk. And in almost every case you will use at least one asterisk per pattern.
  • In most cases you will have to type two asterisks: one at the start of a pattern, and the other at the end of a pattern.
  • If a pattern does not start with http, https, ftp, or file, the program will automatically prepend an asterisk to the pattern when matching the pattern to a URL.
  • If a pattern is not meant to match the endings of URLs (e.g. file extensions like *.html or *.jpeg), you must type an asterisk at the end of a URL pattern.

Note that a filter is not a list of URLs that must be included or excluded. This is a logical expression that is evaluated for each URL. Each URL must match the whole filter to be accepted. This is why you must use OR to join different URLs. If you join two URL patterns with AND, it will mean that, to be accepted, a URL must match both patterns. Joining two fully qualified URLs (not containing wildcard characters) with AND will have no sense at all. This is a logical expression that is evaluated for each URL to decide whether to download it or not.

You can check if a URL is accepted or rejected by a task filter on the Test Filters page. To view the Test Filters page, click on a task and select Test Filters from the Task Menu. Type in a URL, select a type of filter, and press the Test button. The page will reload showing you the result of the test (accepted or not) as well as a detailed explanation of why the URL was accepted or rejected by the filter. Every pattern in both the task filter and the project filter will be painted green or red depending on whether a pattern accepts or rejects the specified URL.

Examples for link filters:

Filter | Description
*domain-1.com* or *domain-2.com* Matches any URL that contains either domain-1.com or domain-2.com such as:
  • http://domain-1.com/
  • http://domain-2.com/
  • http://www.domain-1.com/
  • http://www.domain-2.com/about.htm
  • ftp://ftp.domain-1.com/archive.zip
  • http://go.to/cgi-bin/redirect.exe?domain-1.com
Does not match URLs that do not contain domain-1.com or domain-2.com in them.
http://domain.com/* or http://www.domain.com/* Matches any URL that begins with http://domain.com/ or with http://www.domain.com/

Does not match:

  • ftp://ftp.domain.com/archive.zip
  • http://mail.domain.com/
  • https://www.domain.com/
http://*.domain.com/* Matches http://www.domain.com/

Does not match http://domain.com/ because there is no dot before domain.com

http://*domain.com/* and (*.html or *.htm) Matches any URL from domain.com website ending with *.html or *.htm

Does not match http://domain.com/ because it does not end with *.html or *.htm

(*domain-1.com* or domain-2.com) and (*.html or *.htm) | Matches any URL from domain-1.com or domain-2.com ending with either *.html or *.htm
*domain.com* and not *domain.com/cgi-bin/* | Matches any page from domain.com but not from the cgi-bin directory on that site.
*domain.com* and not (*domain.com/dir1* and not *domain.com/dir1/dir2*)

This can also be written as:

*domain.com* and not *domain.com/dir1* or *domain.com/dir1/dir2*

Matches any page from domain.com but not the pages within dir1 directory except for pages in its subdirectory dir2, which are accepted.

Examples for image filters:

*.jpeg OR *.jpg | will download only jpeg files
not (*doubleclick.com* or *humanclick.com*) | will not retrieve images from those two ad banner serving websites.
*cool-images.com* | will download images only from the cool-images.com website.
*my-domain.com/my-photos/* | will download images only from my-photos directory on the my-domain.com website.
not * | This can be used to exclude all embedded images. It has the same effect as setting the "Do not retrieve" option. This pattern does not match any URL.


How to Use the Built-In Ping Utility

The program uses the built-in ping utility to detect if Internet connection is available. It can detect Internet availability regardless of how you connect to the Internet (dial-up or LAN) and where the problems with connection come up (either you lose connection with your ISP, or the ISP has problems, or the server you are downloading from is rebooted, etc.)

Internet Connection Detector works by trying to reach the hosts supplied by you in the Ping Settings Window. If the program can reach any of the hosts, the Internet is considered to be available. When the Internet connection is available the indicator in the Statistics Window turns green, otherwise it turns red. If you have not supplied the hosts the indicator is grayed and the program assumes the Internet connection is available.

You can supply the hosts to ping in the Ping Settings Tab in the Options Dialog Box (Main Menu -> Tools -> Options). If you download from different servers, supply one or more reliable hosts such as yahoo.com or microsoft.com. If you add the servers to the ping hosts list, the program will assume that the Internet connection is unavailable if both servers could not be reached. If you download only from one server, a better alternative would be to ping the host you download from.


How to View Directory and Web Structure of Websites

Click on the [image] Structure button on the Task Toolbar (or select Website Structure from the Task context menu) to launch a structured view of a task.

You can view a task either as directory structured or as a sitemap view. Sitemap displays the starting page of a task and all pages linked to the page as a tree.


How to Use Online and Offline Modes

When the program is in offline mode, it does not download files from the Internet, even if there are queued files or you browse with the built-in browser.

The mode the program is in is displayed in the title bar of the application.

If the program is online, you can switch it to offline mode by clicking on the [image] Go Offline button on the Project Toolbar. To switch the program back to online mode, click on the [image] Go Online button on the Project Toolbar.


Custom Parsers

The built-in parser does not process Java Scripts. If you want to download a large website with Java Script links to the information that interests you, you should write a custom parser. Custom parser is a simple plug-in that given a HTML page tells the program what additional links it should retrieve.


Browser

  • Messages generated by the browser
  • Browser Toolbar
  • Browser Menu
  • Browser Options Dialog Box

The [image] Refresh button in the browser refreshes only pages downloaded with error.

The [image] Abort button does not abort the downloads. When you press the Abort button, the browser stops waiting for a page to be downloaded. The page will be downloaded in the background.

If you enter in the Address bar a URL that is not available in the project database, a new task will be created in the current folder with the starting page set to the entered URL. If you click on a link that is not available in the project database, the link will be appended to the task you browse.

You can point a mouse cursor on a link to view whether it was retrieved or not in the status bar of the browser. There are also other options to view this status information such as showing tips in a small window, highlighting the links, or changing the cursor. You can set one or more of them in the Browser Options Dialog Box.

Sometimes instead of displaying a webpage the built-in browser generates a message explaining why it was not able to display the webpage. See Messages generated by the browser for more information.


Drop Box

Click on the Drop Box button on the Project Toolbar to launch the Drop Box.

The Drop Box is a small topmost window that accepts links or webpage text selections dropped from external or built-in browsers. A task will be created in a currently selected folder for each URL you drop to the Drop Box.

If you are running Windows 2000 or Windows Me, you will be able to make the Drop Box window transparent. Right-click the Drop Box and choose the level of transparency you are most comfortable with.


Export Project Dialog Box

  • Add .html extension to HTML files: Some web pages retrieved from the Internet may not have the extension associated with them. Check this option on if you want the program to append the extension to all saved web pages.
  • Preserve last modified date: Saves the files with the last modified date as was reported by the Web server at the time of download.
  • Truncate file names longer than N characters: Some URLs can be very long. As a result, some files may not be saved because files cannot be more than 256 characters long. Specify maximum file name length to truncate the URLs to.
  • Not retrieved links should:
    • point to the Internet: clicking on not retrieved links on the exported pages will take the user to the Web
    • display a message: "This link was not retrieved": clicking on not retrieved links on the exported pages will display a message that the page pointed to by this link is not available.
  • Save to: Type in or browse for a folder the tasks should be saved to. If you dropped tasks to a file system folder using the mouse, this box already contains the folder.


Files Window

The Files window lists files from the project database. It is used to show:

  • Files retrieved by a task
  • Files found through a project-wide search
  • Files and links listed in the Link Explorer window


Main Menu

  • Project Menu
  • Edit Menu
  • View Menu
  • Tools Menu
  • Task Menu
  • Window Menu
  • Help Menu


Options Dialog Box

Authorization Confirmations Browser Ping Proxy General

  • General
  • Proxy
  • Ping
  • Browser
  • Confirmations
  • Authorization


Project Properties Dialog Box

Project Properties

General

  • Project path: fully qualified path to the project file on disk.
  • Size on disk: the size the project file occupies on disk.
  • Protected with password: tells if the project file is protected by a password. To set the password, choose 'Set Project Password' from the Tools menu.
  • Text index: tells whether the project has a text index. Text index lets you make full-text searches in the project database and is created the first time you use the full-text search feature. The index usually occupies a considerable ammout of space in a project file. To delete the index, select 'Remove Index' from the Tools menu and then choose 'Compact Database' to actually see any reduction in the project file size.
  • Do not queue new resources from the built-in browser while in offline mode: check this box if you do not want the built-in browser to queue for download every file it does not find in the project database. This option is per project and is applied to the currently opened project only.

Filters

Go to How to use filters topic for information on how to use project filters.

Project Notes

You can type in any text you want to be stored along with the project file. These notes are for your reference only.


Project Toolbar

  • New Project: Opens a new project. Prompts you for the file name of the new project.
  • Open Project: Open an existing project. Prompts you for the file name of the project.
  • / [image] Go Online/Go Offline: Switches the program to offline mode or back to online mode.
  • Export Project: Exports the entire project database to file system or to a HTML Help file.
  • Report: Displays a report for the project.
  • Properties: Shows Project Properties dialog box.
  • Search: Launches the Search dialog box.
  • Launch Browser: Launches an empty Browser Window.
  • Drop Box: Shows the Drop Box.


Search Results Tab

  • Context Menu
  • Toolbar
  • Description of Columns
  • Link Explorer Window
  • Search Dialog Box


Task Files Window

The Task Files window lists the files retrieved by a task.

  • Context Menu
  • Toolbar
  • Description of Columns
  • Link Explorer Window


Website Structure Window

Website Structure Window displays sitemap and directory structure of a task. Directory structure displays the physical location of the files stored on a web server. Sitemap is a logical view of the website and displays how pages are linked to each other.


Browser Menu

  • File->Save As...: saves the current page as a file
  • File->Print: prints the current page
  • Edit->Find (on this page): searches for text on the current page
  • View->Stop: Stops the current page from loading. This does not abort the download. The page will continue to download in the background.
  • View->Refresh: Refreshes the contents of the current page. Only pages retrieved with error are refreshed. To get a fresh copy of a page, locate the page in the task files list and and refresh it there.
  • View->Source: displays the source (HTML) for the current page
  • View->Browser Options: displays the Browser Options Dialog Box.
  • Window->Select as Default: sets this browser window as the default window. Tasks and files are opened in the default browser unless you choose to open them in a new window.


Browser Toolbar

  • Back: Goes to the next page.
  • Forward: Goes to the previous page.
  • Up: Moves up through a List of Tasks or a List of Files
  • Down: Moves down through a List of Tasks or a List of Files.
  • Stop: Stops the current page from loading. This does not abort the download. The page will continue to download in the background.
  • Refresh: Refreshes the contents of the current page. Only pages retrieved with error are refreshed. To get a fresh copy of a page, locate the page in the task files list and and refresh it there.
  • Create task: Creates a task for the page currently loaded in the browser.
  • Locate: Locates a file in the project database corresponding to the page currently loaded in the built-in browser.


Messages Generated by the Built-In Browser

Sometimes instead of displaying a webpage the built-in browser generates a message explaining why it was not able to display the page.

Here is a list of messages you can encounter while browsing:

  • Invalid URL: Please verify that specified URL exists and is correct.
  • Too many redirects: The URL was redirected too many times.
  • Internet connection is not available: The page you are looking for cannot be displayed now because Internet connection is not available.
  • The page is queued: The page you are looking for is not yet retrieved. The page is queued and belongs to a task that is not yet finished. The page was queued either by the task, by a script on the previous page, or because you have clicked on a link that was not found in the project database.
  • The page was not accepted: The page you are looking for was not accepted by project filters.
  • The server returned an empty page: The file returned by the server contains no data. This might have been a temporary problem.
  • The page is not available offline: The page you are looking for was not found in the project database and you specifically requested not to queue additional pages.


Invalid URL

Please verify that specified URL exists and is correct.


Too many redirects

The URL was redirected too many times.


Internet connection is not available

The page you are looking for cannot be displayed now because Internet connection is not available.

Please try the following:

  • Click the Refresh button, or try again later.
  • Check your Ping Settings.


The page is queued

The page you are looking for is not yet retrieved. The page is queued and belongs to a task that is not yet finished. The page was queued either by the task, by a script on the previous page, or because you have clicked on a link that was not found in the project database.

Please try the following:

  • To download all queued resources, connect to the Internet and switch the program to Online mode.
  • If you do not want to download the page, you can Remove it from the queue.


The page was not accepted

The page you are looking for was not accepted by project filters.

Please try the following:

  • Change your project filters and click the Refresh button.


The server returned an empty page

The file returned by the server contains no data. This might have been a temporary problem.

Please try the following:

  • Click the Refresh button, or try again later.


The page is not available offline

The page you are looking for was not found in the project database and you specifically requested not to queue additional pages.

Please try the following:

  • Turn on the Enqueue from the browser mode in the project properties window ànd click Refresh button to load the page


Edit Menu

Help Window Task Tools View Project

  • Search: Launches the Search dialog box.
  • Cut: Removes the selected items and copies them onto the clipboard.
  • Copy: Copies the selected items onto the clipboard. To put them in the new location, use the Paste command.
  • Delete: Deletes the selected items.
  • Paste: Inserts the items you have copied or cut into the selected location.


Help Menu

Window Task Tools View Edit Project

  • Getting Started: Loads the Getting Started tutorial in the built-in browser.
  • Contents and Index: Displays help topics.
  • Visit our website: Launches a browser and navigates it to our website.
  • Register...: Shows the Register dialog window.


Project Menu

Help Window Task Tools View Edit

  • New Project: Opens a new project. Prompts you for the file name of the new project.
  • Open Project: Open an existing project. Prompts you for the file name of the project.
  • Reopen: Opens an existing project. Select from a list of most recently used projects.
  • Close Project: Closes the currently open project.
  • Refresh Project: Refreshes the project. Prompts you for refresh options.
  • Clear Project: Clears the currentely open project. All data is deleted.
  • Export Project...: Exports the entire project database to file system or to a HTML Help file.
  • Import Favorites: Imports favorites from Internet Explorer.
  • Report: Displays a report for the project.
  • Properties: Shows Project Properties dialog box.
  • / [image] Go Online/Go Offline: Switches the program to offline mode or back to online mode.
  • Exit: Closes the program.


Task Menu

Help Window Tools View Edit Project

  • Open: Opens selected task in the current browser window.
  • Open in New Window: Opens selected task in a new browser window.
  • List Files: Lists the files retrieved by the selected task.
  • Website Structure: Displays logical and physical structure of website.
  • View Source: Displays the source (HTML) for the main page of the selected task.
  • Run Task: Runs a stopped task.
  • Stop Task: Stops a queued task. To run it again, use the Run Task command.
  • Refresh: Refreshes a task. Deletes all resources retrieved by the task.
  • New Task: Launches the New Task dialog.
  • Import URLs: Imports a text file with the list of tasks to create.
  • Sort By: Sorts the list of tasks by any field listed in the submenu.
  • Views: Sets a display mode.
  • Legend: Displays the Legend window explaining the meaning of state icons.
  • Cut: Removes the selected tasks and copies them to the clipboard.
  • Copy: Copies the selected tasks to the clipboard. To put them in the new location, use the Paste command.
  • Paste: Inserts the tasks you have copied or cut into the selected folder.
  • Delete: Deletes the selected tasks.
  • Rename: Renames the selected task. You can also select multiple tasks for this opertion.
  • Save As...: Saves the selected tasks to the disk as either linked files or HTML Help files.
  • Report: Displays a report for a selected task.
  • Properties: Edits the current selection's properties.


Tools Menu

Help Window Task View Edit Project

  • Compact Database: Compacts a project database. Prompts you for the name of the project to compact.
  • Repair Database: Repairs a damaged project database. Prompts you for the name of the project to repair.
  • Create Text Index : Creates a text index. Text index lets you make full text searches in the project database. The index is created automatically the first time you use the Search feature.
  • Delete Text Index : Deletes the text index from the project file. The text index usually doubles the size of the project file. You may delete the index to free up the space. After removing the index you should compact the database to actually see any reduction in the project file size.
  • Dial a Connection : Displays a dialog with dialing options.
  • Disconnect : Disconnects the dial-up connection.
  • Set Project Password : Sets a password for the project. The program will prompt you for the password every time you open the project.
  • Remove Password : Removes the password from the project. The project will no longer be protected by the password.
  • Options: Shows the Options dialog box.
  • Execute SQL : Executes an arbitrary SQL. Do not use this feature.


View Menu

Help Window Task Tools Edit Project

  • Windows: Show/hide selected windows.
  • Toolbars: Show/hide selected toolbars.
  • Views: Select from the four available views.
  • Docking Defaults 1 (Easy): Displays windows and toolbars in the same default order as when the program was installed.
  • Docking Defaults 2 (Pro): Reorders windows and toolbars for advanced users.


Windows Menu

Help Task Tools View Edit Project

  • Launch Browser: Launches a new browser window.
  • Close Active Page: Closes currently active page.
  • Undock Active Page: Undocks currently active page. The page becomes a separate window visible in the system Task Bar. To dock it back again, press the Dock button on the undocked window.
  • Opened Browsers: Lists currently opened browsers.
  • Task Files:
  • Search Results:
  • Web Structures:


Autorization

  • Display logon dialog box: uncheck this box if you want authorization requests to be silently ignored (in this case logon dilog box will not be displayed on password protected Web sites).
  • Do not reauthorize on the same realm if authorization was canceled: if this box is checked and you pressed Cancel button on the logon window, all subsequent authorization requests from the same Web site will be ignored.


Settings for the Built-In Browser

Home page

Specify the home page for the integrated browser. To specify a page located on your disk, type file:// in front of the fully qualified path (e.g. file://c:\anyfolder\home.html).

Placing the cursor on a link that was not retrieved

When you place the cursor on a link, the program displays a hint in the status bar telling you whether the link was retrieved or not. You can choose to have this information near to the cursor. There are three options.

You can turn all three on if you wish.

Script Behavior

  • Allow scripts to open new windows: uncheck if you do not want to view popup windows (e.g. advertisments) opened when you visit some websites.
  • Allow scripts to open new windows on close: there are websites that open themselves in another window when you try to close them. Uncheck this box only if you have a good reason for doing this.
  • Allow scripts to undock, resize, and move browser windows: If a page tries to resize and position itself on a screen, and you have chosen to open browser windows in tabs, you still have an option to view them like they were supposed to be viewed.
  • Hide browser toolbars and menus when pages request it: default is on.

Open browser windows in tabs:

Uncheck this option if you want to open browsers in Windows taskbar. By default, browser windows are opened in tabs inside the main window.


Confirmations

You can turn on/off specific popup messages generated by the program. If a box is checked the popup message is displayed. Otherwise it is not displayed and the default answer specific to each popup message is assumed.

  • Enqueue URL in offline mode: This message is displayed every time you click on a not retrieved link. This may become especially annoying if an advertisment embedded on a Web page automatically refreshes.
    • Auto answer with: You can provide a default answer. Yes - queue the links. No - do not queue the links.
  • Enqueue links from text selection: This message is displayed when you select a text on a Web page and choose "Enqueue all links in selected text". If you uncheck this box, the default answer will be Yes.
  • Create tasks from links in a text selection: The same as above but for the command 'Create tasks for all links in a selected text'
  • Create task for opened page: This message is displayed when you click on the Create Task button on the browser toolbar or select a similar command from the browser context menu (available by right-clicking on a Web page). Default answer is Yes.
  • Create duplicate task: This message is displayed when you try to create a task from a browser and the task with such starting address already exists. The default answer is Yes.
  • Delete task: This message is displayed when you try to delete a task from the task list. The default answer is Yes.
  • Delete task item: This message is displayed when you try to delete an item from the list of task files (available by clicking on the List button on the Task toolbar). The default answer is Yes.
  • Delete folder: This message is displayed when you try to delete a folder. The default answer is Yes.
  • Abort connection: This message is displayed when you select a file in the progress window and press Del key. The default answer is Yes. This message is not displayed when you right-click the file in a progress window and choose the Abort command from the context menu.


General Options

  • Create a new task for every link clicked in the built-in browser: If this option is off, when you browse a task with the built-in browser the program saves new pages retrieved by the browser in the task associated with the browser. Check this option if you want new pages retrieved by the browser to be saved as separate tasks.

  • Attach linked resources retrieved by other tasks to the task loaded in the built-in browser: When you browse a task in the built-in browser and click on a link that was not retrieved by this task but was retrieved by another task in the same project file, the linked page will not be attached to the browsed task. Check this box if you want to create a task (usually for export purposes) with a specific set of linked pages. With this option turned on, every link you click in the built-in browser will be added to the task.

Connection Options:

  • Max connections: maximum number of concurrent Internet connections. Increasing this value may boost your throughput several times.
  • Max conneections per server: maximum number of concurrent Internet connections per server. Increasing this value may speed up or slow down the download process. Internet Standards prohibit setting this value higher than 2 connections per server. If the number of simultaneous connections does not increase after setting Max Connections to a higher value, increasing this value may be the only way to speed up the download.
  • Pause between connections:Pause between successive connections to same server in seconds. Set this to several seconds to protect webservers from overloading.
  • Max retries: maximum number of retries if a connection fails.
  • Timeout: number of seconds to wait before a timeout occurs.

Startup Options:

  • Open new project: opens new project at startup.
  • Launch browser: automatically launches a browser window at startup.
  • Go online: automatically switches the program to online mode at startup.

  • Stop if free disk space drops below: set the limit in kilobytes.


Ping Settings for Internet Connection Detector

Go to How to use the built-in ping utility topic for information on how to use the ping utility.


Proxy Settings

  • Do not use proxy: set this option if you have a direct Internet connection and do not want to connect through a proxy server.
  • Use IE proxy settings: set this option to use the same proxy settings as set in your Internet Explorer browser. This is a default setting.
  • Use proxy: specify a proxy server different from the one used by Internet Explorer. If the proxy requires authentication you should also provide your username and password.


Search Dialog Window

Search Dialog Window is used to search for files in the entire project database. A result of a search is displayed in the Search Results Tab. There may be as many Search Result Tabs open at the same time as the number of searches you have made. If you run a new search from a Search Results Tab, the tab will be reused for the new results.

General Search Options

  • URL or mask: type in a URL mask to search for. For instance, *yahoo.com* will find URLs such as http://yahoo.com/, http://www.yahoo.com/, and http://go.to/redirect?yahoo.com. If you do not provide the URL the entire database will be searched. See description of all available wildcard characters.
  • Title or mask: will limit the search to only webpages with titles matching the expression you specified. You can use a single keyword or several keywords. You can also join the keywords with AND, OR, or NOT, use parenthesis, and use quotation marks to search for phrases. Each keyword, including the keywords in phrases, can include wildcard characters. If you do not join keywords with operators AND will be assumed.
  • File type: only files of the indicated file type will be searched. Type of a file is determined at the time of retrieval based on its binary format regardless of filename extension. So you may get files with gif extension that are actually HTML files (these are primarily 404 - Not Found HTTP errors).
  • State: only the files having indicated states will be searched. For instance, you may check the Error box to get a list of all URLs retrieved with errors for the entire project database.
  • Containing text: Only the pages containing specified keywords will be returned. You can join the keywords with AND, OR, or NOT, use parenthesis, and use quotation marks to search for phrases. Each keyword, including the keywords in phrases, can include wildcard characters. If you do not join keywords with operators AND will be assumed. You can use the full text search only if you have built a full-text index. If the index is unavailable or the project database has changed since the last time you built the index, the program will prompt you to create or rebuild the index which can take a while depending on how large is the database and how many text and html files are there. This will alow your search to run faster.
  • Add wildcards automatically: automatically encloses URLs you type in the URL box in asterisks. You will have to uncheck this box if you want to search for URLs that end with the string you typed in the URL box.

Advanced Search Options

  • File Modification Date: you can search for files modified within a specified date range.
  • File Download Date: you can provide a date and time range to find files downloaded within this time range only.
  • Max Size or MinSize-MaxSize: you can limit your search to only files of no more than a certain size. You can also provide a size range.
  • Max Results: you can set the maximum number of returned files in case the result set will be too large.


Search Results Toolbar

  • Open: opens selected file in the current browser window
  • Save As: saves selected files in a file system folder
  • Properties: displays selected file properties
  • Refresh: refreshes selected files
  • Delete: deletes selected files
  • Create Task: creates tasks on selected files in the currently open folder
  • New Search: shows the Search Dialog window with the parameters of the current search. You can modify the parameters or run the search with the same parameters to refresh the search results.
  • Link Explorer: turns the Link Explorer window on or off


Wildcard Characters

The asterisk (*), question mark (?), number sign (#), exclamation point (!), hyphen (-), and brackets ([ ]) are wildcard characters. You can use these characters in queries and expressions to include all records, file names, or other items that begin with specific characters or match a certain pattern.
Symbol | Usage | Example
* | Matches any number of characters, and can be used anywhere in the character string | wh* finds what, white, and why *at finds cat, bat, and what
? | Matches any single character | b?ll finds ball, bell, and bill
# | Matches any single digit | 1#3 finds 103, 113, 123
[ ] | Matches any single character within the brackets | b[ae]ll finds ball and bell but not bill
! | Matches any character not in the list | b[!ae]ll finds bill and bull but not bell or ball
- | Matches any one of a range of characters | b[a-c]d finds bad, bbd, and bcd

Note The wildcard characters * (asterisk), ? (question mark), # (number sign), and [ (opening bracket) can match themselves only if enclosed in brackets.

The following table shows how you can use wildcard characters to test expressions for different patterns.
Kind of match | Pattern | Match (returns True) | No match (returns False)
Multiple characters | a*a | aa, aBa, aBBBa | aBC
 | *ab* | abc, AABB, Xab | aZb, bac
Special character | a[*]a | a*a | aaa
Multiple characters | ab* | abcdefg, abc | cab, aab
Single character | a?a | aaa, a3a, aBa | aBBBa
Single digit | a#a | a0a, a1a, a2a | aaa, a10a
Range of characters | [a-z] | f, p, j | 2, &
Outside a range | [!a-z] | 9, &, % | b, a
Not a digit | [!0-9] | A, a, &, ~ | 0, 1, 9
Combined | a[!b-m]# | An9, az0, a99 | abc, aj0


Description of File List Columns

  • Title: title for HTML pages or file name for other files
  • URL: URL (Web address) of a file
  • Level: the lowest level a file was found on (this column is displayed only for task files)
  • Size: size of a file
  • Content Type: content type of a file as returned by a server
  • Modified: the date and time a file was last modified as returned by a server
  • Retrieved: the date and time a file was retrieved by the program
  • Error Description: error description if a file was not retrieved or was retrieved with error

You can sort by any column by right-clicking on the column.


File Context Menu

Click with the right button of the mouse to view the context menu.

  • Open: opens selected file in the current browser window
  • Open in New Window: opens selected file in a new browser window
  • View Source: opens selected file in the Windows Notepad
  • Create Task: creates tasks on selected files in the currently open folder
  • Refresh: refreshes selected files
  • Delete: deletes selected files
  • Save As: saves selected files in a file system folder
  • Views: displays a window with description of state icons
  • Sort by: displays a window with description of state icons
  • Legend: displays a window with description of state icons
  • Test filter: displays a window with description of state icons
  • Properties: displays selected file properties


Link Explorer Window

  • Contains: shows the files embedded in the selected HTML page. Embedded files are the files that are loaded by the browser along with the page. Embedded files include graphic files, objects (such as Flash or ActiveX), sounds, videos, style sheets, java scripts, java applets, and frames.
  • Links to: shows the files the selected page links to. Any type of file can be linked to HTML page but primarily these are HTML pages. Linked files are not loaded by the browser along with the page.
  • Referenced by: shows the pages that reference the selected file. These are HTML pages that contain or link to the selected file. Only the retrieved pages are shown in this list because the program has no way to determine what pages link to a particular file until it downloads them.

The Link Explorer shows all links on a page including not retrieved links. Do not confuse not retrieved links with links retrieved with error. Not retrieved links are marked with n/a icon. Links retrieved with error are marked with the red cross icon. If an embedded link (listed in the Contains tab) was not retrieved it may be only because the link was not accepted either by the task filter or by the project filter. If a link listed in the Links To tab was not retrieved the reason may be either the filters did not accept it or the link was not accepted by the depth of exploration.

The Links Explorer window has the same context menu as the File List window.


Task Files Toolbar

  • Open: opens selected file in the current browser window
  • Save As: saves selected files in a file system folder
  • Properties: displays selected file properties
  • Refresh: refreshes selected files
  • Delete: deletes selected files
  • Create Task: creates tasks on selected files in the currently open folder
  • Link Explorer: turns the Link Explorer window on or off

You can filter files by state and file type. For instance, you can set state to Error to view only the files retrieved with error, or set file type to HTML to view only HTML pages.

  • State: select a state from the drop down list
  • Type: select a type from the drop down list


Folders Window

If you do not see the Folders Window, click on the [image] Folder button on the Create Tasks Toolbar or choose Windows->Folders from the View Menu. The Folders Window is docked by default to the Tasks Tab. You can redock Folders to outside of the tabs so you could drag links and files to folders from other tabs more easily. You can also hide the Folders window completely.

Context Menu

  • New Folder: Creates a new folder under the specified parent folder.
  • Rename: Renames a folder. Duplicate names are allowed.
  • Expand All: Expands all nodes of the tree.
  • Cut: Expands all nodes of the tree.
  • Copy: Expands all nodes of the tree.
  • Delete: Deletes a folder.
  • Paste: Expands all nodes of the tree.
  • Properties: Shows the Folder Properties dialog box.
  • Help: Shows this help page.

Keyboard Shortcuts

  • Ins: Creates a new folder under the specified parent folder.
  • Del: Deletes a folder.
  • F2: Renames a folder. Duplicate names are allowed.

Folder Properties Dialog Box

  • Name: A folder's name.
  • Location: The name of the perent folder.
  • Size: The size of all tasks in the folder and its subfolders.
  • Contains: The number of subfolders and tasks in this folder and its subfolders.


Progress Window

  • Connections: Maximum number of concurrent Internet connections
  • Per Server: Maximum number of concurrent Internet connections per server
  • More Options: Shows Options dialog box with more connection options.
  • Proxy Settings: Shows Proxy Settings dialog box.

Meaning of Columns:

  • N: connection number
  • URL: location of a file being retrieved
  • Bytes Received: number of bytes received by this time
  • Size: total size of a file (if returned by a server)
  • Type: type of file as returned by a server
  • Time Elapsed: time elapsed since a connection was established

You can abort any connection by selecting it and hitting Delete key or choosing Abort from the context menu. You can create a task on any file listed in the active connections by dragging and dropping the file to the Tasks Window or to a folder.

Maximum number of concurrent Internet connections.

If the Average Speed in the Statistics Window is less than the maximum speed of your Internet connection then increasing this value might speed up the download process.

If you experience a slow download, this may be due to several factors. Either of the following may limit the throughput:

  • You have a slow Internet connection.
  • The server is slow or has a slow Internet connection.
  • The server is overloaded.

Maximum number of concurrent Internet connections per server.

Increasing this value may speed up or slow down the download process. Internet Standards prohibit setting this value higher than 2 connections per server. If the number of simultaneous connections does not increase after setting Max Connections to a higher value, increasing this value may be the only way to speed up the download.


Statistics Window

  • Time Remaining - Estimated time required to finish the download process.
  • Queued - Number of files in the queue (waiting to be downloaded).
  • Retrieved - Number of files retrieved during the session.
  • Elapsed - Time elapsed since the start of the session.
  • Average Speed - Current average speed of the download.
  • Bytes Retrieved - Bytes retrieved during the session.
  • Ping - Internet Connection Detector. Indicates availability of the Internet connection. Click on the indicator to bring up the Ping Dialog Box.

Right click the statistics window to bring up the context menu.


Tasks Window

  • Task Toolbar
  • Create Task bar
  • Task Menu
  • Description of Columns

See Also:

  • What is Task
  • New Task (Task Properties) Dialog Box
  • Task Properties - Multiple Selection Dialog Box.
  • Rename Multiple Tasks Dialog Box.
  • Save Task As... Dialog Box
How To:
  • Multiple selection operations in the Tasks window.


Create Task bar

Use this toolbar instead of the [image] New Task dialog box to create a task when you do not need all of the options provided by the New Task dialog.

  • Show/Hide Folders Window: toggle Folders Window on/off.
  • Address:Type in the starting address of the task you want to create.
  • Depth:Specify the number of levels to download.
  • Get/Change:Click on the button to create a new task or change existing task respectively. Pressing Enter key on the keyboard is equivalent to clicking on this button.

When you create tasks via Create Task bar and set the Depth to anything other than 0 the program automatically provides a default filter "Retrieve pages within this site only". You can change the filter (as well as any other parameters of a task) at any time via Task Properties dialog box. To view Task Properties, select the task and click on Properties button on the Task Toolbar.

If you want to retrieve a webpage and all links from the page, you should also set the Depth of link exploration to 1. To download a webpage, all pages linked to the page, and all pages linked to all those pages, set the Depth to 2. Depth can be set as deep as 255. You can raise or lower the Depth for a task at any time. It is not recommended though to set depths to a value more than 2 without specifying a filter. When you create a task through Create Task bar the task is created with the default filter "Retrieve pages within this site only".

  • Go to How to download... help topic for more information on creating tasks.
  • Go to How to use filters help topic for information on how to specify filters.


Task Properties – Multiple Selection

Task Properties – Multiple Selection Dialog Box

You can apply changes to multiple tasks at once. Select several tasks and click on the Properties button on the Task Toolbar. You can set Depth and filters. When you press Ok, all tasks will be assigned new filters and depth.


Rename Multiple Tasks Dialog Box

Though you can assign a name to a task, you do not have to. If you have not supplied a name, task is named automatically with the title of the main page of the task. If you have downloaded many pages from the same website, the titles of the pages may contain the same substring such as the name of a website.

You can delete a substring from the names of several tasks with a single operation. Type in or cut and copy a substring to the provided text box or leave the suggested substring in the box and press Ok.


Save Task As... Dialog Box

You can save a task to the file system to create a browsable copy of a task on disk as linked files.

Select one or more tasks and press the Save Task As.. button or drag and drop the tasks to a folder where you want the tasks to be saved to.

  • Add .html extension to HTML files: Some web pages retrieved from the Internet may not have the extension associated with them. Check this option on if you want the program to append the extension to all saved web pages.
  • Preserve last modified date: Saves the files with the last modified date as was reported by the Web server at the time of download.
  • Truncate file names longer than N characters: Some URLs can be very long. As a result, some files may not be saved because files cannot be more than 256 characters long. Specify maximum file name length to truncate the URLs to.
  • Not retrieved links should:
    • point to the Internet: clicking on not retrieved links on the exported pages will take the user to the Web
    • display a message: "This link was not retrieved": clicking on not retrieved links on the exported pages will display a message that the page pointed to by this link is not available.
  • Save to: Type in or browse for a folder the tasks should be saved to. If you dropped tasks to a file system folder using the mouse, this box already contains the folder.


Task Columns

Task List

This window lists tasks in a selected folder. Most common operations with tasks have buttons on the Task Toolbar. All possible commands are listed in the Task Menu.

  • State: task can be in one of the following states:
    - Task is retrieved with error. You can try to refresh the task by pressing Refresh button.
    - Task is retrieved.
    - Task is retrieving.
    - Task is stopped. You can resume the task by choosing Run Task from the context menu.
    - Task is queued.

Meaning of columns:

  • Name: name assigned to a task. If the name is not specified, it is automatically set up to the title of the first webpage of a task when the page is retrieved.
  • Task Notes (paper clip icon): displays a paper clip for every task in the list of tasks that has non empty task notes.
  • Url: the starting address of the task.
  • Queued: the number of queued (not yet retrieved) files in a task. To view all the files (including retrieved ones) click on List button on the Task toolbar.
  • Retrieved: the number of retrieved files in a task. To view all the files (including queued ones) click on List button on the Task toolbar.
  • Errors: the number of files retrieved with error (e.g. Not Found, or connection errors). Some errors are permanent, some are transient. To retry the erroneously retrieved files, click on the Refresh button on the Task Toobar.
  • Size: size of a task. This is the sum of sizes of all retrieved files of a task. The sum of sizes of all tasks may be higher than the size of the project file because several tasks may contain the same resource.
  • Depth: the number of levels to download beginning from the starting address. 0 means to retrieve only the starting webpage. 1 means to retrieve the starting webpage plus every link on the page. In most cases you should not set this value higher than 1 without specifying a filter.
  • Created: the time a task was created.
  • Error Description: Description of an error if task was not retrieved.

See Also:

  • What is Task
  • New Task (Task Properties) Dialog Box
  • Task Properties - Multiple Selection Dialog Box.
  • Rename Multiple Tasks Dialog Box.
  • Save Task As... Dialog Box
How To:
  • Multiple selection operations in the Tasks window.


New Task Dialog

New Task (Task Properties) Dialog.

How to Download a Website or a List of URLs for Offline Use

General

  • Starting address: Enter the starting address (URL) for the task.
  • Retrieve pages up to... (Depth): The number of levels to download. Setting this to 0 will download only a single page. Setting this to 1 will download the page referenced by the starting address and all the pages linked to this page. The number of levels can be up to 255 but it is not recommended to set it higher than 2 unless you use filters.
  • Task name: You can name a task. The name is for your reference only.
  • Download new version of task: By default, the program never retrieves the same URL twice. If the URL is already retrieved by another task, that copy will be taken by the new task. Check this box if you want fresh copy of resources for this task.

Filters

For information on task filters, go to How to use filters help topic.

Advanced Options

  • User Agent: Select a User Agent from the drop down list or type in any name you want. User Agent designates a client program (e.g. browser) used to access a web server. The User Agent is what websites see when you browse them. Some websites serve different content depending on the User Agent or may even restrict access to their content to all user agents except known ones (such as Internet Explorer or Netscape Navigator).
  • Identify as Microsoft Internet Explorer: Choose a version in the box at the right.
  • Other: Type in any other user agent name (empty user agent allowed too) or select a user agent from the drop-down list. The preset values in the list are mostly various versions of Netscape Navigator.
  • Retrieve images only after retrieving last webpage of the task: Select this checkbox if you want the program to retrieve webpages first and only then retrieve any other files linked to the webpages.
  • Referrer Page Address: URL of the Referrer for the starting address. Referrer fields for subsequent URLs extracted by the program spider are set automatically.

Notes

You can keep text notes for a task here. The notes can be up to 64 KBytes in size.


Task Toolbar

  • New Task: Launches the New Task dialog
  • Open: Opens selected task in the current browser window
  • List: Lists the files retrieved by the selected task
  • Structure: Displays logical and physical structure of the selected task
  • Report: Displays a report for a selected task.
  • Save As...: Saves the selected tasks to the disk as either linked files on disk or single HTML Help files.
  • Delete: Deletes the selected tasks
  • Refsresh: Refreshes selected tasks. Prompts you for refresh options.
  • Run Task: Runs selected tasks.
  • Stop Task: Stops selected tasks.
  • Properties: Edits the current selection's properties