Url Spider Downloads

This feature enables you to download a web page and all its hyperlinked pages from your local intranet or from the internet.  To create a Url Spider task, select the "Create Task" menu and then the "Url Spider" menu item.  Please note that https downloads require additional SSL  install files which are available from our website.

Url to spider
Enter the full path of the URL to download (http://www.hiteksoftware.com). You must include the protocol (i.e. http://).
This field also supports dynamic variables.

Hyperlink Depth
Select the Hyperlink depth(D) required.  If D=0, only the root Url is downloaded.  If D=1, direct links(DL1) to the root Url are also downloaded.  If D=2, direct links(DL2) to each DL1 are also downloaded, and so on....
WARNING:  Use D=1 to test.  If D>1, and you have a slow connection and/or the web site has many large files, this task could run for hours!!

Local Dir
Enter the full path of the local directory to place all the downloaded files.

Place files in new subdirectory
If you are regularly downloading a certain URL daily/weekly/monthly, you can save each download to a different directory based on current time/date. Select the "Create New Subdirectory" option which best suits your download frequency.  A new subdirectory based on current date or time will be created.  All downloaded files will be placed in this subdirectory.

Proxy Parameters
If you are behind a firewall, enter the proxy settings.  Your network administrator should be able to provide your proxy settings or you may also be able to obtain them from your web browser.