Url Monitor
This feature enables you to monitor web pages for changes within multiple web sites. Each time this task runs it updates a log file on your system with the modified time stamp of each web page it finds within the web site. If a change is detected, it will download the file to a selected directory. To create a Url Monitor task, select the "Create Task" menu and then the "Url Monitor" menu item. Please note that https downloads require additional SSL install files which are available from our website.
Url to spider
Enter the full path of the URL's to monitor. You must include the protocol (i.e. http://). You can specify multiple URL's by separating then with a comma. Example - http://www.hiteksoftware.com,http://www.espn.com. This field also supports dynamic variables.
Hyperlink Depth
Select the Hyperlink depth(D) required. If D=0, only the root Url is monitored. If D=1, direct links(DL1) to the root Url are also monitored. If D=2, direct links(DL2) to each DL1 are also monitored, and so on....
WARNING: Use D=1 to test. If D1, and you have a slow connection and/or the web site has many large files, this task could run for hours!!
Local Dir
Enter the full path of the local directory to place all the results. If a file is found to have been changed, then it is downloaded to this directory.
Filename Filter
Leave this field blank if you want all files to be downloaded, or enter the name filter for the files you wish to dwonload using the following wildcard rules . This field also supports dynamic variables. Using dynamic variables, you can select a file based on current date/time in addition to the normal wildcard search.
You can also select only specific sub folder(s) to be downloaded by entering the sub folders relative path. For the task to realize, that a sub folder filter needs to be used, the filename filter has to include atleast one path separator '/'. Example: /test or /test/subfolder etc..
Maintain Timestamp
Choose this option if you need the downloaded file to keep the same timestamp as the file on the web server.
Download Option
There are 3 download options:
1) Download always - This will overwrite the existing local file if it exists irrespective of timestamp
2) Download based on local file timestamp - If a local file exists in the download folder, a new file will be downloaded only if the timestamp of the file on the web server is newer than that of the existing local file.
3) Download based on archived url timestamp - Every time a file is downloaded by this task, its timestamp on the web server is locally archived. If this option is selected, a new file is downloaded from the web server only if its timestamp is newer than the archived value.
Place files in new subdirectory
If you are regularly monitoring certain URL's daily/weekly/monthly, you can save each download to a different directory based on current time/date. Select the "Create New Subdirectory" option which best suits your download frequency. A new subdirectory based on current date or time will be created. All downloaded files and results will be placed in this subdirectory.
Proxy Parameters
If you are behind a firewall, enter the proxy settings. Your network administrator should be able to provide your proxy settings or you may also be able to obtain them from your web browser.