Advice & what not to do

• Do not overload the websites!

Downloading a site can overload it, if you have a fast pipe, or if you capture too many simultaneous cgi (dynamically generated pages).

• Do not download too large websites: use filters
• Do not use too many simultaneous connections
• Use bandwidth limits
• Use connection limits
• Use size limits
• Use time limits
• Only disable robots.txt rules with great care
• Try not to download during working hours
• Check your mirror transfer rate/size
• For large mirrors, first ask the webmaster of the site


• Ensure that you can copy the website
• Are the pages copyrighted?
• Can you copy them only for private purpose?
• Do not make online mirrors unless you are authorized to do so


• Do not overload your network
• Is your (corporate, private..) network connected through dialup ISP?
• Is your network bandwidth limited (and expensive)?
• Are you slowing down the traffic?


• Do not steal private information
• Do not grab emails
• Do not grab private information