# robots.txt for www.mech-eng.leeds.ac.uk     Last updated: 24 April 2003
#
# The robots.txt file is a standard file that well-behaved web harvesting
# "robots" look for when scouring the web looking for content to add to
# their search engines.  The first URL they try is
# "http://your.site.your.domain/robots.txt", before harvesting any further.  
# Sites such as AltaVista, Lycos, WebCrawler use these robot or "spiders" 
#
# So, the "robot exclusion standard" (well it's only as good as the
# respect of the robot writers) was created.  Used properly, this prevents
# standard-following robots from harvesting pages from your servers that
# you don't want indexed e.g. HTBIN scripts, VMShelp (Helpgate) output,
# 
# Documentation is available at
# http://info.webcrawler.com/mak/projects/robots/norobots.html
# http://www.robotstxt.org/wc/robots.html
#
# shut off robots
User-agent: *
Disallow: /htbin/
Disallow: /help/