WebWatch /robots.txt checker

This utility will perform some simple checking of the server-wide /robots.txt file for any URL entered below.

Common errors that this CGI script will scan for:

Errors	Warnings	Optimisations
User-agent fields without values	Disallow fields without value	Multiple User-agent fields with the same value
A missing User-agent field	Allow fields without values
Disallow directives not corresponding to a User-agent	Unrecognised fields
Allow directives not corresponding to a User-agent
No Disallow directives corresponding to a User-agent