UKOLN AHDS Error Detection on the UKOLN Web site



Background

The UKOLN Web [1] site runs off a number of different Apache and IIS servers at the University of Bath. There are now thousands of pages on the site, which are maintained by a number of different people.

Problem Being Addressed

CGI scripts and forms are used occasionally on the site, for example for booking forms for conferences. Sometimes scripts and the forms connected with them get broken by people moving files, editing scripts and by out of date data. In addition errors may occur when the end user inputs unexpected or illegal data, if another service such as a back end database, fails. In this way, script errors may be indicators of larger problems that need to be quickly addressed.

It was previously impossible to locate specific errors unless a user emailed the site to let tell the Web master that a script error was appearing on a Web page.

The Approach Taken

It was decided that a new internal server error page would be created which would allow the Web support team to establish what and where problems are happening. The error page would also generate an email to the Web support mailing list saying what had happened and why.

When the support team receives these emails they then decide if this is a problem that requires immediate attention, or, as is sometimes the case if this was a "correct" error. For example, if a robot visits a script and attempts to access the script in an unexpected way, it is entirely appropriate that the robot should see an error. Not all server errors are bad.

The internal-server-error.cgi script created was fairly simple. The process involves:

A template was used for the HTML so the look and feel could be edited. Once broken CGI scripts were reported the responsibility for changing the page then lies with the owner of that page to remove or repair the particular script.

Report Information

The report information sent to the Web support team includes the following information:

Internal Server Error - www.ukoln.ac.uk - report follows:
REMOTE_ADDR -> 138.38.32.85

This is the host of the machine running the script, it may be useful to track users who have seen the 500 error message.

REDIRECT_SCRIPT_FILENAME ->/opt/Web/content/www.ukoln.ac.uk/cgi-bin/booking-form.pl

This is the script that is failing, which is very useful to know.

HTTP_X_FORWARDED_FOR -> 138.38.146.50

If the end user is going via a proxy this may show the actual address of their machine.

REDIRECT_HTTP_USER_AGENT -> Mozilla/4.0 (compatible; MSIE 6.0; Windows NT5.0)

It is useful to see if a particular browser is causing an error.

REDIRECT_ERROR_NOTES -> Premature end of scriptheaders: /opt/Web/content/www.ukoln.ac.uk/cgi-bin/booking-form.pl

This is the error message received, which is also logged to the Apache error log.

SCRIPT_URI ->http://www.ukoln.ac.uk/cgi-bin/booking-form.pl

This is the Web page the user was attempting to view that generated the error.

HTTP_REFERER ->http://www.ukoln.ac.uk/events/bath-profile/form.html

This is the page from which the failing script was linked from.

Problems Encountered

The main problem was that after the system was set up there were a lot of error messages being sent to the Web support team, though it was anticipated that this would change as the broken scripts are edited and removed. To address this problem the messages are sent to a separate Web-errors email list from which members of support can opt out.

Things We Would Do Differently

There are various improvements that could be made to the error detection script. Firstly the message could get sent to the server administrator (from the environment variable), which would make it easier to configure the email address of the person to send to, etc.

Another very useful improvement would be for each script on our servers to specify who the error report should go to, however we are unsure if this is configurable. We are currently considering if it would be possible for whoever writes a script to set a particular variable to their username. The error script could then read this and attach this username to the email, such as 'for the attention of xxxusernamexxx'. Or even just send that person the error email.

Once this problem has been tackled it would be helpful to put in place an error logging system that notes who is responsible for fixing the error and marks off when they have done it.

Note: This was initially an RDN [2] development that was reused by UKOLN.

References

  1. UKOLN Web site
    <http://www.ukoln.ac.uk/>
  2. Resource Discovery Network
    <http://www.rdn.ac.uk/>

Contact details

Pete Cliff and Eddie Young
UKOLN
University of Bath
Bath
BA2 7AY

QA Focus Comments

For QA Focus use.