URL:<http://ukoln.bath.ac.uk/ukoln/publications/dz3950.rtf>

This is a ‘manuscript’ version of the following paper:
Distributed library and information systems: the significance of Z39.50. Managing Information, 1(6), June 1994.
In any citation please refer to the printed version.

Copyright to this work is retained by the author. Permission is granted for non-commercial reproduction of this work provided full acknowledgement of authorship and copyright is given.

Distributed library and information systems: the significance of Z39.50

Lorcan Dempsey
UKOLN: the Office for Library and Information Networking

Introduction

Z39.50 is an application layer protocol which supports the construction of distributed information retrieval applications. It is not yet widely deployed, nor does it support the full functional requirements of many typical library or bibliographic applications. Nevertheless, 'Z39.50' has become a buzz-word, and its proponents see it as central to the future of library and other information systems. Why is this? This brief article will try to answer this question, and will sketch how it is being used in some current projects.  For convenience, it adopts a library perspective, although the Z39.50 protocol potentially has wider applicability as a general purpose retrieval protocol. (It should be noted that Z39.50 is a US national standard. There is a parallel ISO standard, ISO 10162/3, Search and Retrieve, usually abbreviated to SR, which is effectively a subset of Z39.50. Unless the context suggests otherwise, Z39.50 is used here to include SR. See the sidebar  Z39.50 and SR.) 

A standard for client-server retrieval applications

In the client server model, one program (the server, or target in Z39.50 terminology) makes services available on request; it does not initiate communication. Another program (the client, or origin in Z39.50 terminology) contacts the server and requests services. The pattern of interaction and the content of exchanged messages are defined by application protocols. These protocols may be proprietary (as they are in some new integrated library systems) or they may be standard. Typically a user application interacts with the client program, and is shielded by it from the complexity of the underlying communications process. Users of the Internet Gopher or WorldWibeWeb will be familiar with this pattern. A user of a Gopher client, for example, can request various services from Gopher servers and can browse distributed servers seamlessly, without leaving the simple menu-based interface. In fact, most common Internet services operate along client-server lines, in which communicating software components combine to support applications. For example, a user can initiate file transfer with a file transfer client which uses the FTP protocol to transfer files between machines. Quite often, the user application that interacts with the client will be a user interface program, which presents services directly to a human end-user. However, that is not necessarily the case; the "end-user" may in some cases be another program or application. 

In the case of Z39.50, server programs make database applications available. The client formulates queries, manages the search procedure, and presents the results of searches.  On the client's instruction, the server passes queries to the database application and results back to the client. The clients and servers communicate using the Z39.50 protocol, which makes use of underlying network services (see SR and Z39.50 sidebar).  Z39.50 defines the mechanics of interaction, an apparatus for the expression of the syntax and semantics of queries, and formats for the exchange of data. In this way, user interface and search services can be designed independently of each other - the output of each is converted into the protocol formats for transmission. Descriptions of how the protocol actually works can be found elsewhere. ,  This arrangement has a number of general advantages. Database applications can be made available through standard server interfaces which can be accessed by diverse user applications. Conversely, a user application can communicate with diverse servers. Furthermore, the client and server exchange structured data which can be further processed. 

What this actually means in practice can best be seen by describing some motivating service environments:

	End-users cannot use endlessly proliferating different user interfaces. Z39.50 does not prescribe a standard user or dialogue interface; it offers a standard way for a particular interface to communicate with servers. The interface may be implemented in association with a standalone client, such as the one marketed by VTLS for example, or it may be implemented in association with an existing interface, as is the case with some emerging OPAC offerings. In this way, a user can access different database applications through a single user interface. In a library context, for example, one could imagine a user having access to the local catalogue, to locally mounted databases, as well as to various remote resources through the same local interface. (In an interesting recent development, some CD-ROM publishers are investigating the delivery of products with Z39.50 server interfaces.)
Some European implementors

IRIS. This is an Irish current awareness and document supply service. As part of the service users will be able to search the OPACs of six Irish libraries and to request items. The user interacts with a user application which hides the different OPAC interfaces and connection details and which provides integrated search and request functionality. The user application, developed by Fretwell-Downing, communicates with the OPAC servers through Z39.50. The project is supported by the  European Regional Development Fund through the Telematique Programme of the EU.

Nordic SR-Net. This is a project sponsored by NordInfo to link union catalogue organisations in Nordic countries, and to develop a consolidated bibliographic resource. Users should have transparent access to the various systems, and be able to easily transfer records between them. Participants include shared and union catalogue organisations in Norway, Sweden, Finland and Denmark. Iceland participates as an observer.

Project ION*. The Interlending OSI Network aims to link national interlending systems in the UK, The Netherlands and France using the ILL protocol (ISO 10160/1). One component of this initiative involves implementation of SR on the Viscount and Pica databases, to allow mutual lookup.

DVB-OSI. It is proposed to link several German regional Verbundsysteme and online host systems, again with the aim of presenting the user with transparent access to a consolidated resource for search and request of materials.

Socker*. This project is implementing SR client applications in various different environments. IME is integrating a client with its library system; UNI-C, the Danish academic computing and networking organisation, is implementing it within a standalone workstation application; FEK, a library computing organisation, is implementing it as a network 'information gateway' application, providing a network accessible client as part of a larger service.

Europagate* Danish, Irish andSpanish partners are implementing a gateway between SR and Z39.50.

Other projects. The British Library is implementing Z39.50 and SR server interfaces to its OPAC, with a view to participating in experimental international projects. This experience will feed into planning for incorporation into possible service offerings. Other organisations are implementing in various contexts: Pica, Karlsruhe University, ESA-IRS, Library system vendors (SLS, BLCMP, Fretwell-Downing...). Further projects will probably be funded as a result of the third call for proposals under the Libraries Action Plan of DGXIII. At the time of writing, proposals are still being evaluated.

* supported under the libraries' programme of DGXIII of the EU

	Future distributed bibliographic information systems will require various forms of program to program communication. Examples of such links are between a union catalogue and local circulation systems to determine the availability of a particular item, or between a table of contents service and a serials holding list to determine whether an article is held at a particular library or not. User requirements will be served by distributed applications which link searching tools (e.g. BIDS, A&I services, table of contents services) with locating tools (e.g. catalogues, DSC files) and requesting tools in such a way that the user is presented with a unified interface and does not have to separately interact with services through different channels. This type of application is not now common, and a number of customised solutions are being put in place. However, it is important that current technical choices do not unnecessarily constrain future cooperative or information service activities. Z39.50 is emerging as the standard of choice for specifying some of the required interfaces.

	Because it supports retrieval, Z39.50 will be a transport mechanism, for bibliographic data, but also potentially for other data as well. This is important not only for the delivery of structured data, as in cataloguing environments or where data is imported into personal bibliographic systems, but also where further processing or manipulation is required. For example, one could imagine an application formulating a query to a serial holdings file based on its understanding of the content of a structured record returned in the search of a table of contents service.

	Library resources are currently poorly integrated into the emerging resource discovery systems on the Internet. For example, to connect to a library catalogue, Gopher or WWW typically drops into a telnet session. In some cases, gateways are being built between these services and library systems, the Norwegian BIBSYS system, for example, which allows access to the system from within Gopher. A natural future development is that gateways are built between these systems and Z39.50.

Currently, most bibliographic systems allow terminal access; they are not "network" applications as such. Library and bibliographic services need to be integrated into the emerging world of networked information services, and distributed applications will be built which provide the user with a unified service interface to a range of services. These will be facilitated by a protocol infrastructure which constrains the number of interfaces that have to be written. Currently, Z39.50 seems to be the protocol of choice for moving bibliographic applications onto the network and for linking bibliographic applications.

Z39.50 and SR: how they stack up

Z39.50 is developed under the auspices of NISO, a committee of the ANSI. Technical work is largely done by a very active Z39.50 Implementors' Group (ZIG) which provides a forum for input by developers, and the Z39.50 Maintenance Agency is hosted by the Library of Congress. SR is an ISO standard, developed through TC46/SC4/WG4.

SR is a subset of Z39.50. It is largely being implemented in Europe, but does not have the critical mass of development effort that Z39.50 has. Some European service providers are implementing both Z39.50 and SR. There is some discussion about merging this dual stream of activities, but, at the time of writing, it is not clear what will happen.

SR and Z39.50 are both defined within an OSI framework, and assume services from underlying OSI protocol layers. They are application layer protocols, which can interoperate with little modification. However, because it is being implemented in an Internet context, Z39.50 is often implemented directly over TCP/IP. In some projects SR has been implemented using ISODE, which provides upper layer OSI services, over TCP/IP. In some, it is being implemented in a pure OSI environment. Currently these three environments are incompatible, and networking will rely on various gatewaying solutions.

This dual standards effort and the heterogeneous network environment are complicating factors for those considering implementation.

Prospects

Most implementors are still in prototype or test phases. Some production servers do exist: OCLC and RLG, for example, each make some databases available through Z39.50 servers. It is possible to buy both server and client products in various configurations from a number of suppliers, and several public domain versions exist. Current implementors include libraries, library system vendors, information providers (e.g. Chemical Abstracts and Mead Data Central). Some European implementations are listed in a sidebar.

How quickly will Z39.50-based services become common? It is difficult to say. The standard is being refined, and currently falls short in various ways of being able to support routine production services. Many potential implementors are waiting until the protocols are more stable, or have been proved in production services, before they invest in development.

Of course, Z39.50 is not important in itself. The ability to build distributed bibliographic applications is, and Z39.50 is the current lead candidate in support of such applications. It is part of the required 'plumbing', not, despite its current fetishization, an end in itself. It may be that other protocols or approaches emerge in the future. However, the library community now needs to begin to get experience of building the distributed services mentioned above.  How do you support access to multiple regional OPACs, a table of contents service, BL files etc., in such a way that a user is provided with a seamless interface which allows them to identify, locate and request items they are looking for, subject to whatever agreements are in place?  How do you create appropriate links between existing library services and the Internet world of resource discovery tools, ftp sites, and so on?  How do you provide integrated access to diverse local and remote databases? These are important challenges, and Z39.50 will be an important initial building block in their solution. 

References