URI: <http://ukoln.bath.ac.uk/ukoln/publications/allthrough.rtf>

This is a ‘manuscript’ version of the following paper:
Publishers and libraries: an all-through system for bibliographic data?  International Cataloguing and Bibliographic Control. 20 (3), July/September, 1991. pp.37-41.
In any citation please refer to the printed version.

Copyright to this work is retained by the author. Permission is granted for noncommercial reproduction of this work provided full acknowledgement of authorship and copyright is given.

Publishers and libraries: 
an all-through system for bibliographic data?

Lorcan Dempsey

Lorcan Dempsey is Research Officer at the Centre for Bibliographic Management which is funded by the British Library Research and Development Department.  Views expressed in this article are the author's and do not necessarily reflect those of the Department.

May 1991

Introduction

The current apparatus of bibliographic control has been the subject of much debate in the UK over the past few years.  This is in response to a number of pressures operating at local and at national levels, and which parallel developments in other countries.

It has become apparent that the production and exchange of bibliographic data is not the exclusive concern of the library community, and that greater collaboration between different sectors of the book world (publishing, bookselling, libraries, commercial and national bibliographic agencies) may be to the benefit of all.  This brief article explores some lines of change, and examines in particular how publisher-generated information may be made available for use by others in the book world, describing some recent UK experience.  The focus is largely on record creation and supply for monographs; I do not consider in detail wider changes that may have an impact on the role of the OPAC or of bibliographic databases.

Factors for change

For convenience these can be grouped under three headings: the nature and volume of the publications to be dealt with, the penetration of automation, and the pressure of costs.

Problems of UK publishing output were described at the Newbury Seminar [] in 1987:

... the output of book materials was likely to increase and change in character. Desktop publishing, on-demand publishing, the ready provision of camera ready copy would all extend the range of materials presented as books.  Competition between publishers, the pressure on academic authors to publish and the opening in London of offices by American and continental European publishers were all factors leading to greater profusion of titles.  Books would have smaller print runs and remain in print for less time.

Not only the volume but the volatility of the data to be captured is increasing.  At the same time there is an awareness that a growing proportion of items to be recorded will not be books and that the problems of control of newer electronic formats will have to be addressed.  A recent predictive report [] completed under the auspices of the British Library R&D Department suggests that "the problems of abstracting, indexing and bibliographic control of the new publications will be enormous, and with economic pressures greater than ever, organisations such as the BL may have to resort to drastic solutions."

The penetration of automation into all areas of the book world means that the bibliographic record is no longer merely a carrier for the exchange of cataloguing data.  The electronic exchange of bibliographic records in the library world is now commonplace; this is not the case in the booktrade where the focus has been on the exchange of commercial and trade data.  Recent years have seen greater convergence of interests as booksellers (especially library suppliers and export booksellers) and publishers begin to appreciate the importance of bibliographic data for marketing, and libraries continue to automate their acquisitions operations.  As the generation and exchange of electronic data within each sector of the book world (publishers, booksellers, libraries, bibliographic agencies) grows, so will exchange between sectors; such exchange may be encouraged by parallel EDI activities.  Library formats may not be ideal.  At the same time libraries use bibliographic data over the whole range of library operations: from acquisition to public access systems.  There is then, it seems to some, a growing mismatch between the current pattern of record creation and supply and the requirements of the environment just sketched.

A major example is the conflicting demands of currency and enhanced content.  The use of automated acquisition systems has created a demand for current data for generation and online verification of orders.  There is also a growing call for fuller subject information for OPACs and for selection purposes, a demand that may grow as networked access to remote resources becomes more common.  However, currency and content enhancement are conflicting aims within existing patterns of record provision.  The Currency with coverage policy of the British Library involves cataloguing certain categories of material to AACR2 Level 1 and no longer creating new Library of Congress subject data for records.  The main aim of the policy was to improve currency without the expenditure of additional resources, it being accepted that currency is a major user requirement.  One of the main complaints levelled against this change was that while currency might be improved, the reduction of content (LCSH, subtitles) badly affected subject access [].

A 1988 survey [] of national libraries showed they "have had to function in an environment of restraint in which resources have decreased at the same time as the demand for library services has increased both in volume and complexity ..."  This situation is paralleled in other libraries.  One aspect of this problem is that libraries are having to invest in access to new electronic information resources, while maintaining investment in traditional stock and its control.  Emphasis on costs is prompting consideration of alternative partnerships and sources of data.  It is especially in the interest of national libraries to look at the structure of codes and formats with a view to improving the cost-effective creation of records.

So, together these and other factors are causing a reappraisal of current practices.  Initiatives are taking place in two related areas:

-	reconsideration of cataloguing practices, record structure, and formats
-	reconsideration of the pattern of dependencies and relationships within the bibliographic world.

In the former case various national and international projects are examining the suitability of existing formats for the range of functions the record is required to perform, the use of UNIMARC to facilitate international exchange, and the content of existing library records.  In the UK, the BL Advisory Committee for Bibliographic Services has set up a sub group to examine the future of the UKMARC format, and, in association with the Centre for Bibliographic Management, is conducting a survey into the use of the format, user perceptions of its adequacy for their purposes, and preferred future adaptations of it.

The pattern of creation and supply is also changing: there is greater collaboration between record producing organisations and between the library and booktrade communities.  Again this is a development that is taking place at all levels.  For example, at national level the five other UK copyright libraries are cooperating with the BL in the production of the national bibliography.  Book Data has been cooperating in the production of CIP information and Whitaker has been awarded a contract to create CIP records from the end of 1991.  At the local level more libraries are considering the use of library supplier or Whitaker records in their catalogues because of problems of currency.  There are new cooperative ventures (CURL - Consortium of University Research Libraries) and cooperative arrangements between various bibliographic players (SLS and OCLC, and OCLC and the BL, for example).

No consensus has emerged as to how the current pattern of record supply might be best arranged to meet the increasingly various requirements of the library world.  A recurrent suggestion is that a way should be found for information generated by publishers to be released into some sort of  "all through system", which allows exploitation as appropriate by booksellers and libraries.  In this way problems of currency and content enhancement could be simultaneously addressed.  The remainder of this article explores aspects of this scenario, and describes recent UK initiatives which are bringing book world partners closer together.

Recent UK initiatives

Three recent developments are of special importance - the activities leading up to the recent formation of BIC (Book Industry Communication); the emergence of Book Data, a company set up to supply publishers, booksellers and their customers with product and market information; and changes in the production of Cataloguing in Publication (CIP) data.

BIC is the latest stage of initiatives begun in 1986 by Malcolm Peters and the MARC Users' Group, and was set up as a company in March 1991.  It will replace BTECC, the Book Trade Electronic Communications Committee, and BEDIS, the Book Trade Electronic Data Interchange Standards Committee, and promote and continue their work.  BIC is supported by the British Library, the Publishers' Association, the Booksellers' Association and the Library Association.  One important component of its work will be the promotion of the BEDIS recommendations for the communication of trade and bibliographic data.  Of special interest here is the recommendation of Working Party One on the content of publishers' bibliographic databases [].  It begins:

This report proposes appropriate bibliographic data elements for use in publishers' databases, with a view to the further use of publishers' records in other parts of the book trade and library community.  It includes recommendations for the establishment of a clearinghouse that would facilitate the distribution of such records, consideration of the future of the library-based UKMARC format and the possible extension of the system described beyond the UK.

It enumerates the circumstances under which bibliographic information created by publishers might be exchanged:

... bibliographic information in a standard form might be exchanged between publisher and bookseller or library supplier, publisher and wholesaler, wholesaler and library supplier or library, library supplier and library, and for the purpose of alerting bibliographic listing services such as Whitaker, Book Data and the British Library's CIP programme

The participation of publishers in the electronic exchange of bibliographic information would, it suggests, extend the continuum, from the first appearance of a book record in a publisher's database to the use of that record in library routines.  The standard for exchange along such a continuum would have to be MARC, given its current weight of use in the library community.  The report lists data elements for use in publishers' databases, and suggests that "an interface that would enable publishers to translate data from their own systems into UKMARC terms" would be required.  The data elements are classified as mandatory, highly desirable and optional.  Mandatory data elements which "all publishers and suppliers of bibliographic data" should carry are:

-	Title
-	Series title
-	Volume/part number
-	Volume title & subtitle
-	Year of annual
-	Edition
-	Main author/editor
-	Binding
-	Language of text
-	Product type
-	Number of physical parts
-	Current imprint
-	Name of publisher
-	Original date of publication
-	ISBN
-	Availability status
-	Full UK current retail price
-	Net/non - net
-	VAT code
-	Handling of multi-part sets
-	Readership level/interest age
-	Control data

The Working Party, chaired by Ross Bourne of the British Library National Bibliographic Service, also recommends that the British Library be asked to consider the extension of the UKMARC format to include additional elements (short title, author data and thickness for example) and that the long term future of the UKMARC format be considered.

However, the number of publishers who maintain databases with this type of bibliographic information is not great; though many of course have commercial systems.  Some use Book Data for the generation of flyers, catalogues and other marketing material; some will develop this capability themselves, influenced by a variety of factors including the activities of BIC.  There is a perceived advantage in the central control and dissemination of product information in a variety of targeted forms, and at least two suppliers are marketing publisher database systems.  Publishers have traditionally accorded a low priority to the dissemination of bibliographic information, and have not always recognised the importance to sales of accurate and up to date bibliographic tools.

One element  of the success of Book Data is that it has persuaded publishers otherwise.  Book Data acts as "a management company for bibliographic and other trade data for publishers, to whom it provides a world-wide marketing service.  It operates as a clearinghouse for new book information, and as a repository of information on all titles currently available; receiving data from 550 participating publishers and processing about 25,000 titles a year (rising to 30,000 in 1991) into our database ..." [].  It handles the submission of CIP and British Book News data for subscribing publishers.  Book Data makes available (at a cost) consistent and very full records in a variety of formats which, in addition to trade information, contain a range of bibliographic and content-descriptive textual fields.  It is not, however, complete in its coverage of UK publishing output.  Apart from its impact on the book trade, and its potential as a supplier of current and subject rich records for libraries, Book Data has demonstrated the mutually beneficial relationship between the private sector and the National Bibliographic Service.  Book Data's cooperation seems to be one contributory factor in the improved currency of BNBMARC services, as measured by the ongoing currency survey carried out by the Centre for Bibliographic Management.

The relationship between the private and public sectors is likely to increase.  From the end of 1991 responsibility for the creation of CIP records will be undertaken on contract from the National Bibliographic Service by Whitaker, the publishers of Whitaker's (formerly British) Books in Print.  The new contract will provide for the supply of advance information in MARC format for 30,000 titles a year.  One outcome of this is that Whitaker records may be used by a wider range of libraries and other organisations concerned with library standards.

So some of the links necessary for an all-through system are there, and the groundwork is being laid for extension of these links.  In some areas at least, there is appropriate overlap of elements between the data used by publishers, libraries and booksellers.  The duplication of effort between sectors was repeatedly emphasized at a significant seminar held in Newbury in late 1987, referred to above, at which over 40 leading representatives of the book world discussed these matters.  It was concluded that there already existed resources to satisfy most requirements if only effective structures for their exploitation could be found.  The present author, reporting discussions that day, noted: 

Records serve different requirements and needs and yet have overlapping functions, and involve redundancy of content and duplication of effort.  Throughout its history, in some cases from the moment a contract is signed between author and publisher, the book has an attendant record or records - so much so that, ..., it looks as though there already exists within the overall book community the resources to satisfy most needs.  Although there is continuity of supply of information, continuity of record content and format is intermittent; a user will not always have available at the appropriate moment a usable record of appropriate detail.  The subject-rich information generated by publishers is at present available as printed publicity material.  High quality British Library records are not always timely enough.  Currency is supplied by Whitaker, but these records are poor in subject information.  It was pointed out that it was a failure in record supply which had led to the creation of large costly library supplier databases and further sources of records where continuing to appear.

These words were written over three and half years ago [], and are even more true today.  Book Data is now well established and generates subject rich records for a variety of uses but does not have complete coverage.  The seminar concluded that it might be possible to harness the type of 'continuum' mentioned by the BEDIS Working Party to the benefit of all.  

Sharing bibliographic data

It may be useful to characterize the bibliographic data that might move along this continuum in terms of specific data elements.  I will describe four overlapping categories of data element: descriptive data, data for bibliographic control or structure, subject data, content-descriptive data [].  These categories are not mutually exclusive.  Nor are they firmly defined: in some cases they are determined more by existing practices of record production and use, than by essential differences.

We can imagine some "all through system", stretching from the publisher into the book trade and library worlds to the end-user, and examine continuity and change in data element requirements and existing provision.  There are two important considerations; are the same set of data elements required at each stage? and does a given data element contain the same kind of information each time it occurs?

But first: what is a book?  Surprisingly, this is not a simple or uncontentious issue.  The use of the terms work, item, book, title, volume, edition and so on vary with context and occasion; a provisional answer is suggested here [].  The word 'book' names entities at at least four levels of abstraction.  At the most abstract the 'work' has a fugitive ghostly existence.  It has a number of bibliographic manifestations; the boundaries of such entities (titles?) are decided by the cataloguer.  Each of these manifestations has one or more physical 'occurrences' (copies?).  Each copy comprises one or more physical containers, or parts of containers.

Entities are not absolutely defined at each level.  A work may be defined more or less inclusively: the extent to which revisions, adaptations, commentaries, translations, and so on, are treated as manifestations of a work or as individual works is a matter of local discretion.  Specific requirements or policy may require a greater or lesser discrimination of titles.  Normally copy-level data will not be exchanged, but in the case of early printed books there is a greater variety of physical manifestations, and a greater requirement to record them separately.

Of the categories of data element, the descriptive is the most straightforward, and within the library community has been standardised in the ISBDs.  These elements uniquely identify a title, and there is obvious overlap between publisher and library requirements here.  The book always needs to be identified and a core of common data elements is apparent: ISBN, author, title, edition, publisher/imprint, date of publication for example.  However outside the library world comprehensive shared standards for description do not exist; either for the range of data elements or their structure, and existing library codes do not cover all descriptive requirements, for author data or contents listings, for example.  Steps, described above, are being taken to standardise these elements for the publishing world and to canvas for appropriate extensions to library-oriented formats.

Of course the function of the library catalogue is not just to retrieve descriptions of specific titles, but to integrate individual items into a larger pattern of authors and works and the relationships between them, which reflects a library collection or some other aggregate of books.  A record describes a discrete title, and is then integrated into the structure of the catalogue by the assignation of headings, references and use of authority control.  In this way, not only the finding, but the gathering function is realised.  Conventionally, what is gathered are titles by the same author, the various manifestations of a work, and titles connected by subject matter.

In fact the intellectual model of structured catalogue access is an ideal only intermittently or partially achieved in UK libraries, for a variety of reasons to do with the nature of the codes and formats and the manner of their application, but these need not concern us here.

Descriptive data describes entities at the 'title' level.  The bibliographic information of concern to publishers describes titles and their physical manifestations.  Structuring data identify works, authors and their interrelationships, and has been of less concern to publishers, interested in the promotion of their own current titles.  What then is the possible contribution of publisher data in these two areas: descriptive and structuring elements?  Elaine Svenonius has recently forcefully argued that in neither case do publishers have a major contribution to make [].  She suggests that the recording of descriptive elements, or 'identifying' elements is a "fairly straightforward clerical task" and points out that "portions of this task can be performed by computer".  This is because "Identifying data elements tend to appear in obvious places on a bibliographic item, for instance on a title page or its verso".  However, this is to ignore the requirement of currency.  Libraries increasingly want current, even pre-publication data, for acquisitions purposes; they want records before they get the books.  The catalogue record may be a modified or enhanced acquisition record, or the acquisitions record may be seen as a functional but crude interim stage to be superseded by a later catalogue record: in either case current trade data - supplied as now from library supplier, CIP, or commercial bibliographic agency or in the future supplied direct from the publisher or through some 'clearinghouse' - would be useful.  As already noted many libraries, especially public libraries, are using existing trade data as the basis for their catalogue records.  

Svenonius rightly argues that "the providing of organising data elements on bibliographic records is a considerably more intellectual task, in that the establishing of bibliographic connections often requires that research be done" and that this work is "time consuming and demonstrably the most expensive aspect of cataloguing".  It is, as she notes, "precisely here, in the supplying of organising data elements that publishers are least able to help libraries".  However, precisely because of the cost, but also for other reasons, library policies and practices vary considerably with regard to investment in authority control and other more "intellectual" work.  

Decisions are made locally: policy about fullness of entry, authority control and other aspects of record creation vary enormously.  Many libraries are concerned to create full 'archival' records and there is some concern that the British Library has defaulted in its role as a creator of the 'national archival record'.  Other libraries feel that such effort is wastefully perfectionist, that the benefits of such efforts have not been proven.  Accordingly, the level of fullness of content and sophistication of created structure can range from the minimal apparatus of a finding list to the intricate machinery of some national bibliographies.  There is argument about the intervenient level appropriate for library needs, and more generally about the criteria for establishing such needs.  The debate about the nature and usefulness of the catalogue is a long and tedious one, hampered by lack of empirical data about costs, and little sense of what the components of a 'goodness of catalogue' measure might be [].  

Some UK online catalogues are probably less effective tools for bringing together titles by a single author, bringing together the manifestations of a work, and distinguishing between those manifestations than the card catalogues from which they derive.  Problems will be compounded in the network environment where records created in different libraries are consolidated in searches, and internationally where efforts are made to merge records from different countries.  These developments may help to focus discussions of existing practices.  What is clear is that more libraries will try to reduce the level of in-house creation of records; and some, at least, will move away from the notion of cataloguing from the book in hand.  For better or for worse some UK libraries would use publisher supplied records along the lines of the BEDIS recommendation in their catalogues, if they were available, with some checking for consistency of authors and titles, and as much or as little further modification as policy dictates; many others would not.  Svenonius' paper perhaps assumes the US environment where a major part of catalogue data are derived 'high quality' Library of Congress or utility records.  

The third category of data element is subject data.  There is continuity of requirement across the bookworld for subject headings or categories of some sort but very little consistency of provision or use.  This is not even an area of uniform practice within the library community: some form of subject heading (Library of Congress, PRECIS, or other) may be used, and a classification scheme (usually, but not always, Dewey).  Many libraries have no headings, few offer a phrase search on headings, and in many cases where they are used, subject headings are exploited only as a source of keywords.  In general, libraries welcome the extra textual information they provide, rather then use them in a rigourously controlled or structured way.  It is because of this latter fact that US debate about controlled vocabularies in OPACs has less relevance in the UK.  The decision not to create new Library of Congress subject data was one of the most criticized aspects of the British Library's Currency with Coverage policy.  One comment by a respondent to a MARC Users' Group survey [] on this policy is interesting:

Provided the subject of the book is revealed in the title or subtitle, or any other field, the absence of Library of Congress Subject Headings is not crucial.  Provision should be made in the UKMARC format for keywords to convey subject matter.  Reliance on LCSH is wasteful of time and money in a period of severe financial constraint.

The British National Bibliographic Service now uses a simplified version of PRECIS to provide subject access.  

Citing an internal report on acquisitions seminars run by the Publishers' Association and the British Council in China, Gillian Roche [] reports that efforts should be made to "adopt an agreed thesaurus of subject headings for publishers' subject information, enabling overseas customers to seek material on their subject interest under compatible broad and narrow headings ..."  Book Data has developed a list of headings; Whitaker a rather smaller one.

It is difficult to know how practicable or acceptable an "all through" subject system would be.  Some libraries, and certainly booksellers, would be prepared to make use of a suitable scheme of publisher generated headings, or the current Book Data headings.  Similarly some libraries would be prepared to accept a publisher generated class number, much as some now accept the class number on CIP records.  Others would not; and would follow their own procedures.  However the dominant form of subject access in OPACs is likely to be keyword searching of available fields, perhaps with growing use of classification numbers for browsing and to extend searches [].

Content-descriptive elements are required at various stages along the continuum, for promotion and marketing by booksellers and publishers, for selection by libraries.  There is also a growing clamour for subject enhancement of library bibliographic records.  Whether individual libraries enhance their own records, or bibliographic tools rich in content information (abstracts, contents pages, ...) become more widely available with possible links to local files, clearly some form of enhanced access to monograph records will have to be provided.

These data elements are used in an unstructured form by publishers and export and library booksellers and are often not available in machine readable form.  They are present in Book Data records.  But they have not traditionally found a place in library catalogues, national bibliographies or in-print listings.  Standards for their description and content are not in place.  Different agencies use different levels of detail.  One element for which a bookworld wide standard would be useful is readership level, again an element not found in library catalogues.

Current library practice points towards reduced investment in cataloguing, manifest in less full records; despite perceived advantages few libraries could invest in local addition of contents pages, abstracts, or intellectual levels.  If publishers are persuaded of the advantages of dissemination of accurate, non-judgemental content-descriptive data for selection purposes, the rest of the book world would benefit.  It may not be feasible for very many libraries to enhance their catalogues locally, but the case for ready local access to enhanced book information is very strong.  Existing access to monograph collections is very shallow, leading certainly to underutilisation of stock.  Furthermore where an increasing amount of catalogue enquires are carried out remotely, it will become more important to know more about the books from the records.

Conclusion

It is clear that there is scope for collaborative action.  Libraries would benefit from improvements in currency.  They would also benefit from access to records with content-descriptive data; whether that data is added to local catalogues or not.  An "all-through" system as such may never exist - it may be impossible to reconcile this with the interests of existing players.  What may happen is that libraries use a number of databases, some publisher derived, to satisfy their requirements.  

References