Prepared by

The Tavistock Institute

Evaluation Development and Review Unit


















Framework for Annual Reporting




Structure of this Report








The contribution of projects to eLib programme goals




Enhancing project potential




Obstacles to Progress




Project organisation and management
















Development and enhancement of services




Promoting the project




Training and awareness raising




Evaluation activities




Main areas of development








Assuring quality of information resources




Implementation processes




Sustainability issues for subject gateways
















Materials/resource development




Developing systems for copyright management




Implementing field trials








Promotional work








Project findings




Key lessons




The organisational context of ODP


















User requirements assessment and (market) research




Systems development and integration




Copyright matters








Experience with copyright clearance




Cultural change




Demand and usefulness
















Activities and Progress








Difficulties and challenges








Future Developments












Technical approaches




Editors, authors and readers




Organisational Issues




















Digitisation and OCR work




Copyright clearance




Interface design and end-user feedback




Service delivery












Cost implications




The publishers’ perspective




























Key lessons and findings

















Development of training materials




Delivery of workshops, seminars and on-line training




Formative evaluation and quality assurance




The launch of new services




Some key findings








Inter-project links and influences




Sustainability/exit strategy








Benefits of eLib




Access to Network Resources




On Demand Publishing and Electronic Short Loan




Electronic Document Delivery




Electronic Journals, Pre-prints and Quality Assurance












Training and Awareness



Appendix 1: Format for Second Year Annual Reporting



Appendix 2: Project Reports Reviewed








This document provides an overview of the annual reports of forty-five eLib projects, mainly concerning their second year of project activities. While last year’s Synthesis of 1996 Project Annual Reports emphasised start-up and implementation issues in projects, this year’s synthesis emphasises actual project outputs and outcomes and substantive findings.

Programme wide achievements, challenges and issues

Most eLib projects are now engaging with real implementation issues in a diverse set of organisational contexts. Projects are beginning to have effects in the programme’s external environment:

Significant improvements can be observed in project management and development practices due to widespread adoption of iterative development cycle approaches, supported by formative evaluation. Project potentials have been further enhanced by

Projects continue to encounter a number of obstacles to their progress, although these are not so pronounced or widely shared as last year:

As projects move in their second year to testing their projects and services with real users, a number of challenges in the external environment to the sustainability of projects are becoming clearer:

Access to network resources

The second year annual reports reveal significant differences between the subject based gateway projects in terms of their achievements and likely futures, reflecting varying circumstances including

During the second year, most of the ANR projects moved beyond the uncertainties and difficulties of the start-up phase and began to consolidate their position. At least three of the projects felt they had moved to a stage of becoming an operational service rather than a pilot demonstrator.

Successful strategies for service development have included a greater emphasis on engagement with user communities, especially as regards building partnerships in the supply and use of subject based resources, as well as additional emphasis on training and awareness and promotional activities.

ANR projects are beginning to clarify possible ‘exit strategies’ with most foreseeing some mixture of subscription income, commercial alliances, sale of related products and continuing core funding.

On Demand Publishing and Electronic Short Loan

The ODP and ESC projects continue to confront significant operational challenges most notably:

Nevertheless, the ODP projects (which are in the main longer established than the ESC projects) have made great strides in clarifying the operational requirements of workable systems, and produced a significant body of evaluation results. However, it now appears that the capital investment required for ongoing systems may be beyond the capacity of any one project or small set of institutions. The longer term future of ODP/ESC initiatives in HE will be probably be primarily shaped by the current rapid changes in the attitudes of publishers towards electronic publishing.

Electronic Document Delivery

The EDD projects are among the most ambitious of the eLib projects, both technically and in terms of service and organisation. As the EDD projects move into their second year, some projects have developed strong collaborative links with commercial partners and closer convergence and inter-operability has emerged between individual projects.

The EDD projects have uncovered the relative unreadiness of publishers when faced with the practical technical task of supporting electronic document delivery, and the projects themselves have begun to emerge as resources for other HE journal access initiatives. Nevertheless the future of the EDD projects remains uncertain, and they themselves raise questions about the appropriate role of universities in areas of complex technical development where the level of investment and associated risks are more suited to commercial software and systems companies.

Electronic Journals (including Pre-prints and Quality Assurance)

Electronic Journals (EJs) has been the most innovative and speculative area of the eLib programme, and an enormous amount both technologically (concerning the actual technical form of EJs) and conceptually (as regards new modes of scholarly communication) has been achieved since these projects began their work. As is their function as demonstration projects these projects tell us much about what can be done now, what scope there is for building on to current capacity and the obstacles to exploiting current capacity.

There are a range of different but interrelated development areas which these projects are producing significant learning:

Overall, it may be observed that the EJ projects are now achieving the status of less ‘pilot projects’ (representing lines of development which may be found wanting and abandoned) as ‘path finding’ projects whose learning is likely to influence a coming generation of EJ initiatives.


The three digitisation projects have successfully overcome or defined ways to overcome the inherent technical challenges in digitisation. Now as they approach the end of JISC funding, questions of sustainability are being raised both in relation to the level of demand and the basis of cost recovery. Towards the end of the reporting period, projects have been giving much greater emphasis to investigating the needs of potential users as a guide to better defining future service offers.


Having addressed the considerable technical demands of establishing specific imaging services, and the development of workflow strategies around the image acquisition process, the imaging projects have been more closely concerned this year with the use of their resources in practice by differing types of end users.

Training and Awareness

In their second year the T&A projects have continued with their existing programme of activities employing both cascade (train the trainers) and direct mass training approaches including;

Generally, as their knowledge of trainee needs has increased the training offered is becoming more tailored and diverse. Demand for training continues to outstrip supply and the T&A projects are emerging as an important organisational capacity within HE quite distinct from their role within eLib itself.


This report provides an overview of forty-five eLib projects, based on information provided in their annual reports. In the large majority of cases, projects were reporting on their second year of activities. The remaining projects started up in the second phase of eLib funding and thus were mainly reporting on the first twelve months of the project lifecycle. The issues relating to start-up and early implementation of eLib projects were well covered in the Synthesis of 1996 Project Annual Reports and so we do not give a great deal of attention to these in the 1997 Report.

As we found in the first overview of annual reports, projects provided a wealth of data about their activities, progress, achievements and findings. It has been difficult to do justice to the material, and our synthesis is no substitute for closer examination of the individual annual reports. In addition, projects have produced many other deliverables or outputs which amplify different aspects of their work, and these provide valuable information for those looking for exemplars of good practice, details on technical choices, baseline data and relevant market research, and manuals and procedures that encapsulate learning that has often been painfully, and painstakingly, acquired.

1.1 Framework for annual reporting

The pro-forma schema for the presentation of 1997 annual reports varied in only minor ways from the one used in 1996. Projects were requested to structure their reports around four key areas, each with a set of open-ended questions to be addressed (see Appendix 1 for the amended 1997 annual report format). These were:

For our analysis of this year’s reports, however, we have adopted a different framework that is more appropriate to the lifecycle stage of the majority of eLib projects. Whereas our principal focus last year was on the generic activities associated with the start up phase, this year we have concentrated more on the individual domain areas. As projects have moved to a more stable, mature phase they have a great deal more to say about the experiences of implementation and the initial findings from their evaluations, both of which are largely domain based. The issues, problems and challenges pertinent to each domain are also becoming more apparent.

We have therefore eschewed a common format as the basis for a synthetic overview of each domain area in favour of a more pluralistic one. We have something to say about each of the four areas above, but we have not presented the findings of our analysis exclusively within this framework, nor is there complete consistency in the headings used for the different domain reports.


1.2 Structure of this report

There are two main sections to this report. The first section draws together the themes and issues that run across the programme as a whole or are shared by a number of domain areas. Thus we report on the contribution of projects to achieving eLib programme goals, the important factors which are shaping the projects and enhancing their potential, and the obstacles to progress and longer term sustainability.


The second section provides a domain by domain overview. We try both to convey areas of commonality and difference between the projects, and also to highlight what seem to be the key themes, issues and findings for each domain. In the main, we have identified projects by name as this makes it easier for the reader to follow up a point of interest by reading the relevant project annual report.


2.1 The contribution of projects to eLib programme goals

The close-on fifty eLib project annual reports demonstrate clearly how much has been achieved since the Programme’s inception. Whilst the analysis of the domain areas below documents many of the achievements of individual projects, it is their collective contribution that concerns us here.

The eLib projects are ranged across a spectrum from piloting actions in tested environments through to the delivery of services in real world contexts. The second year of the eLib programme has seen substantial movement along this spectrum with many projects now engaging with the real issues of implementation in different organisational contexts. As projects have become more stable and mature, so they have been able to direct project energies to making real progress towards their goals rather than being pre-occupied with internal project management and fire-fighting on technical glitches. As one project commented, ‘our progress has generated confidence within project teams, leading to a shared sense of mission and a will to overcome the problems which inevitably existed initially’.

The achievements of the eLib projects in this second year reflect this more outward orientation.

First, the annual reports indicate a number of areas where eLib projects are playing a high profile in debates and have been able to contribute to the development of international standards. One notable area is the development of (Dublin Core) metadata standards and conventions for the Internet. Some projects are taking a strong lead in this, but nearly all projects were able to report on actions taken to embed metadata in their electronic resource materials.

Second, the eLib projects have served to mobilise many companies within the publishing industry into taking a position on electronic information and making more transparent their copyright practices and defining charges. Although inconsistency is still rife and there has been some reversal of more flexible attitudes, the fact that many more publishers now have developed, or are developing, policies in this area is welcomed by eLib projects. At a programme-wide level, it is now more apparent where HE libraries are best placed to negotiate at an institutional level and where there is potential leverage in looking to a centralist or consortium solution.

Third we see many examples where eLib projects have found new synergies with other developments, to the mutual advantage of both. Fruitful partnerships have been forged with EU funded projects in parallel areas of development, both on the technical side but also in the incorporation of standards, protocols and quality assurance procedures. An eLib project experimenting with digitisation of a leading journal in the design field, for example, was able to take advantage of an EU technology development project to test the delivery of quality text and images on high speed networks across Europe. Some projects have enjoyed similarly productive relationships with commercial sector companies, engaging in co-learning through joint production. A collaborative venture between an ANR project and two other agencies, for example, has led to the development of a shared social science Thesaurus, not previously configured in the project proposal.

Fourth, the eLib projects have fostered networking and dialogue across the system in all manner of unanticipated ways. Numerous projects have made effective use of the Web, not only as a means for dissemination their various outputs, but more importantly as a tool for stimulating debate, sharing information and learning. Whilst there is much still to learn about what makes for effective communication, it is noteworthy that reaching out to the wider HE library community is a goal shared by many, not just those in the Training and Awareness domain. Lis-discussions have been set up on such topics as metadata debates, strategies for copyright clearance, networked learner support and the use of EU documentation in European Studies courses.

2.2 Enhancing project potential

The potential and relevance of projects has been enhanced by the widespread adoption of an iterative development cycle, drawing on formative evaluation. This has provided the basis for continuous improvement of the various products and services under development. We commented in the 1996 Synthesis Report on examples of good practice in this area (as well as some of the limitations of the approach taken by most projects), and the second round projects have largely followed suit.

Two further ways in which projects have enhanced the relevance and usefulness of their products and services and increased their exploitation potential have been 1) through building flexibility into the requirements specification, and 2) through cross-project borrowing and synergy, and links to supporting studies and other research.

Flexible Design Choices

We observed many instances where projects were taking advantage of new technologies to increase their capacity for exploitation. First and foremost, there has been widespread take up of Web technology in response to the growing universality of access to the Web. This has in many cases meant abandoning the original technical specification to take advantage of a networked version, or developing the two in tandem.

Another example is the common adoption of an open systems architecture. In an evolving and maturing IT market, such a toolkit approach makes it much easier to incorporate new features or to validate to new standards as well as opening up more market opportunities for a differentiated product. Various projects have been experimenting in this area, including making provision for changing the software base whilst maintaining existing data. Many of the software packages being developed anticipate the emergence of new solutions to problematic areas, such as for example charging systems, and can readily incorporate such changes.

Cross-project borrowing and links to research

Projects provided many examples of the benefits of being part of a wider domain area and programme, particularly where cross-project borrowing of outputs, technical solutions, strategies or methodologies, has added value to their own project.

Some projects have also benefited enormously from being located within a ‘centre of excellence’ bringing together a concentration of related projects. SOSIG, for example, is located within the Institute for Learning Research Technology which is co-hosting four other data-based social science research services. All benefit from the sharing of expertise, hardware and software as well as from interoperability - so providing a ‘virtual resource centre’ or one-stop shop for all social science research data.

The contribution made by supporting studies and other forms of linked research is also noteworthy. Projects that were grounded on a good research base, were informed by research undertaken in parallel, or which had adopted an action research methodology commented favourably on the contribution that research made to their understanding of the innovation process and the way in which it had informed their strategies and interventions. A good many projects learned to their cost that ‘user feedback’ downstream in the development path is no substitute for a good knowledge of the user population and the context in which the innovation is being introduced, at the start of the design process. Exemplar models of good practice in this respect include the ACORN project which was underpinned by in-depth user surveys and profiles, and SOSIG and OMNI which both benefited from a parallel research study of the effect of subject based gateways on the working patterns of end users and their changing use of the Internet.

2.3 Obstacles to progress

Elib projects encountered a number of obstacles to their continuing progress in the second year, although not so pronounced or widely shared as those which dominated the first year of the project lifecycle.

The main obstacles identified by projects in their annual reports originated from three different sources:

Inherent in the design process

Several projects have realised, in the course of their second year, that it was a mistake to have focused their project on a technical ‘solution’ without being clear either about the nature of the problem (or for whom), or about the downstream implications for its uptake and embedding in user practices. The design process was the classic, now discredited, linear one with implementation seen as a last stage, rather than as part of the first critical stage of analysing the future context of use.

Such a realisation was most clearly articulated by a project in the On Demand Publishing domain:

‘It was originally thought that we would be developing a method of supplying materials electronically and this was to be the main output of the project but the technology was there and it was just a matter of setting up the service (buying hardware/software, route for academics to supply material and get it mounted on the server). However we are no longer trying to find a method to supply electronic material, the focus really needs to be on changing the attitude of academics so that they are using the technology and are encouraging students to use the technology. This point should possibly have been emphasised from the beginning but only in hindsight can we see that main stumbling block has been the reluctance of academics to be involved. Therefore the technology was there all along and it is the management, staffing and culture that needs to be looked at.’

A second shortcoming in project design (inferred by us from reading the annual reports) which impeded progress was the lack of an adequate mechanism or procedure to support effective dialogue with publishers as stakeholders. There were notable exceptions here (ACORN for example, which set up a Participating Publishers Seminar) but our impression from reading the annual reports is that most projects dealt with publishers on an individual, case by case basis, rather than engaging with them as part of a broad process of dialogue.

2.4 Project organisation and management

For many projects, the main setback to their progress in the second year stemmed either from staff turnover and the time taken to fill a vacant position, or to an inability to recruit someone with the requisite skills. In cases where staff mobility involved the prime driver of the project, it was even more likely that the project would experience a set back. Staffing difficulties were a main difficulty in the projects’ first year (1996 Synthesis Report), and it would appear that the market for staff with the necessary skills is still very buoyant, with little incentive for people to be in short-term contracts unless the rewards are very high.

Several projects also experienced ongoing difficulty with geographic dispersal, especially where the technical task of systems development and integration was distributed between partners. This was resolved in some cases by consolidating development tasks and relocating them in one site, and in others by working hard to improve communications. Whereas this was a widely shared difficulty in the first year, by the second year many project team members had however become more familiar with a mode of collaborative, dispersed working. A comment in one of the annual reports well illustrates this point:

The project has learned to work across geographically dispersed sites. But also project team members have come to know each other better, through meetings, and have developed an understanding of and respect for each other’s work. The communication difficulties experienced during the first year have practically disappeared, due to the development of a good working relationship between the production staff at each site.

The external environment

As projects increasingly move to testing their products and services in the ‘real world’ and have to think more about the post-funding stage, so they have become more aware of impediments or barriers in the external environment. Their evaluations are also highlighting the difficulties of implementation, especially cultural and organisational ones that were unanticipated.

We highlight below three aspects of the external environment which are impeding the sustainability of projects.

First, various of the domain reports highlight the changing market environment in which eLib projects are functioning, creating uncertainty about the future demand for their products and services. Key actors include commercial companies, the publishing industry and database vendors and their direct entry into the networked information market will inevitably impact on the niche market environments in which eLib projects are operating. Elib projects will need to provide value-added services to the HE community if they are not able to compete on price, but it is unclear whether individual HE libraries will have the resources to buy many of the services on offer.

Second, for several domains, copyright clearance remains a significant bottleneck, requiring a different solution from the institution based approach taken by projects.

As we noted above, publishers are becoming more organised and many now have policies on copyright for electronic materials in place, but the net effect just in the lifetime of projects has been a step backwards. For the majority of projects, the request was limited to the duration of the trial with a narrowly defined group of students. The refusal rate for permissions has gone up, the charges in many cases have increased, and a substantial number of requests for copyright clearance have gone unanswered. Copyright clearance can thus incur high labour costs with no return. However, any consideration of the copyright issue needs to begin with the conclusions of the important supporting staff on On Demand Publishing produced in this period by a consortium of ODP projects.

Third, attitudes and behaviours among professional librarians, academics and students continue in many instances to be a barrier to the effective embedding of eLib products and services. Librarians ‘on the ground’ are reluctant about passing on the costs of electronic resource materials to students, academics are as yet unwilling to change well-established habits of preparing reading lists ‘just-in-time’ (but not ‘soon enough’ for the lead-time required for electronic delivery), and many students (and staff) still lack the confidence or the skills for effective use of networked information retrieval. On a more serious level, academics have concerns about questions of equity and access in the use of electronic materials as well as about the impact on student’s search strategies until there is a critical mass of material available on line. Librarians for their part are not convinced that in some areas of development, such as electronic short loan, their work load will decrease rather than increase because of the need to run parallel print-based and electronic systems.

One or two projects expressed hope that the Dearing Committee Report would provide a more hospitable climate for eLib project activities in the Training and Awareness domain, particularly with respect to the ongoing professional development of librarians and academic teaching staff.


PROJECTS: ADAM: Art, Design, Architecture & Media Information Gateway; BIZ/ED: Business Education on the Internet; CAIN: Conflict Archive on the Internet; CATRIONA II: Management of Electronic Information Resources; EEVL: Edinburgh Engineering Virtual Library; IHR-Info: Information Server for History;; OMNI: Organising Medical Networked Information; ROADS: Resource Organisation and Discovery in Subject-based services; RUDI: Resources for Urban Design Information; SOSIG: Social Science Information Gateway

3.1 Introduction

The nine ANR projects are diverse in nature. The main group of seven projects provide subject-based gateways in a number of disciplinary areas and fields of study, predominantly utilising the ROADS software. ROADS is a distinctive project, set up to develop tools, methods and protocols that enable subject specialists to build and maintain information gateways. Catriona II also stands alone as an ANR project, its concern being the management and creation of institutional and departmental electronic resources in Scottish universities, and associated infrastructure and policy issues.

Given the synthesising and comparative aspect of this Report, the focus in this domain area is very largely on the subject-based gateways. These projects are primarily concerned about adding value to information through a process of selection, description, indexing and cataloguing, and through provision of ancillary services.

Our analysis of the second year annual reports reveals quite significant differences between projects in their achievements and the extent to which they have potential sustainable futures beyond the life of the eLib program. The reasons for this would seem to be many and varied, and can only be teased out of the reports. Our reading of the reports suggests the following to be important:

3.2 Activities

During the second year, most of the ANR projects moved beyond the uncertainties and difficulties of the start-up phase and began to consolidate their position. At least three of the projects felt they had moved to a stage of becoming an operational service rather than a pilot demonstrator. We discerned a broadly comparable set of activities among the main group of subject-based gateway projects, although the emphasis given to each reflected their idiosyncratic natures and development path.

These were:

- expanding resource description base and extending services

- developing a more sophisticated interface tool

- setting up links and collaborative ventures

3.3 Development and enhancement of services

The achievement of projects in developing the subject-based gateways to information resources is evident from the table below which summarises the range of resource descriptions and services offered by three different ANR projects.


SOSIG catalogue of resources

EEVL resources and services

RUDI resources and services

  • over 2000 resources records in the catalogue
  • databases and datasets
  • bibliographies
  • home pages of social science organisations
  • electronic journals and newsletters
  • reports and papers
  • educational software
  • digitised books
  • scholarly mailing lists and archives
  • separate database of Web pages of UK university social science departments
  • main database of over 2200 resources
  • search system to a 40-day Engineering Newsgroup Archive
  • trial access to subset of key database Recent Advances in Manufacturing
  • Harvest Index to UK engineering sites
  • Directory of members of University Science and Technology Librarians Group
  • Announcing service for e-journals
  • full text sample issues of SENN
  • links to 50 quality urban design sites
  • corpus of 820 HTML documents which reference 2350 images
  • information about conferences, calls for papers, seminars, exhibitions and other events
  • full text (from 1995) of the leading UK urban design journal
  • on-line journal contents
  • design schemes, city profiles, Urban Design Campaign documents
  • bibliographies
  • urban design course details
  • web based discussion forums


Strategies for expanding the resource database

Expanding the number of resource descriptions in the database is a central activity of all subject-based ANR projects, achieved with differing degrees of success. Projects reported on and sometimes reviewed the effectiveness of different strategies for reaching their targets. Alongside more formal strategies aimed at mobilising users as information providers, projects also emphasised the importance of ongoing ad hoc activity in following up opportunities from discussions at conferences or meetings, or via visits to the project from other organisations. Factors limiting the growth rate of content acquisition and resource discovery were in one instance attributed to an inadequate level of staffing to follow up potential information sources and contacts, and in another to the extensive reallocation of the Resource Officers’ time to standards development work and associated quality assurance procedures.





User involvement strategies for database expansion

  • OMNI sought to involve its users in building the databases by calling for volunteers to submit resource descriptions using the ROADS interface. After a promising start to the Volunteer Programme, with over 60 people attending the training workshops in use of the software, it became clear that the creation of records using ROADS administration tools is too labour intensive for occasional users and the rewards are not great enough. OMNI has since substituted a simple form for any user to submit a suggestion. In a short two-month period, 210 resources were submitted in this way, the majority being transferred to the public databases after the normal quality checks.
  • Biz/ed encountered similar difficulties in the level of resources generated by then Trusted Information Providers (TIPs) drawn on a voluntary basis from the Association of Business Schools. Despite a small honorarium and signed contract, it has proved difficult for many to contribute regular time inputs.
  • SOSIG attracted additional funding to institute a team of Section Editors, drawn from UK university libraries, who spend half a day per week each building up the SOSIG collection. A SOSIG Administration Centre has been developed as an online focus and entry-point for the editors who are responsible for identification and cataloguing of resources in different social science subject areas. The Centre provides supporting document e.g. on cataloguing rules, selection criteria, scope policy, collection management policy and guidelines on how to find social science resources on the Internet.
  • Concern about the amount of staff time involved in managing the proposed Trusted Information Providers programme led EEVL to delay its implementation. The project hopes that the employment of a Database Officer later in 1997 will allow its introduction in cost-effective way.
  • ADAM hopes to redress the shortfall in its resource description record target through the ‘Friends of Adam’ volunteer scheme.
  • Active involvement in the EU DESIRE project has allowed SOSIG to link up with the network of European correspondents, set up to supply information and resources cross-nationally

Developing a more sophisticated interface tool

Most projects have also devoted considerable efforts to enhancing their interfaces by fostering improved search possibilities.

Enhancements to the interface tool

  • OMNI sought to involve its users in building the databases by calling for volunteers to submit resource descriptions using the ROADS interface. After a promising start to the Volunteer Programme, with over 60 people attending the training workshops in use of the software, it became clear that the creation of records using ROADS administration tools is too labour intensive for occasional users and the rewards are not great enough. OMNI has since substituted a simple form for any user to submit a suggestion. In a short two-month period, 210 resources were submitted in this way, the majority being transferred to the public databases after the normal quality checks.



Enhancements to the interface tool (continued)

  • The initial search interface built with public domain tools from Excite was replaced by the Verity search engine which was licensed with the Netscape server software. This gives the user a choice between free text and Boolean searching and improves presentation of results. Proposed changes to the browsing interface will allow alternative views of the RUDI corpus which will be automatically derived from embedded metadata.
  • A new release of ROADS enabled the launch of a ‘new look’ for SOSIG. The SOSIG Section Editors and User Group were consulted during the development stage and the whole of the SOSIG team was involved in the design and implementation. Among the new facilities are: much more online information about the service as well as extensive help; more options for browsing including geographical parameters; and the incorporation of a Thesaurus to refine and extend search facilities.
  • ADAM has started development on a browser interface to its Service, in response to user feedback from evaluation activities.
  • An experimental version of an interface to the OMNI data using UMLS (the U>S> National Library of Medicine’s Unified Medical Language System) is being trialled by OMNI. This interface allows OMNI users to browse through Medical Subject headings in a logical manner, and follow links to broader, narrower or other related terms.
  • OMNI redesigned the front page of its Web site to allow more services to be directly linked to the main OMNI page, and to incorporate a search function into the main page. This addition of a ‘Quick Search’, a response to user feedback, was intended to ensure that the most popular elements of the OMNI service could be found rapidly by both new and regular users.
  • Biz/ed have been testing two new mechanisms for producing 239.50 views of the Biz/ed/ROADS catalogue, although there is so far little demand from users for the facility.


3.4 Promoting the project

Promotional work was a main activity of projects during the first year and has continued to be an important strand of work as projects become more established. In addition to the core promotional elements characteristic of nearly all eLib projects, several of the individual ANR projects have undertaken supplementary activities, giving them a distinctive promotional profile.

Promotional Activities

  • OMNI mounted a major annual seminar attracting 150 people, circulated ten issues of its Newsletter, contributed a regular column to ARIADNE on OMNI and other issues related to the Internet and medicine, and produced and distributed more than 10,000 copies of a 2nd edition of its popular guide to resources.
  • EEVL has achieved wide coverage in the media, both the library and engineering press, with a number of awards as testimony to its success. These include Library Hi-Tech ‘Best Library Related Site’, Information World Review ‘Best New Site’, Scout Report Collection, and inclusion in the Top 50 UK Web Sites.


Promotional Activities (continued)

  • SOSIG presented a ‘new look’ to the academic community with redesigned publicity materials. Over 8,000 copies of its popular Internet for Social Scientists desktop reference card were distributed to libraries and social science departments in universities through the UK. Collaboration with colleagues in the DESIRE project also led to production of joint DESIRE/SOSIG leaflets, bilingual French/English, which has been distributed widely.

3.5 Training and awareness raising

Several projects have continued to invest significant amounts of time and effort in the development and delivery of training materials whilst one or two others have relied mainly on the Netskills project to raise awareness and develop competencies in use of the Internet. Biz/ed has effectively built on materials developed in Sosig to produce training material relevant to the project. Some projects do not have a training component at all, either because of the nature of the project or as a reflection of the stage reached in their development.

Among the developments in the way training is managed and delivered, the following are noteworthy:

Training events continue to be a main source of user feedback on ANR demonstrator services, captured through various evaluation instruments. The data indicate that workshops not only encourage participants to use the services more but also to ‘spread the word’ to colleagues and other users.

3.6 Evaluation activities

Evaluation activities vary across projects, but most commonly effort has been directed towards user feedback to inform continuous improvement of the demonstrator service and the analysis of usage data gathered through log files. In several cases, evaluation forms have been simplified and on-line surveys abandoned in favour of paper-based questionnaires. ROADS is also making use of customised defect tracking and enhancement tracking software which will highlight those areas of most concern to end users.

Although project evaluators are cautious about the use of raw usage figures, trend figures are seen as a more reliable indicator of performance or usefulness. The four projects presenting usage data reported a steadily upward trend, with fluctuations broadly following the cycle of the academic year. Peak usage figures included an increase in genuine user accesses to 120,000 per month for SOSIG, and to 11,000 per week for EEVL.

The more fruitful evaluations have a longitudinal dimension that allows changing patterns of Internet usage among workshop participants to be mapped. By gathering baseline data on participants’ profile of computer and Internet usage prior to he training workshops, two of the projects with a shared evaluator have been able to assess how the workshop has changed participants’ use of the Internet as well as to explore the effect of subject based information gateways, over time, on the working patterns of end users.

Descriptive analyses of the data suggest that attendees at SOSIG and OMNI training workshops have significantly increased their use of subject-based services since attending the workshop. They have also increased their use of other Internet services. This increase in use exists regardless of the different levels of reported ability and use before attending the training.

The picture which emerges from follow up interviews with participants is that most do not plan a search beforehand, use only one word, less often a phrase, and only few people use Boolean connectors or truncations. Searches are most often based on starting from a known address and develop as they go along by modifying or adding words, using synonyms or proper names. The addition of the Thesaurus to the interface of several projects is expected to assist those who use a single search term.

3.7 Main areas of development

Facilities to support more refined searching

A main development reported by several projects related to the integration into the service of a Thesaurus to assist users searching the database to refine their search terms:


ANR projects have also implemented classification schemes to allow more efficient indexing and searching:


3.8 Metadata

Several of the ANR projects have played a high profile in the metadata debate and in the development of the Dublin Core (CD), a metadata standard which attempts to provide a common core set of elements which can be applied to a variety of information sources. Metadata, in the form of resource descriptions entered and indexed in a database, can then be browsed or searched by end-users seeking to locate high quality Internet resources in relevant subject areas. Although projects are following down the DC metadata path, one project questioned aspects of the usefulness of DC metadata in the context of its database service whilst another is maintaining contact in parallel with developers of the Data Documentation Initiative which potentially offers a different international standard to describe social science datasets.


3.9 Assuring quality of information resources

In parallel, additional efforts have been made to assure the quality of information resources:

In addition ADAM has instituted Quality Assurance Procedures in order to minimise the risk of errors in the database. This entailed the implementation of an additional ‘private’ database’, into which all new catalogue records are entered and checked by another Resource Office before being transferred to the public database.

The ROADS team has been responsible for much of the early deployment and concrete syntax specification for the Dublin Core, in addition to hosting the main DC working group forum. This illustrates that the UK community of librarians and other information professionals can play a leading role in the development of metadata standards and conventions for the Internet.

As tools have become available for validating to later standards, projects have taken advantage of them. Careful attention to detail in OMNI, for example, has ensured that it is possible to use the OMNI service from both the latest version of Netscape and text-based browser such as Lynx.

3.10 Implementation processes

Projects encountered a number of difficulties in the second year of operation, although these were not dissimilar from those reported in the first year annual reports. The main areas of difficulty reported by ANR projects concerned technical matters, staff and project management, and copyright.

Some critical lessons were learned by ROADS users as by ROADS itself Although many of the difficulties encountered by subject gateway project based on ROADS were experienced as technical shortcomings of the ROADS software, their origin was often socio-technical in nature and stemmed from inherent tensions in the development process. The conclusion drawn by one project that it was a mistake to base the service around a piece of software under development may be a valid one, but at the same time ROADS is also right in its analysis of the collaborative/symbiotic relationship which existed between it and the ANR services and the points in the development cycle where consensus broke down. ROADS sees in retrospect some possible strategies that might have dealt with differing expectations and conflicting requirements.

Several projects continued in the second year to be disrupted by staffing problems, with key staff leaving the project bereft of necessary leadership or technical expertise. The withdrawal of one of the partners slowed progress in one consortium, whilst another found that significant changes to the staffing arrangements have meant that roles, working practices and communication protocols all had to be reallocated, redesigned and relearnt. A few projects experienced ongoing staff management difficulties arising from the geographically dispersed nature of team members and the shortcomings that stem from having to rely on formal reporting structures to monitor progress and take corrective action rather than on direct first hand observation.

Copyright clearance was a problem for CAIN, relating to the particular type of grey literature included within its resource materials. Offsetting this difficulty has been a positive response from publishers to the use of substantial sections of text, including chapters from books, accessible from the gateway.

3.11 Sustainability issues for subject gateways

A number of the ANR reports commented on their perceptions of the changing market environment for subject-based gateways and the possible future for their project within this context. Among the developments noted were:

Likely futures for project sustainability

The ANR projects envisage different futures for themselves, tempered in part by how they see the longer term market environment. None of the projects has a well developed business plan, although they are beginning to identify what might be the key elements. Two projects have set income targets for the coming year.

The main sources of ongoing financial support, which would sustain projects to varying degrees, include: subscription income; revenue from commercial alliances for on-line services; income from products such as publications and events, consultancy and advertising; and central funding from JISC and relevant Research Councils and professional bodies. Most projects envisage operating in a mixed economy.

ROADS sees the potential to operate in a commercial environment beyond the ANR information gateways, by extending its toolkit and development services to a wider set of clients both in the UK and beyond. Within the market domain of subject gateways, ROAD maintains there is definitely a place for a free (or at least low cost) toolkit. It is likely that development will continue on an unfunded basis e.g. under GNU Public License and/or supported by charging for support.

EEVL has also given consideration to producing a more generic version for use with other services, but the project is aware of other systems such as ROADS and Isearch and other commercial database access routes now possible through the Web. Conversely, EEVL believes that if other systems offer features which cannot be incorporated into the EEVL software, it would be possible for EEVL to change its software base whilst maintaining its existing data. This has already been demonstrated experimentally by converting the data to operate with the beta release of ROADS v.1.


PROJECTS. Edbank; On-Demand Publishing in the Humanities (was ‘Only Connect’); SCOPE: Scottish On Demand Publishing Enterprise; ERIMS: Electronic Readings in Management Studies; Eurotext: A Collaborative Resource Bank of Learning Materials on Europe

4.1 Introduction

Within the ODP domain, seven eLib projects have been set up. One of these has already been completed whilst another only began in 1997 and so is reporting mainly on start-up activities. This synthesis of ODP project reports focuses for the most part on the four projects for which second year annual reports are available.

The OPD projects are engaging with a similar set of issues although the underpinning strategy varies from one to another. Two projects are primarily subject based, another is cross-institutional, a third is document source-based and a fourth might be described as system based (a Scottish consortium of 13 members). They range from those that are still grappling with technical issues to those that are mainly concerned with implementation. The projects are experimenting with different systems for supplying materials electronically, assessing the effectiveness of supply-side approaches to generating a bank of resource materials, testing users’ preferences and exploring the benefits from cross-institutional collaboration.

At the broadest level, the ODP projects raise a number of key questions about the viability of ODP, taking account not only of how publishers are responding to a changing electronic publishing environment but also of user (students and academics) behaviours and attitudes. To anticipate what seems to be emerging as the key finding from this group of projects, the costs of delivering ODP appear prohibitively expensive for a small-scale operation. To achieve economies of scale, with respect to the copyright clearance process as well as to digitisation, some centralised model would seem to be the way forward.

4.2 Activities

Whilst ODP projects have undertaken many of the same activities, their main preoccupations during the second year reflect how far they are along the path to implementing a demonstrator or full service in a real user environment.

The various activities reported on by projects include:

- the scanning process

- obtaining copyright clearance

- putting together course packs/databases and integrating them into an electronic environment


4.3 Materials/resource development

The scanning process

Several of the ODP projects are faced with converting hard copy materials into a digitised form. These include printed texts such as EU documentation, lecturers’ notes, and other course materials with a range of text, graphics and numerics. The scanning process involves a number of steps - scanning documents, re-formatting, spell-checking and proof-reading. This process has proved immensely time-consuming (and therefore expensive) as well as posing various technical challenges.

There is debate about what are the most appropriate database formats, with different approaches evident among the eLib projects. Projects seem to be equally divided between a preference for HTML and for PDF files, each having its own advantages and limitations. HTML for example, offers the opportunity to create hot links to WWW sites - an important consideration for updating database references. It is a much more expensive process however than mounting materials as simple PDF files. There is a view that PDF is emerging as the de facto standard for full-text documents on the Internet and it would appear that publishers prefer PDF and may therefore be more likely to give permission for their material to be used in this format.

Obtaining copyright clearance

Obtaining permission to digitise material belonging to a third party and make it available for electronic access and/or distribution has been one of the biggest and most common problems eLib projects have had to face. Electronic clearances can be very time consuming and expensive and many of the projects under-estimated the amount of time and money that copyright clearances can take. (Experiences and lessons concerning copyright matters are taken up below).

Actions taken by project to facilitate copyright clearance include:

Developing course packs and expanding database resources

ODP projects have taken different strategies to coursepack and database development. Those consistent with a demand driven model include:

On the supply side, strategies have included:

All the ODP projects have made attempts to integrate database materials into library catalogue systems, whether through their OPACs on the Web or a reading list on a Web server. One project is working to build in search facilities to support document retrieval on a subject basis in addition to the reading list mode of access. Linking in this way has presented a serious technical challenge and has to date defeated the institutional partners in most projects.

Encouragement to eLib projects to embed Dublin Core metadata in electronic resource materials has been taken seriously by the ODP projects, with most reporting that this presented few difficulties.

4.4 Developing systems for copyright management

Strategies for copyright management adopted by the ODP projects include both educational and technological approaches.

The SCOPE project for example has been working on the development of a document delivery system incorporating five levels of security for network transmission and on-line delivery. The software system is cross-platform and will not require libraries to invest in additional hardware. The On Demand Publishing in the Humanities team has adopted a four layer strategy based on IP addresses, access passwords to the LAN and course material, and simple stenography with HTML pages encoded with user ID. EDBANK controls access to the computer network through the central administration system. Publishers have made it clear that they consider IP address restrictions to be very important especially when delivering materials overseas.

4.5 Implementing field trials

ODP projects have experimented with a number of different subject areas including Management Studies, Education, Sociology, Economic, Engineering, Nursing and European Studies. In several cases, the intention has been to collaborate at a cross-institutional level both in the development of a shared database of published (and in-house) bibliographic


material as well in the sharing of teaching materials that have been put together by individual lecturers. Other projects have adopted a more localised approach, offering the facility of customising a set of resource materials that can be integrated with other sources.

An ongoing task for all project teams has been gathering reading list information, and ensuring that these have been updated to take account of new developments in the subject and new publications.

Various strategies have been adopted for increasing academic involvement in the coursepack/database development process. In Eurotext, for example, academics involved in teaching European modules were invited to participate in an Academic network. A closed list was set up on Mailbase, with a view to involving network members in identifying the key documents in their specific areas of expertise or to supply copies of their reading lists.

Some projects have found it necessary to adjust their strategy. In one institution, the preferred approach has been to liaise closely with individual academics, whilst in another the preferred approach has been to invite academics to joint the project team so that are not merely a client of the service but have a voice in its development.

4.6 Evaluation

The ODP projects have given prominence to evaluation, both formative and summative. Among the methods used are: focus groups with students and librarians, questionnaires and on-line evaluation. As a domain area, the ODP projects are oriented more towards implementation and the evaluations reflect the projects’ concerns with assessing impacts and effects - in areas such as the acceptability of online delivery of materials, changing patterns of library use, students’ searching strategies and use of materials, substitution effects (electronic resources for sales of texts) and so forth.

4.7 Promotional work

The ODP second year annual reports make little mention of promotional work, other than in a negative sense. One of the projects, reflecting on its experiences, noted that the need for liaison, marketing and publicity skills which are essential for a new project’s development were overlooked.

In common with the first year projects (as noted in the 1996 Synthesis Report, the new project in this domain, EDBANK, has been active in profiling itself and generating interest in its activities.


4.8 Achievements

Notable achievements of the ODP projects can be seen in the following Table:

Achievements of On-Demand Publishing Projects

  • consolidation of an ERIMS reading database with over 3,700 bibliographic records processed, and a corpus of Management Studies literature, with approximately 14,000 pages scanned and converted to full text
  • success in incorporating coursepack materials into a Web OPAC system (eOn)
  • agreements for licensing on-demand publishing in 13 SCOPE consortium members with more than 55 copyright holders
  • adoption of the SCOPE model agreement by a wide range of publishers
  • significant enhancement of the EUROTEXT interface to deal with the user-unfriendly characteristics of European Union documentation
  • establishment of an Academic Network of European subject specialists to advise on and guide the selection of materials (EUROTEXT)
  • license agreements with four leading commercial publishers (Blackwells, Oxford University Press, Routledge and Elsevier Science) (ERIMS)
  • development of a document delivery system incorporating five levels of security for network transmission and on-line delivery, using a cross-platform software system


4.9 Project findings

Some of the main project findings from evaluation studies are presented in the Table below. These are reported as individual project findings, although in places we have indicated where they seem to be common to other ODP projects on the one hand, or to suggest significant differences between projects on the other.


Key Evaluation Findings

The acceptability of on-line materials

  • Text based materials have not proved very appealing to academics and students alike. The demand appears to be for interactive materials which have been specifically designed for electronic publication.
  • Student satisfaction with the course packs appeared to vary with the proportion of lecturer-authored material, presumably because this type of material is tailored directly to student requirements. Students in subject areas such as sociology find pre-pared course packs to be less useful than those studying certain other subjects. In future, students that read from a wide range of sources are likely to benefit more from delivery on-line or from the facility to self-select the contents of a customised course pack.

The effect of course pack provision on library use

  • The effect of course pack provision on library use varies with the course. Library issues for books from short loan which were selected for an economics class supplied with SCOPE course packs fell by 46%; issues from the main collection also dropped. In contrast, library issues to engineering students supplied with SCOPE course packs barely fell but the base level was very low anyway suggesting that students rely on core materials distributed at lecturers or contained in texts that they purchase.

Pricing effects

  • ODP resource packs are very price sensitive.
  • publishers are concerned that ODP resources will substitute for sales of original hard copies. Generally, resources do not substitute for sales of texts. Students that buy resources tend also to buy texts and to spend money on photocopies. while copyright fees are transaction based, publisher revenue generated by the Project is related to payment per copy made. A second-hand market could reduce the income publishers may expect from SCOPE. Our limited experience suggests that there is no significant second-hand market for course packs in classes supplied with SCOPE resources.


4.10 Key lessons

Changing electronic publishing environment

A key issue for the ODP domain concerns the behaviours of publishers in the maturing electronic publishing environment. The SCOPE report has observed that ‘mobilisation in this area is evident in that many more publishers now have developed or are developing policies on licensing their copyright materials for electronic use in academic libraries’.


Unfortunately, this has not meant a more favourable environment to support the growth of ODP. Among emergent difficulties noted by ODD project are:

Some publishers it seems are keen to make decisions about licensing with reference to the forecast effect on sales of the course text at the institution for which an extract is selected.

Another development impinging on projects such as ERIMS which are building up a database of bibliographic resources is the launch by big commercial publishers such as Blackwells and Elsevier Science of their own systems for delivering journals in electronic format. In the long term, this is likely to mean that books will be more pertinent to the requirements of a system such as ERIMS.

At the broadest strategic level, the development of a more mature market for EP materials, as evidenced by the mobilisation of the publishers, suggests the need for libraries/institutions in turn to negotiate from a position of strength in numbers. It is instructive that SCOPE, negotiating on behalf of 13 Scottish consortium HEIs, has found the terms it agreed with publishers to be increasingly weakened in this new environment. The proposal for a centralised body to deal with copyright clearance would seem to be consistent with the experiences of ODP projects in this area.

4.11 The organisational context of ODP

Some of the projects in the ODP domain saw their project in primarily technical terms. It has become increasingly evident to these projects, as they moved progressively towards implementation and uptake of their coursepack materials, that the challenge was above all one of organisational and cultural change. Most importantly, the lack of a strong end-user involvement at all stages of the development process (a common feature of projects which we noted in the 1996 Synthesis Report) has proved a major inhibitor to project success.


PROJECTS. ACORN: Access to COurse Readings via Networks; ERCOMS: Electronic Reserve Copyright Management System; PATRON: Performing Arts Teaching Resources Online

5.1 Introduction

Five projects were funded by eLib within this domain, most of them starting up in the second phase. Thus the annual reports tend to reflect the issues that characterise the start-up and early implementation stages of a project life-cycle and which were clearly documented in the first Synthesis of Annual Reports. In this section, our analysis is based on three projects - of the remaining two, one had finished and the annual report of another was not available.

Each of the projects in this domain has some unique or special feature. Thus ACORN has involved a key publishers as project partner, enabling the project to explore the role of a third party agent in the rights management process. PATRON is unusual in dealing with multi-media in the performing arts area, raising not only additional technical issues pertaining to audio and visual but also to the more complex rights issues. ERCOMS’ aim is to try and ease the copyright negotiation process by building a generic electronic copyright management system for electronic reserve systems, to be tested in two partner institutions.

Aside from the technical challenges of systems development and integration, copyright is the main bottleneck for the electronic short loan projects (in common with other domains such as ODP and EDD). Managers have to negotiate with publishers for electronic use of copyright works and publishers are not prepared to give permission unless some means of controlled access and tracking of the use of their resources on the institutional network is in place. Whilst tracking copyright ownership has always been quite problematic, it is heightened in the electronic age. The projects’ experiences in this regard give some indication of the downstream implications of publishers’ present attitudes and requirements.

5.2 Activities

The three ESL projects have been engaged in a broadly similar set of activities relating to:

Most of their time has been taken up with the first three on this list, and our overview of project activities addresses these priorities. At a more systemic level, PATRON is an excellent example of a user-centred development process, with focus group discussion proceeding in parallel with development of the user interface and underpinned by a multi-stranded evaluation. This is noteworthy in view of our observation in the 1996 Synthesis Report of a widespread lack of awareness of user-centred design (and not merely user feedback).


5.3 User requirements assessment and (market) research

In seeing user requirements assessment as a key activity, the three projects have laid a strong base for the eventual uptake of their product and services. Examples of the techniques adopted for identifying user requirements, and the functionalities of the system, are given in the Table overleaf.

Electronic Short Loan: Techniques for User Needs Analysis

  • distribution of a questionnaire on the eLib and US electronic reserves discussion lists inviting user views on a) the need for a generic electronic reserves copyright management system, and b) the requirements for such a system (ERCOMS).

  • market research into existing electronic copyright management systems, including those under development in eLib, European Union projects and available on a commercial basis (ERCOMS).

  • focus group discussion involving staff and undergraduates in the dance and music departments and the library, with the aim of stimulating and bringing together ideas on what was required in terms of content and user interface and how the service could be achieved. Participants were provided with a one-page briefing document several days prior to the session, and tentative conclusions were drawn after each session which could be tested in subsequent groups. A summary report was produced at the end of the formative stage (PATRON).

  • production of a story board of the user interaction and a functional specification which were discussed and modified by the project team with reference to the departments and the focus groups which were simultaneously in progress (PATRON).

  • setting up of Participating Publishers Seminar as a mechanism for gathering an understanding of publishers ‘real world’ charges and attitudes towards charging. Other topics on the agenda were document security issues, copyright clearance and the role of an intermediary, and digitisation. A follow up questionnaire was sent to all participating publishers, and others from overseas, with a view to gathering detailed information (ACORN).

  • user survey of the Short Loan Collection in a University library to gather baseline data on its function and usage, including feedback from users on any difficulties they have with the current Short Loan Collection; academic staff survey to provide the project with information on the views and behaviours of academic staff with regard to the current paper-based system on reading lists and the current traditional library short-loan collection (ACORN)


5.4 Systems development and integration

The ESL projects have a variety of software solutions to fulfil the functional requirements, based both on proprietary software as well as existing WWW technologies. All three projects have been very aware of the need to reduce development costs and time through using off-the-shelf solutions as well as establishing clearly defined boundaries and interfaces. The important of adopting an international standard which implies a degree of compatibility with future developments has also been emphasised. The project annual reports make light of any technical difficulties that have been encountered, most of them it would seem having been fairly readily resolved.

Electronic Short Loan: Technology Choices for Systems Architecture

  • ERCOMS, has adopted Web control technology wherever possible, in contrast to the earlier round of eLib Projects (SCOPE, ERIMS) when only proprietary systems were available to meet the publishers’ conditions. A key feature of the ERCOMS software design is the incorporation of management tools for handling increasing numbers of rights transactions, so making provision for scaling up.

  • PATRON made the decision to source equipment and specialised software from one company, IBM, to reduce the implementation tasks and possible conflicts. The basic design of the system architecture was to adopt a modular approach to maintain an open system where modules could be changed or replaced. Although designing initially for PC and MAC platforms, the emphasis has been on the PC as an easier development environment.

  • ACORN has been developed using widely available and proven software tools.


5.5 Copyright matters

Copyright issues lie at the heart of the ESL projects. These concern initial rights clearance and ongoing copyright management. The approaches taken by projects to this sensitive area involve both technical solutions as well as negotiations with publishers to establish common ground and an agreed way forward. .Discussions with rights associations and publishers indicate there is a great deal of uncertainty about the whole area of rights management and the range of possible solutions, particularly in relation to network distribution from an electronic resource.

Rights clearance is particularly complex for PATRON, given that each of the media has a different set of associated rights reflecting different industry and historical backgrounds. For example, the concept of fair dealing is not widely accepted, and there are key intermediary bodies namely the rights collecting bodies. This project has negotiated its way through the minefield of rights through discussion of a proposed course of action (as recommended by professor Charles Oppenheim) with relevant industry bodies, rights associations and the publishers itemising the works concerned. PATRON has sought to establish agreement wherever possible with industry bodies, so eliminating arduous and time-consuming negotiations with individual rights holders. The approach adopted has been to suggest that publishers become participants in the pilot, offering them experience of multi-media and rights management with a well-defined group of users for a specified time.

ACORN has devoted a great deal of time pursuing copyright permissions from publishers and other copyright owners. An output of its work has been A Manual of procedures on gaining copyright clearance for journal articles, and the entire permissions seeking process has been analysed in detail in the Phase One Permissions Report.

On a technical level, the ESL projects have experimented with different solutions to copyright management, although there is ready acknowledgement that these are at a very early stage of development:.

The ERCOMS package makes provision, among other things, for different approaches to ‘defining charges’ such as licensing or metering. In addition to these simple models, the ERCOMS annual report identifies a number of other common business models that could be the basis for a charging system. Ongoing development is expected to make it possible to include within the ‘defining charges software module’ more creative business models as they emerge.

5.6 Findings

The findings that are emerging from the early experiences of the ELS projects relate broadly to three areas:

5.7 Experience with copyright clearance

The three ESL projects have had varying success in gaining copyright clearance for the material to be included in the electronic reserve collection. It is noteworthy that difficulties were experienced, even when it was established that clearance was sought for the duration of the trial only and for access to a limited number of students. As noted above, two or the projects actively sought publisher involvement in the trials as an opportunity for both parties to explore the issues in a safe, testbed environment.

The Table below summarises the experiences of the three projects, each of them dealing with a range of copyright (and non-copyright) materials. In the case of ACORN and ERCOMS, these included books and journal articles (primarily text but also figures and graphs) in a range of disciplinary areas in two different universities; in PATRON’s case, the material included audio and visual (dance).

Electronic Short Loan - Experiences of Copyright Clearance




  • trial period of 19 months
  • permission received for 75% of the 316 requested articles and books
  • only two nations to charge were UK and USA
  • no university presses charged, only commercial and Learned Society publishing houses
  • most popular form of charge was a royalty based on number of pages printed from the ACORN system
  • one permission requires 2.5 chases and 2 months to receive
  • one article an take 8 hours to OCR


  • permissions have been obtained for all types of media, but problems experienced particularly with ballet
  • more success with small companies, but problems with distribution companies such as Decca and dance companies such as the Royal Ballet
  • areas on which intermediaries/rights holders have concerns include: quality of reproduction, multiple rights, retrospective changes of rights, use
  • agreements are provisional for the trial only

OU testbed

  • copyright clearance handled by OU Rights Department
  • 47% response rate in 5 months for 36 requests
  • 64.7% of these were positive
  • almost all positive responses are for journal articles
  • only two books cleared so far (one for text only)

Southampton testbed

  • selection from each of 4 libraries on different campuses, covering different disciplines
  • materials included key texts, journal articles,, government publications, recent newspaper articles
  • copyright clearance obtained for most, with very few restrictions and no charges


5.8 Cultural change

The ACORN project is very firmly grounded in an understanding of current user behaviours (both students and academics) as these relate to short loan collections, and the changes that are required to accommodate the concept of electronic collections.

5.9 Demand and usefulness

Only ACORN is currently in a position to report on user feedback on the service provided in a real institutional context. Usage statistics, shortly after launch, revealed:

5.10 Sustainability

The three ESL projects see different futures for themselves, reflecting their different market niches.

ERCOMS has an elaborated business plan that identifies possible options for the commercialisation of its system based on projections of software sales and incomes. It sees the most appropriate exit strategy to explore a commercial partnership with a supplier experienced in library and publisher software markets.

PATRON believes the potential exists for applying its interface in other multimedia areas such as linguistics where audio and text are used concurrently. It is currently preparing an exit strategy and business plan.

ACORN highlights the problem of scalability and believes the sustainability of the service in its current form would depend on a reduction in digitisation costs, perhaps through consortia rates, and the reduction in permission seeking costs perhaps via a central clearing house.


PROJECTS. EDDIS: Electronic Document Delivery; SEREN: Sharing of Educational Resources in an Electronic Network in Wales; Infobike: Browsing and Reading Access

6.1 Introduction

Five projects were funded by eLib in the EDD domain, and this overview deals with the annual reports of three of them. One project, JEDDS, was completed earlier and did not furnish a second year annual report whilst the report of another project was still forthcoming.

The EDD projects are among the most ambitious of the eLib projects, both technically but also in service and organisational terms. EDD represents a very real challenge and the way forward for some projects lies in strong collaboration with commercial partners rather than a solo effort.

A main lesson to come out from the EDD projects has been the need for closer convergence and inter-operability between the individual projects in order to achieve a real value-added service. This has been recognised at an eLib programme level, and a measure of rationalisation and convergence has already occurred.

At a more strategic level, the experience of the EDD projects raises questions about what is the appropriate role of universities in areas of complex technical development where the level of investment and associated risks might more appropriately be undertaken by commercial software companies. It is also significant to note the overlap, and possible competition, between eLib and EU funded projects and the desirability of gaining synergy between these, as is now the outcome for EDDIS following its merging with DALI.

6.2 Activities and Progress

The progress of all three EDD projects considered here has been eventful, and to more or less extent redirected into new paths or development trajectories. Infobike was realigned as a technical feed to the JournalsOnline service for HE, the convergence between the two providing a national service to support the National Site Licence initiative for access to electronic journals. EDDIS underwent a major shift from a project strategy of in-house software development to a partnership with FDI in which the latter would provide the software leaving the consortium to concentrate on developing relationships with data and document suppliers and potential customers in HE. SEREN is looking to greater interoperability with LAMDA, EDDIS and JEDDS whilst also recognising the need to refocus on the organisational task of building a resource sharing consortium.

The realignments or redirections which have occurred have been welcomed by the projects as a fruitful way forward, notwithstanding some loss of autonomy or disempowerment for the university as partner in the case of EDDIS. A shared view is that the tasks taken on by projects were more complex than had been imagined, and that closer cooperation and convergence had the potential to benefit all parties. The decision to merge the Infobike and Higher Education Electronic Journals Access Service projects into a single project is expected to deliver clear cost savings as well as clarifying the roles of both services.


6.3 Achievements

The achievements reported by the EDD projects are mainly technical, reflecting the efforts that have been put into this area of project development. They include:

6.4 Difficulties and challenges

For SEREN, the main challenge confronting the project is organisational. The complexity of the institution building task was not appreciated at the outset and the imbalance in project activities towards the technical has left the project in a vulnerable position in terms of uptake of the service and its embedding in institutional cultures. No contact was made with the broadly based user group of participating institutions (beyond the lead partners) from the time the proposal was submitted until Spring 1997 and so involvement and contribution has been significantly lacking.

Infobike also faces difficulties in the delivery of electronic content, although from publishers rather than consortium members. This it attributes to the immaturity of the market, whereby the only publishers geared up to supply bibliographic header material, article content and subscription content relating to articles are those who already have their own electronic content delivery systems. The vast majority of publishers are having great difficulty in providing this information. Even Blackwell Publishers, with over 100 journals and a competent IT department have taken several months to load 200 issues of their journals.

As a result, Infobike has had to expend unexpectedly large amounts of development effort in massaging electronic content data, particularly subscriptions data, into the necessary formats for JournalsOnline leading to delays in providing a critical mass of electronic content within the market place.

For EDDIS, the addition of a commercial partner (FDI) to the EDDIS Board , although generally seen as a good thing, has introduced complications for business planning. Whilst the EDDIS product has been loosely spoken of a ‘free’ to the HE community, it will likely require not insignificant sums to sustain it as a durable product in a price sensitive market.

Both EDDIS and SEREN have reported difficulties with Z39.50 protocol and targets. In the latter case, the problems are technical and relate to the server, whilst for the former the difficulties stem from the unwillingness and incapability of third parties to provide Z39.50 targets. The availability of such targets is essential is EDDIS is to provide useful service, but is outside consortium control.

6.5 Evaluation

Two projects have undertaken evaluations as part of an iterative development cycle and the third plans to undertaken acceptance testing of the software and in-use evaluation of the product.

Evaluation findings from focus groups discussions in SEREN highlighted the problems that the project faces in not having consulted users more fully at the outset, Users expressed uncertainty about what its added value would be and concern at possible negative indirect effects as a result of introducing competition with BLDSC.

In the case of Infobike, the evaluation indicated user satisfaction with the navigation interface and document delivery, but negative comments on presentation, content and web presence. These findings have prompted a ‘makeover’ to the JournalsOnline interface that provides a more attractive ‘look and feel’ and more ‘punch’.

6.6 Future Developments

The future for the EDD projects is uncertain, given their stage of development and the significant shifts in direction which have occurred to date. Whilst the business case has been explored in one instance, the annual report notes that there is no requirement from eLib that projects should be self-supporting operating after the project, less so a profitable business.

SEREN is exploring the possibility of integration with the public library network, so providing an ILL mechanism for Wales. The funding model for this is unclear however, and no business case has been developed. Some possible spin-offs from the SEREN software suggest possibilities for a document distribution service around site libraries, and the adaptation of the ordering side of the system to provide a management system for LAMDA.

Infobike has not looked to the future beyond delivering Phase 2 which provides third party access to journals online through standard interfaces, thereby facilitating connection between JournalsOnline and services wishing to use it as a document supplier.

EDDIS is developing a business model that attempts to reconcile eLib’s requirement for free software, FDI’s requirement for a return on investment and the consortium’s requirement to be self-supporting. A business operation could be based around support, maintenance and customer services, and the elements of a marketing plan are being explored .




PROJECTS. CLIC: A parallel electronic version of an established journal - Chemical Communications; Internet Archaeology: an international electronic journals for archaeology; PPT: Parallel Publishing for Transactions; The Superjournal project: Development and testing of infrastructures and tools for Electronic Journal Publishing; Electronic support for Scholarly Communication; Electronic Seminars in History; Electronic Reviews in History; DeLiberations on Teaching and Learning in Higher Education; NewsAgent for libraries: a personalised current awareness service for library and information staff; JILT: The Journal of Information, Law and Technology; Open Journal: The integration of electronic journals with networked information resources; Sociological Research Online:

7.1 Overview

Electronic Journals (EJs) has been the most innovative and speculative area of the eLib programme, and an enormous amount both technologically (concerning the actual technical form of EJs) and conceptually (as regards new modes of scholarly communication) has been achieved since these projects began their work. As is their function as demonstration projects these projects tell us much about what can be done now, what scope there is for building on to current capacity and the obstacles to exploiting current capacity.

It is difficult to do justice in such a short synthesis to the range of project learning and interested readers are strongly directed to the individual project annual reports (available on the Web). However it is worth noting here the sheer range of different but interrelated development areas which these projects are producing significant learning:

Overall, it may be observed that the EJ projects are now achieving the status of less ‘pilot projects’ (representing lines of development which may be found wanting and abandoned) as ‘path finding’ projects whose learning is likely to influence a coming generation of EJ initiatives.

7.2 Technical approaches

All the electronic journal projects are engaged in a continuous struggle to reconcile different formats, languages, protocols and procedures in a context where every achievement may be immediately superseded by developments elsewhere (for example Java to name but one). Among the projects, some aim to make use only of established technologies and software to provide on-line journals only. Some seek to develop environments and applications to provide both electronic and print versions, which involves choosing between, upgrading, adjusting and linking different software applications. It is useful to focus on a sub-set of the EJ projects under review to point up some of the technical complexity involved here:


Examples of technical approaches


The CLIC consortium focuses on developing an electronic journal for an existing community of Chemists. The original aim of this project was to provide for a diversity of numerical and symbolic information, including information with no print equivalent such as three dimensional models and dynamic information. This community operates at the cutting edge in terms of (electronic) scholarly practice and state of the art software and chemists (or at least a sub-group of Chemists) are used to importing data in a variety of software formats. A particular aim of CLIC is to provide ‘on the fly’ (instantaneous and automatic) conversions from print to electronic versions and was achieved by the end of the project’s first year, including automatic placement of all graphics (as GIF images) and non-standard characters (as bit mapped images). The organisation of articles is also more complex than in paper based journals with a rich section and sub-section structure, which allows CLIC to create a ‘dynamically, hyper-linked table of contents for each article as it is converted’ or authored. The project is now seeking to exploit and make inter-operable the full range of data presentation and manipulation tools currently available so as to maximise authoring scope.




Examples of technical approaches (continued)

Open Journal

The open journal framework seeks to integrate electronic journals with networked information resources through hypertext links. The project is developing ways of supporting link authoring, link management and presentation of links in different document formats. The linking framework of the project uses a number of software tools: DLS (Distributed Link Service) - a server-side link management and inclusion facility that enables links to be superimposed on previously authored documents. For users this is a platform independent service that requires no software installation, performing the necessary link computing on the Web and allows user selection of links. PDF toolkit, including ‘plus-in’ applications to support one to one and one to many link inclusion in Adobe Acrobat documents. And software agents to support information retrieval and automate link creation (including a citation agent which performs text recognition and matches references within an article against a bibliographic database; and a Web search agent which turns the results of searches into link bases if requested).


The technical aim of transactions is to implement parallel print and electronic publishing of a journal preserving printed page fidelity in the electronic format and facilitating intra- and inter-journal cross referencing on a specified range of features for a readerships (Geographers) unfamiliar with electronic publishing. Like CLIC, Transactions aimed to automate the processes and to provide three dimensional and multi-media options. Transactions settled on specifying a (widely used) standardised electronic format EPS for the submission of graphs, diagrams and maps.


7.3 Editors, authors and readers

Editorial resources

Typically EJ projects only have limited resources for managing editors (e.g. Internet Archaeology has 1.5 days per week). This means a considerable burden has to off loaded on to authors (for example conversion costs) and unremunerated editors. More generally electronic journals challenge the ways in which tasks and roles have been allocated in the past, and seem to tend towards creating new (more demanding) standards for authors. This can be seen most clearly in the electronic journals which parallel print journals, where the question of who is responsible for tagging or marking up electronic copy is often a matter of dispute, but is in fact true for all electronic journals. Certainly the development of standard specifications for submissions has been a particular focus of a number of projects.


It is a commonplace that new journals, and all the more so electronic journals, have a problem in attracting ‘good’ authors (everybody likes to publish in the most prestigious disciplinary journals which are normally the longest established). However this is by no means the experience within the eLib programme: While JILT reports that submissions were at first slow in coming, both Sociological Research Online and Internet Archaeology report considerable success in attracting good quality submissions.

In the case of the two latter journals it is worth observing that in the case of Sociological Research Online, UK sociology has had for some time a pressing need for at least one more mainstream journal given the current scale of activity in the discipline, while in the case of Internet Archaeology the opportunity to effectively publish multi-dimensional material is being offered to the profession for the first time, emphasising the point that electronic journals need an additional selling point beyond simply that they are ‘on-line’. Yet it must also be said that all the electronic journals have had difficulties in persuading authors to include new multi-media elements - even CLIC operating from perhaps the highest skill base of all has expressed disappointment with the low take-up of the text enhancement features available to their authors.

One factor that continues to inhibit authors, highlighted by a number of projects, is the failure of the Research Assessment Exercise to adequately acknowledge electronic publication. The low status given to electronic publication in the RAE is also to be found at the disciplinary level, for example, the Chemical Abstract Society refuses to include abstracts from the electronic version of Chemical Communications until they have been published in the parallel paper version.


When we turn to the actual readership of the electronic journals, the raw figures reported suggest a very significant programme success: To give a flavour of this, 3,100 readers from 100 different countries have registered with Sociological Research Online and no less than 291 Web sites provide links to JILT.

In order to capitalise on this initial success in attracting readers most EJ projects are now attempting to ‘add-value’ to the offer to readers through providing more and more authoritative data content; enhancing text with additional multi- or hyper-media elements; providing greater flexibility in searching and browsing; and by improving turn around times to more speedily communicate research results.

However it is generally acknowledged that really attractive offers to readers still requires solutions to two more fundamental problems:

Firstly, the need for effective but non-punitive cost recovery whether by pay-per-view, subscriptions, site licences or some combination of the three.

Secondly, the need for standard multi-media set-ups for articles and standard software platforms for ‘reading’ those articles.

As has most clearly emerged in the SuperJournals project (which involves multiple journals and multiple publishers) finding compatible if not common solutions to these issues across journals can be difficult as reflects the different processes and procedures to be found in different publishing organisations. (It should also be noted here that while publishers and printers are described as stakeholders in many project reports, typically project evaluations seem to have little to say about the issues publishers and printers see as most critical, copyright maintenance and cost recovery).


Interactive journals

In the more interactive journals such as Formations and Deliberations where the traditional distinction between authors and readers are being broken down, readers appear to be uncomfortable with this challenge to traditional forms - ‘journals’ are something which you read, mail lists are somewhere where you discuss - or sometimes simply to existing ‘social’ patterns of communication. For example, while members of a mail list (ISL) were happy to use Deliberations as a source of reference and even as a source of new ideas to discuss, they continued to use the mailbase rather than Deliberations for that discussion.

7.4 Organisational Issues

Interactive journals also raise a raft of specific technical issues: (such as) selecting between users to provide targeted communications (Reviews in History) or end user configurations (Newsagent). Generally, the technical and organisational backdrop to projects continuously shapes the projects’ projected product offer and the characteristics of that product, inevitably projects are as much shaped by available technical capacity and competence as by editors insights or users’ expressed wishes. In one extreme example, key software features identified by the project design/editorial team remain unmet due to the absence of resource in the (separate) organisation charged with technical development.

In many other projects a multi-site division of labour has proved frustrating as the full complexity of the task in hand has had to be faced, as summed up in the following observation:

‘As last year, we continue to be surprised at how interrelated the various project activities are. Key activities can not be compartmentalised and ‘given’ to on project partner, they are a joint and co-ordinated effort. Last year we found this to be true in planning functionality. This year we are finding it equally true in planning the author research, the usage statistics to be gathered and how they are analysed’ (SuperJournal)

7.5 Conclusions

There is a degree of consensus reflected in the annual reports of the Electronic Journal projects about the necessary preconditions if real progress is to be made with exploiting the new technologies in innovative scholarly communication practices. These preconditions include;

While progress can be made (and is being made) in each of these areas, overall progress requires progress in all areas simultaneously. What is difficult for UK HE is that so much depends on developments beyond UK HE, in commercial software systems development, among publishers and in the international academic community. Nevertheless, the ferment of activity around EJs in eLib is likely to effect these external actors and CEI may wish to ensure that UK HE continues to provide a pathfinding role in this area. In any case, some attempt to relate eLib activities in this area to wider developments would be a very useful component of the envisaged eventual final summative evaluation of eLib.




PROJECTS: Internet library of Early Journals; DIAD: Digitisation in Art and Design; Higher Education Digitisation Service

8.1 Introduction

The three digitisation projects are grappling with a number of issues in common, although each is also seeking to resolve difficulties or challenges that are specific to its unique focus. Two of the projects are primarily oriented towards digitising a back run of journals, the one concerned with the design field and its attendant problems of high image quality and uncertainty about where copyright ownership rests, the other with archival material and the particular technical challenge of digitising material without dismembering bound volumes. The third project offers a digitisation service to clients in both the academic and commercial communities where it must in the longer term compete with commercial digitisation companies. At they approach the end of JISC funding, the projects in this domain raise central questions about sustainability, both in relation to the level of demand and the basis for cost recovery. The core concept of the services project, as a provider of customised or bespoke value-added services to the HE community, is also severely challenged in real market environment.

8.2 Activities

Among the main activities undertaken by the digitisation projects, are the following:

Although originally seen as a sequential set of discrete activities, projects now perceive these as inter-related and iterative. Some activities clearly have a more technical focus, but there is also recognition of the socio- dimension to what might previously have been seen as an exclusively technical matter.

8.3 Digitisation and OCR work

For the project in the Design field, this technical side of the project has been largely unproblematic despite having to deal with the higher incidence of colour in the later years of the journal back run. The second project, however, has faced continuing difficulties into the second year of the project stemming in the main from the inadequacy of the technology for dealing with grey-scale provision. Although promised an upgrade of the IMAX interface card that would provide full grey scale capability, the new card has been further delayed.

Much effort by this second project has been devoted to maximising image quality, through progressive procedural refinements, made possible by increasing operator skill in using the technology. The learning from this process has been captured in two reports, and a draft procedural manual has been prepared.

Another area where the second project has encountered difficulties and set backs has been in OCR processing. OCR throughput takes longer than the initial scanning process and is currently the bottleneck in the production process. Experience to date indicates that whilst OCRing will be feasible with some titles, the added value will be much lower with others and not possible at all in certain cases.

The service oriented project is heavily orientated towards the digitisation of textual content and reported no difficulty in this area of its operations.

8.4 Copyright clearance

Securing the copyright clearance of articles and images in the digitised journals has been the major pre-occupation for the project in the design field. The checking process is a time-consuming and labour intensive process, entailing a variety of search strategies for tracking down copyright holders. In many instances, particularly in the case of photographs featured in the journal, it is not always clear where copyright ownership resides.

The service project has similarly found copyright clearance to figure large as an issue for its client base. Although not initially part of the services provided to clients, it has emerged as one of the key elements of a consultancy service that would need to be on offer to clients.

8.5 Interface design and end-user feedback

End-users in the two journal/archival projects have played a role in the iterative design process, although primarily through providing feedback on prototypes rather than as active partners at different stages of the development process. Areas where users have had an influence include interface design, imaging methodology, indexing and the criteria of acceptable OCR quality.

It is noteworthy that none of the projects in the digitisation domain appears to have begun with a market assessment or needs analysis. Rather, having undertaken the technical work with some end user input, projects are now looking to establish through focus group discussion in different test sites what level of demand might exist for the services on offer.

8.6 Service delivery

The digitisation services project offers a range of customised valued added services to support the conversion of high volumes of learning and scholarly materials into electronic format. These include consultancy and advice, pre-production testing to define the quality and accepted standards to be agreed with the client together with costings, and initial work in identifying the copyright owners of the material.


8.7 Achievements

Achievements of the Digitisation Projects

  • the compilation of the copyright holders database containing circa 12,000 records (DIAD)
  • completion of digitisation of the prototype journal Design over a ten year period (DIAD)
  • Scanning of the first title of the early journals, Notes and Queries, with 26,000 images mounted on the servers (ILEJ)
  • Scanning completed for 18 volumes (nine years) of Blackwoods (ILEJ).
  • review of the HEDS concept and refocusing of the project’s development path


8.8 Developments

Network Developments

The increasing universality of access to the Web has led both journal projects to modify their strategy and take advantage of changes in available technology. In one case, this has meant abandoning the original X Windows interface in favour of a Web interface, whilst in the other consideration is being given to a networked version via the Web of the current CD-ROM prototype.

Some synergy in the design field is also resulting from links with the EU project MAID (Multimedia Assets for Industrial Design). It has developed a technological solution for networked delivery and the DIAD project was tested on high speed networks across Europe with considerable success. Both DIAD and MAID have benefited from sharing developments, due to similarities in technology, content and scope.

8.9 Cost implications

The cost of digitisation is an important issue although none of the projects is yet in the position to provide information of this kind.

This issue of cost is absolutely central to the HEDS project with its client orientation. Through focus group discussions, it has begun to identify the cost elements throughout the lifetime of the digitisation process. These include the cost requirements for different types of documents, the costs of accessibility (including also security of information) and preservation, as well as ongoing costs associated with future systems development and the migration of technology.

In seeking to extend its digitisation service and copyright clearance to images as well as text, HEDS is potentially confronted with a vast array of documents which it must deal with on a bespoke individual basis. To deal with the problem HEDS would need to provide some sort of pricing ‘menu’ such that material of a similar nature requiring similar treatment could be batched together.

8.10 The publishers’ perspective

Publishers’ attitudes towards copyright raised a number of important issues with regard to digitisation. Publishers clearly are concerned about the potential loss of revenue which could follow from making information freely available. Publishing on the Internet rather than on platforms that make it easier to limit access to certain sub-groups of the educational community is thus an area of critical concern. Stringent security measures, whilst favoured by publishers, are seen by the user community as discouraging access and inhibiting cultural change.

8.11 Sustainability

The two journal projects are at an early stage of formulating an exit strategy, including the viability of a commercial outcome. In the case of DIAD, one short term option is to link more closely with the ADAM gateway or to deposit the total database with VADS or AHDS.

The ILEJ project is exploring options for development beyond the period of present project funding in three key areas:

This scenario would require substantial resources from either commercial sources or JISC funding;

The costing data on the present experiment would provide the starting point for identifying the substantial resources that will be required for any expansion.

The HEDS project is able to compete with commercial companies because it presently benefits from substantially reduced tariffs through JISC funding. Even so, in the eyes of its clients there is confusion about the similarity between some services offered by HEDS and those offered by other services like TASI and VADS. These would need to be resolved through the project marketing effort and by forming closer links with other services. In the longer term, the HEDS business case would need to rest on the value added services it is able to provide to the HE community, beyond what the commercial sector offers. It is unclear at present whether its pricing structure could be competitive in the context of offering a bespoke service for the diverse digitisation needs of the academic community. In any event, it sees advantages to forming a strategic alliance with an established commercial copyright clearance service.


PROJECTS. DIGIMAP: National on-line Access to Ordinance Survey Digital Map Data; HELIX: Higher Education Libraries Image Exchange; MIDRIB: Medical Images: Digitised Reference Information Bank

9.1. Introduction

The small group of three Images projects were funded in the second phase of eLib. One of these, HELIX, was only in its first year during this reporting period. As such it was encountering many of the start-up issues commonly faced by projects, which were largely covered in last year’s Synthesis of Annual Reports. Helix aims to build a ‘general image database’ which can be delivered free at point of use to the UK HE community. However, many of the technical and business challenges involved in this are being developed in the associated ELISE project, as outlined further below under Activities. This report focuses mainly on the work of the other two project, which involve greater technical complexity in digitising their databases and which also bear responsibility for resolving copyright and demand issues. MIDRIB deals with a comprehensive database of medical images for use in teaching and learning, available nationally via the Internet; Digimap aims to provide staff and students with access to Ordnance Survey digital map data that is both timely and cost effective. Both are concerned with how these resources may be used in practice and to best effect, with differing types of end user in mind.

Although the two projects have commonalities with other electronic documents, their main challenge has been dealing with the complexity of the technical aspects that stem from the very particular characteristics of their databases, and with related issues such as copyright and models of data supply. In HELIX these issues are being addressed more directly by ELISE II with which HELIX is associated.

Both Digimap and Midrib projects have come to a better appreciation of the importance of seeing a specific project innovation in the context of the setting in which it is to be used. Whilst their preoccupation has largely been with the technical side in the first year of operation, they recognise the need to refocus attention on how the resource may be used in practice.

9.2 Activities

MIDRIB is following an essentially linear process of research, development and production. The main effort in the first year has been on research, leading on to development, while production lies ahead in the second year of the project.

Among the challenges to which the project must find solutions are: the sheer potential size of the resource, its distributed nature, the multiplicity of image databases to be incorporated, user registration and access control, online review, and the multimedia nature of the resource (video, sound, hypermedia as well as still images).


Principal areas of activity have been:

HELIX is largely a content-building project with delivery systems being developed in the associated EU-funded project ELISE (Electronic Library Image Service for Europe) also located at De Montfort. Such technical work as is necessary in HELIX is being carried out in accordance with models developed in ELISE II, the successor to the ELISE project. The content building encompasses a ‘general image database’ of 45,000 images, 15,000 from each of its three sites plus an additional database module specific to the Social and Political History of Britain from 1859 to the present day.

Decisions around digitisation strategies, identifying suitable materials and beginning the process of digitisation and indexing as well as preparing for the additional module have been the main activities in the first year. The technical challenges have been minor and successfully resolved, though there has been some delay. HELIX has also been confronted by a need to reconcile different goal aims and processes of academic and commercial partners.


MIDRIB - Activities

classification, thesauri and glossaries

work is continuing on the development of links to thesauri and concept-based linking using systems such as UMLS, greatly assisted by its close links with MeSH, the classification system at MIDRIB’s core

image delivery

ongoing research into the resolutions of image required for different output technologies; design on system based usage monitoring and caching to allow for the different versions to be generated on the fly to minimise disk space requirement


digitisation equipment has improved over the period of the project at a dramatic pace, allowing the project to meet its requirements for higher resolution and improved productivity through automation.

delivery platform

experimentation with certain sets of images made available on a CD interface so that they can be networked locally thus limiting external network traffic. CD-ROM is seen as offering an alternative delivery mechanism for tools enabling lecturers and students to generate their own teaching and learning materials from the resource.

interface development

evaluation data from focus groups and interviews has been fed into the initial interface design for CD-based materials, following sound HCI principles.


a standard operating procedure has now been specified to govern all stages of the collection process from the initial location of a potential donor through obtaining licensing agreements to the final submission of images and data to the database.

cataloguing and indexing

an interim cataloguing and indexing system, using MeSH as the basic classification system together with UMLS to overcome the problems of searching across a number of systems,, has been implemented in the database

development of tools for teaching and research

very early testing (at over 20 sites) of some basic models of tools to assist teaching staff in the development of lecturing materials and simple CAL


For Digimap similarly, the main focus in the first year of the project has been on the technical side in developing a working trial service demonstrator but is now shifting to one of service implementation with the launch of the service demonstrator and a consequent concern with observation, measurement and evaluation.

The technical and data issues which had to be overcome by the project stem from a mismatch between the way in which OS data is structured and user needs. Whereas user groups desire maps to be digital facsimiles of OS paper products, OS does not include that level of symbol specification with the digital data they supply and do not wish to provide this (possibly for copyright reasons). Second, each OS product is structured in a slightly different way, making it difficult to manage in an automated way as well as confusing users when they try to use the data. While it would be technically possible to edit/restructure the data, this is not feasible given the resources available. Nor would Digimap wish to deviate from a policy of minimum change to the underlying data provided by the OS. Thirdly, the service presents an OS product view and requires users to be familiar with OS digital products. Users however want a geography based view which is simple to use and hides questions of scale and product from the user. This represents an intellectual and design challenge, currently beyond the resources of the project.

The main challenges faced by the project have to do with managing the expectations of different potential user groups; dealing with digital map data which is subject to constant revision; and reorienting the user interface in the prototype to allow for retrieval by geographic and topic content rather than publisher’s product.

Digimap is characterised by a strong user-led development approach, employing an action research methodology. Its principal areas of activity during the first year underline this concern with building ownership of the project with participants in the test sites, and with enabling the central project team to learn about and understand the anticipated structure of demand and the local infrastructure at participating sites.

Digimap - Activities

setting up six test sites

  • site vistas and liaison activities
  • purchase and installation of equipment

development of service demonstrator

  • 4 prototype versions of a Java based system
  • continued improvements to the user interface/usability

selection and acquisition of data

  • liaison with user groups on a ‘shopping list’ of data needs

preparation of promotional material and user documentation

  • on-line tutorial for new users, and an on-line help facility

liaison with map library and academic communities

  • presentations, preparation of web pages, demos

9.3 Achievements

9.4 Issues

Ethical issues are paramount when considering the distribution of medical images. An electronic environment complicates further what is already a difficult area. In the light of expert clarification of some important ethical issues, MIDRIB withdrew the documentation it had prepared (guidelines to donors and clinicians, patient information sheet and patient consent form) and which had been cleared by the relevant ethical committees, and is beginning again.

Within such an environment, registration and system security are therefore extremely important. This has posed a potentially enormous task for MIDRIB with a possible 800,000 users who must be verified as legitimate viewers of the resource prior to registration. The NISS model of user registration is currently being explored, and several separate trials are under way to investigate further the system security aspects and to investigate performance limitations.

Quality control and assurance, both of the images and the associated metadata is key. The MIDRIB collection system provides QA phases for digitisation, image registration in the database, and catalogue and indexing processes. This is supplemented by the facility for on-line review and comment by visitors to the site. ( a form of online peer review) and subsequent analysis by the team.

A key issue for Digimap, in common with some other eLib projects, has been whether to continue to develop bespoke software for the link to the WWW or to buy into the (less satisfactory) technology now on offer. The current commercial technology, offered by GIS vendors, offers neither the flexibility nor the capabilities that are available within the JAVA-based software developed by the project.


9.5 Key lessons and findings from MIDRIB and Digimap



  • a recognition of the level of basic research that is needed in a project of this complexity
  • the stringency of management required to keep the project on track. (In the case of MIDRIB, a collaboration breakdown with one of the parties withdrawing from the project, has provided an opportunity to look more closely at management issues)
  • the need to envisage the context of use at the outset, and to integrate an understanding of how the resource may be used in practice, into the design process rather than as an end stage activity.
  • the importance of training and awareness on the use of the collection running alongside the project, allied with the need to select images on the basis of their relevance to the curriculum


  • better understanding through discussion and monitoring of what is involved in experimental projects
  • awareness of the considerable complications of creating an effective and user-friendly front-end to OS data
  • the necessity for institutions to commit sustained technical support from the earliest stages
  • the unacceptably slow response times of web-based applications
  • appreciation of the common pool of experience built up by the project which librarians can share and learn from, particularly useful where they are not map specialists
  • the need to dedicate a nominated member of staff with map knowledge to the project in order to gain maximum benefit for the users in the library


  • The issues involved in specifying resolution, harmonising equipment and developing workflow strategies in digitisation were considerably harder to resolve than in any other area of year one research. Medical images are extremely varied, and require a wide range of criteria to be met.


Links with other projects

MIDRIB has forged links in a number of areas of activity with OMNI, the other eLib project in the medical domain. Planning is underway for joint efforts in training events, evaluation and publicity. OMNI and SOSIG have also been used to test MIDRIB’s alpha model search engine. HELIX is dependent on the ELISE II project which is managed by the same team and follows on from the ELISE project of 1993-95. This project is working on producing a robust working system for the interconnection of image banks and addressing the range of factors involved including: metadata and indexing; database structures; Internet protocols; copyright and IPR; licensing and charging models; business planning; content building.


Project management

Both MIDRIB and Digimap reported management and organisational difficulties, relating among other things to the availability of staff with the requisite profile, and the unanticipated demands that good management makes on a dispersed, multi-partner project. HELIX has also suffered from staff changes but are confident that they now have a stable team.

In the case of MIDRIB, a key partner pulled out from the project consortium which necessitated a certain amount of restructuring but which also consolidated activities to a more limited number of sites within easy geographic reach. Digimap gained from establishing a partnership with OS (and also adding another trial site partner), after an initial period in which it was considerably hampered by JISC-OS negotiations over the supply of data to HEIs.


Evaluation to date has taken the form in both MIDRIB and Digimap of an ongoing iterative process throughout the project, involving such techniques as questionnaires, focus groups and individual interviews. The results have been fed back into the research and development processes. HELIX planned to begin its evaluation activities in October 1997.

Project futures

Both Digimap and MIDRIB face uncertainty about the future. In the case of MIDRIB, there are legal complications following the withdrawal of one of the partners whilst for Digimap the uncertainty relates to outcomes from the JISC-OS negotiations, technical developments by commercial and academic competitors in the field, and the role of other developments in this area.

Digimap is however able to go further in exploring a number of options which include the possibility of collaboration with the British Library, the use of Digimap in public libraries, and further collaboration with Glasgow University Computing Sciences Department .

HELIX has already undertaken to make the images delivered available to the UK HE community free at point of use for four years beyond the end of the project and at the time of writing were confident that the framework being developed by ELISE II would be in place within the necessary time frame.


PROJECTS. Ariadne: A parallel Web and print newsletter for librarians and information scientists; EduLib: Educational Development for Higher Education Library Staff; Netlinks: Networked Learner Support; Netskills: Network Skills Training for Users of the Electronic Library; TAPin: Training and Awareness Programme in networks

10.1 Introduction

The Training and Awareness projects demonstrate a number of different strategies for promoting and sustaining cultural and organisational change as well as for fostering the professional development of librarians in a new information age. The analysis which follows is based on five of the T&A projects, the annual reports of the remaining two projects not yet being available.

The strategies of two projects are largely predicated on the idea of cascading, with training and support focused on key individuals who are expected to be agents of cultural change and promoters of professional skills development. In both cases, the methods adopted include primarily face-to-face training events, supported by Web-based resource materials. Another two projects have adopted a strategy aimed at making a mass impact through delivering large numbers of workshops and seminars or through the development of networked training opportunities, aimed mainly at individuals in library/information services. Both projects make use of networked learning resources, one almost exclusively so. The fifth project, an authoritative parallel publication (print and Web versions), uses the media to inform, to stimulate debate and to strengthen the quality of Internet information services and resources of use to the profession.

A key finding for three T&A projects has been the limitations of its strategy for achieving the desired changes, coupled with an increased awareness of the importance of institutional factors and the more immediate organisational context as constraints on what can be achieved.

Two T&A projects, Netskills and EduLib, are the subject of an external impact evaluation study undertaken by IMPEL 2, which is seeking to establish how far the projects have been successful in consolidating and disseminating the professional development gains that have accrued to the individual participants so that they are more fully embedded in an institutional context.

Perhaps more than any of the other domains, the Training and Awareness projects have identified areas of complementary strength and made ready use of the training materials, workshop programs and other resources in support of their own strategies.

In the Table below, we summarise what are the distinctive approaches or strategies of the four T&A training-focused projects during the second year, including their target group(s), the methods used, the support services on offer, and the main outputs.


T&A Projects: Training Strategies and Approaches






target group

Development Team members cascading training to self-selecting librarians in First Wave programme

HE community; (Stage 3) more specific focus on information managers and trainers

library/IT community; teaching staff with interest in NLS

subject librarians in the six partner institutions, cascading training to a core of ‘champions’ within each subject department


pilot programme of 18 workshops for Development Team members

regional programme of 9 one-day workshops with additional ‘taster’ sessions

over 180 workshops and seminars

Web based Internet tutorial (TONIC)

distance learning course on NLS delivered on-line

4 workshops and follow-up on-line conferencing for staff interested in coordinating NLS institutional reviews

40 hr training program conducted in a networked computer lab

customised teaching delivered to individual academics

support services

optional linked assignments (possible APL accreditation)

Web-mounted portfolio of Training Kits on networked information topics for use by people with a network training role

Network Education and Training Electronic Gateway (NETEG)

‘Facts Sheets’ reference documents

Web site:

- resource base

- NIs Forum

Discussion List

Web pages

personal support from the Research Team

co-participant support through inter-university visits

main training elated outputs

training materials (will be) available in public domain

network training materials freely available to UK HE community, licensing scheme for wider uptake

document on Quality Assurance procedures

- review paper on


- annotated

bibliography of

current literature

- current practice

case base

a Model for delivering support to academics

materials and tools for use by librarians in training

10.2 Activities

During the second year, most T&A projects have moved from laying the groundwork including training needs analysis, institutional audits, course design, preparation of materials, and so on, to the delivery of a workshop programme and other training support services. The main activities reported by the various training-focused projects were:

10.3 Development of training materials

Most projects continued to be involved in developing and updating training and support materials, reflecting both the constantly changing world of IT and the consequent need to include new developments in networked information, and also a recognition of the need to make provision for new target audiences (e.g. for those taking on a training role, or individuals wanting intermediate or advanced skills in this area).

The training materials have in all cases being developed by an extensive and continual cycle of development, delivery, feedback from workshop participants and trainers, and subsequent modification. In some projects, this development took place in parallel with training staff getting up to speed and delivering training; in others, there was a clearly defined pilot stage before the materials were used with the intended target audience.

The training materials and support resources produced by the T&A projects can be grouped as follows:

10.4 Delivery of workshops, seminars and on-line training

Much of the second year of the T&A projects has been spent in delivering training in different sites across the UK in a face-to-face teaching mode. The workshops and seminars have generally been conducted on a regional basis so allowing reasonable access by participants across the HE system.

Activities range from training sessions open to all comers on a self-selecting basis, to events that are targeted at a specific audience such as information managers or subject specialists, to a full program of workshops mounted weekly over a 9-week period for library nominees.

The scale of training activity varies widely, reflecting the different strategies adopted by the T&A projects. Netskills, for instance has delivered more than 180 workshops and seminars to 6,000 people, giving a total of 18,000 hours of training. EduLib involved 136 librarians in its extended workshop program, with the expectation that the participants would in turn cascade their knowledge and skills to colleagues and academics back in their institutions.

One project reported a demand driven shift away from the general introduction course, to subject tailored introductory courses, authoring courses and other specialist courses.

Another found that the academics identified as ‘champions’ in their subject area were not interested in generic training modules put together by librarians, but instead wanted individual, tailored sessions that were responsive to their needs at that particular time. They had little time for discovery and exploration, being primarily interested in core resources.

NetLinkS has designed and piloted the interface for a distance learning, networked professional development course on NLS. The course will run for 16 weeks from September-February and can accommodate up to 40 participants. Other resource materials for use in training are mounted on the project’s Web site.

10.5 Formative evaluation and quality assurance

The T&A projects have all devoted considerable resources to formative evaluation as well as laying the basis for a summative evaluation. Ariadne, the publication project, has used focus groups to good effect along with a ‘snapshot telephone survey’ of its professional librarian readership. Data from access logs has yielded useful information for several projects, especially when used in conjunction with other qualitative data.

Course level evaluation is a common activity, based on feedback forms and an on-line variant. Participants’ and trainers’ views on the workshop have generally been gathered at the time of the event, and in one or two projects there has been a follow-up request for evaluative information a month or so later. Netskills for instance made provision for on-line discussion with and between participants following the workshops. The scale of formative evaluation activity can be demanding on participants no less than on evaluators, as EduLib has found. It has used NUDIST software to analyse more than 1000 evaluation proformas that have been returned from participants at its workshop programme. Whilst formative evaluation is very important as a means of continuous improvement, there is a point at which the collection of such data ceases to provide a significant return on effort.

Other evaluative activities have focused more on the input side. Netskills developed extensive but practical quality assurance procedures so as to maintain the high quality of its products given the number of trainers producing and delivering materials in a ‘mass training’ context. The QA work, presented in a document that has been revised and updated in the light of experience and changing requirements, covers planning and designing materials, proof reading and house, and piloting. In addition, the project sought feedback on its course materials from external trainers and Advisory Group members, as a further input into the QA procedures and to the materials themselves.

Efforts have also been made to look at the impacts of the training and awareness activities on longer term attitudes, behaviours and practices at an individual and institutional level. IMPEL2 is charged with this task in the case of two of the T&A projects, working in collaboration with the project evaluators. TAPin on the other hand has adopted a different approach to impact evaluation, based on a change process model. The research team, in the course of gathering evaluative data in case studies of the five partner institutions, has sought to engage participant librarians in dialogue and discussion about the changing culture of the library and their role in this process as change agents and opinion leaders

10.6 The launch of new services

Among the new services on offer by two T&A projects are a Network Education and Training Electronic Gateway (NETEG) that was publicly launched in May 1997 after a trial period, and a moderated Mailbase discussion list as the central platform for the development of an on-line community of interest around NLS which was established in September 1996.

NETEG provides support for trainers who are delivering their own workshops and courses in the use of the network through giving access to a collection of network training materials and other resources. As well as building up the database of resources, the project team undertook a reviewing process to establish the quality and consistency of resources and their descriptions. One output of this activity has been the development of quality assurance procedures for entering and evaluating resources.

The NIs-Forum Discussion List, which has to data attracted 925 members, now hosts a programme of ‘mini-conferences’ in addition to the more usual open format for unstructured exchange of information and ideas. Four mini conferences have been held so far, facilitated by invited discussion leaders, and attracting UK and international contributors from library, IT and teaching staff with an interest in NLS.

10.7 Some key findings

In addition to the evaluation findings reported by projects that relate to uptake of project materials and services and to participant feedback on aspects of course delivery, most of the annual reports also draw out some key findings or raise issues that are important for the project’s future direction.

One set of findings from different projects relates to the changing role of the librarian:

Another set of findings raises questions about the likely longer term impact of T&A activities on institutional behaviours:


One project among the T&A group reflected on the importance of a ‘reflective culture’ among the team as an essential ingredient for innovation projects such as eLib, and the integration of that learning into ongoing development:

The valuing of professional exchange was highlighted by one project as an important finding for eLib:

10.8 Lessons

In a number of annual reports, it was evident that projects had taken time to stand back from their activities and reflect broadly on their experiences. In at least three projects, this process of reflection had led projects to question the effectiveness of their overall strategy for achieving their individual project goals. In two cases, projects had come to realise the limitations of an individual-centred approach when seen in the broader context of the institutional changes needed to support a wholesale shift to a networked environment. In another, the project saw clearly that to move from pilot status to a real marketplace would require a partnership with a publisher.

The originally proposed strategy of appointing and working with change-agents within a limited number of institutions, that could act as demonstrator sites, might have fulfilled the Project’s objectives more satisfactorily. ‘

10.9 Inter-project links and influences

The T&A projects have a number of cross-project linkages, most of which are seen as beneficial to the parties involved. This can be at the level of Steering Group membership, as is the case with two projects which are represented on the SG of another project. More commonly, however, the T&A projects have made effective use of one another’s materials or training courses for their own participant/contributor group. NetLinkS and TAPin, for instance, have cooperated on training and future collaboration is planned with Netskills.

One project reported that it had not found the other T&A projects within eLib as useful as id did in year one. The management team felt that the training provision on offer did not take sufficient account of advances in IT, nor was it well suited to those wanting intermediate or advanced skills in working with networked information.

The projects involved with IMPEL2 found this to be a valued relationship, both adding an independent perspective on the impact of the T&A projects as well as relieving projects of the burden of a summative evaluation. The relationship was a collaborative one, and seen as mutually beneficial.

10.11 Sustainability/exit strategy

The T&A projects are at different points along the path of considering their future sustainability or the appropriate steps in an exit strategy. Some are actively pursuing options as part of putting together a business plan; others expect to give serious consideration to the project’s future in the coming period, and one project makes no mention at all of the post-funding period.

Among the options for a sustainable future under consideration by the various projects are the following:

As part of an exit strategy, the institutional partners of some projects have offered to maintain certain services funded through eLib on an limited term basis. This would include the updating of databases as well as the amendment of training materials in the light of emerging technologies and changing circumstances.

11. Reflections on eLib Second year Annual Reports

11.1 Benefits of eLib

When we consider on the content of the eLib second year annual project reports, it is particularly striking how we are now beginning to see some of the benefits anticipated in early FIGIT decisions actually emerging in practice.

We see, for example, the formative impact of project management workshops, evaluation workshops/clinics, supporting studies, and the demands of annual reporting itself, on project organisation and behaviour. This is particularly so in the greater emphasis given by projects now to user centred design approaches, good project management, and decision making informed by evaluation.

We may reflect that a) there will be wider benefits for HE as a whole as the learning and organisational skills first acquired in the context of the eLib projects is applied eventually in other projects and situations, particularly if these improved standards of operation are reinforced by explicit stipulations (by JISC and by individual HEIs) when funding future activities.

And b) that while a great deal of individual and organisational learning has taken place simply by virtue of the space for practical learning created by the eLib programme, much of the improvement in project organisation observed in the second year of the programme has also taken place by design. (This, of course, was key to FIGIT’s overall programme management strategy - provide projects with funding for activities but also provide the support necessary for undertaking those activities successfully.) Hopefully, we will now see the further beneficial effects of a reinforced emphasis on organisational issues in the selection and support of projects in Phase III of eLib.

We can also observe through the project reports actual mobilisation and catalytic effects of the programme - on researchers, publishers, EU programmes, and individual disciplines and HEIs. And the strategy of broad engagement through a large number of projects adopted by FIGIT, allied to the role of the Programme Director and Assistant Director and various FIGIT (now CEI) members as ‘missionaries’ for the programme, as well of the extensive round of workshops and conferences arising from, inspired by or drawing input from eLib, seem to have cumulatively created a ‘buzz’ and climate of expectation (not least in JISC and the HEFCs themselves) which is beginning to effect HE at many different levels from policy making to actual library provision and even, to some extent, research and teaching practice.

However, it be wise not to take these gains for granted nor to simply consider them to be the inevitable but somehow un-planable effects of an large initiative like eLib - Other HE initiatives in the past have struggle to achieve such like broader effects, and some reflection on why and how particular eLib elements have ‘worked’ (and others have not) and the implications for future initiatives is probably now timely.

It should also be noted that eLib has not managed to provide ‘solutions’ for a number of problems which threaten to become perennial in JISC programmes. These are:

Staffing: The short term contracts associated with project based funding inevitably create problems of staff recruitment and retention: JISC might usefully consider how creating more continuity of employment through better secondment arrangements, bundling of projects, or rolling funding of ‘core’ teams (e.g. UKLON, Netskills), might ease some of the human resource problems in programmes like eLib.

Exit strategies: It is difficult for individual projects, often with limited funding, short timescales and demanding ‘deliverables’ to generate effective exit strategies in the context of a difficult and rapidly changing external environment: There is a need to move beyond merely calling for business plans and exit strategies to more strategic interventions which provide evolutionary strategies for areas (ANR, EDD, ODP and so on) as a whole.

Cultural change’: Every attempted innovation brings in it’s train the need for cultural change: The arrangements (CALT, JISC Assist etc.) made by JISC in the last year to address the need for a greater concentration on training, awareness and cultural change issues, will be important in addressing this challenge.

11.2 Access to Network Resources

The ANR projects have accumulated a mass of practical experience on providing subject specific Web resources, covering issues such as user needs, suitable interfaces and technical strategies as well as providing quality assured access to a significant mass of actual information.

However, the future of individual ANR projects will in part depend on broader JISC/CEI developments around subject based resources and broader decisions to be made around developing the distributed national electronic resource. While there is a need for JISC/CEI to take some decisions across the board about ANR projects, it would be to misunderstand their nature as subject based services not to also relate any further core funding to the particular nature and needs of their disciplines especially as regards convergence and complementarity with other disciplinary initiatives.

11.3 On Demand Publishing and Electronic Short Loan

We have observed that the future of individual ODP and ESC projects will largely depend on broader developments in electronic publishing beyond their immediate control yet which they themselves have been catalysts for. Beyond these catalytic roles and their important contribution to increasing knowledge in this area, the most important result of these projects may well be something they paid least attention to at their outset, namely, clarifying the requirements and increasing the preparedness to undertake the necessary changes/’re-engineering’ in library and teaching practices that will be required if the potential of electronic publishing is to be effectively exploited in HE contexts. Hopefully elements of this learning will be seen to be applied directly in some of the eLib Phase III hybrid library projects.

11.4 Electronic Document Delivery

Even more strikingly than in the case of ODP/ESL, the EDD projects have moved on our knowledge of and understanding about the technical, economic and organisational requirements for such new services to emerge. As with the case of ODP/ESL, it is by no means certain that actual sustainable EDD services will emerge within UK HE, but it is reasonable to argue that the actual event of sustainable EDD services to UK HE has in any case been significantly brought forward by the pathfinding activities of these eLib projects. The paradox of the EDD projects, as well as that of the ODP projects, is that the emergence of commercial competition is likely to make HE based services difficult to sustain, but if such competition is called forth this can only be seen as a eLib programme success.

  1. Electronic Journals, Pre-prints and Quality Assurance

We have observed that there is a degree of consensus reflected in the annual reports of the Electronic Journal projects about the necessary preconditions if real progress is to be made with exploiting the new technologies in innovative scholarly communication practices. These preconditions include;

While progress can be made (and is being made) in each of these areas, overall progress requires progress in all areas simultaneously. What is difficult for UK HE is that so much depends on developments beyond UK HE, in commercial software systems development, among publishers and in the international academic community. Nevertheless, the ferment of activity around EJs in eLib is likely to effect these external actors and CEI may wish to ensure that UK HE continues to provide a pathfinding role in this area. In any case, some attempt to relate eLib activities in this area to wider developments would be a very useful component of the envisaged eventual final summative evaluation of eLib.

  1. Digitisation

The core concept of the digitisation projects as niche providers of customised or bespoke value-added services to the HE community is likely to be severely challenged in a real market environment. However, the need for digitisation services within HE will persist until time as competitive commercial services emerge and publishers begin to digitise their own back catalogues. A steady supply of ‘business’ from projects within HE will be important to the continuance of the digitisation services.

  1. Images

It would be panglossian to imagine that the diverse imagining needs of academic researchers will ever be fully met by commercial offerings alone. The eLib imaging projects provide us with useful prototypes of future HE based specialist imaging services, and regardless of their own individual sustainability offer valuable examples for other services that will be needed within HE.

  1. Training and Awareness

We have observed that the various eLib T&A projects have emerged as important resources to UK HE in their own right. There are clearly opportunities here to continue to exploit this capacity beyond the lifetime of the eLib programme by commissioning additional targeted efforts in other associated areas.


Appendix 1: Format for second year annual reporting


Format for 2nd Year Project Annual Reporting (August 1997)


As innovative projects from which others are expecting to learn, it is important that eLib projects provide information and knowledge that will be speedily accessible to the wider community. This annual reporting structure is one way of ensuring that the lessons emerging from monitoring and evaluating the progress and success of the project are recorded, systematised and disseminated.

The framework elaborated below is intended to provide eLib in its overall management role with a consistent and coherent set of data from all projects about activities and progress, the process of implementation, reflections on what has been learned and revised understandings and expectations about the project innovation. Regular reporting of this kind is also useful for project self-evaluation and reflexivity among the partners about what is being learned.

The framework takes the form of a set of leading questions that are relatively unstructured. In the first year, we sought to elicit 'soft' data from projects, informed by your formative evaluation and experiences of implementation. In this second and subsequent years, reporting should include more systematic feedback from trials or demonstrators, evidence of outcomes and effects, and clearer plans for post-project exploitation .

The annual reporting structure proposed here is intended to dovetail with management agreements made with eLib about regular reporting. In preparing an annual report, we would ask you to report for the period subsequent to that covered by last year’s report and up to the last major milestone as specified in your own project plan.

Projects are requested to follow the format below when preparing their annual report. We would also ask you to append any relevant documentation (evaluation reports, business plans, publicity materials etc.) when forwarding your report to eLib.

1. Activities and progress

This section is concerned with activities and progress in relation to your proposal and contract with eLib. It should identify what has been achieved since the start of the project/since the last report.


Questions to address:

- What have been the major activities undertaken by the project?

- What have been the effects of any changes you may have made to the project plan in the light of the experience of the first year of the project and indicated in last year’s annual report?

- What have been your main objectives or targets during the period and how far have you been able to meet them?

- What outputs have you produced from your activities (such as prototypes, models, demonstrator services, actual services, events, reports etc.)? Please quantify dimensions where possible.

- Do you have any particular successes to report?


2. Learning from the process of implementation

This section of the report is concerned with differences between what was planned and what actually occurred, and the reasons for any such changes to plan. Please tell us about any difficulties you have encountered as well as unexpected opportunities that may have opened up. Projects are encouraged to report on all forms of change including 'learning from failure'.

Check list of questions:

- Have you encountered any difficulties in managing the project and carrying out your activities (e.g. staffing problems, technical delays, increased costs, involving relevant users)?

- What influence, if any, (positive or negative) have other projects, the programme as a whole, or the programme office had on your project?

- What changes have you made to your plan (aims, objectives, staffing, activities, etc.) in the light of your experiences?

- What are the reasons for these changes?

- Has your project thrown up any unanticipated outcomes or unexpected opportunities and how have you taken account of these?

- What have you learned from your experiences of innovation and development?

Do you now have a different understanding of what you are trying to achieve, or the nature of the innovation?



3. Interim evaluation results

Projects will be collecting systematic, structured feedback at key stages in the project lifecycle, as outlined in the Guidelines for eLib Project Evaluation and in line with your own project evaluation plan agreed with FIGIT (as was). These data are likely to be both formative (informing ongoing development and decision-making in the project) as well as summative (providing evidence of effects and longer term impacts).

Please report on the findings which are emerging from your evaluation activities, commenting in particular on any general outcomes, effects and impacts. (Please also note and explain here any difficulties or delays you have experienced in carrying out your evaluation activities.)

In keeping with the eLib programme's overall evaluation preoccupations, you might usefully comment on the project outcomes and effects in relation to the following:

- mobilisation

- cultural change

- cost-effectiveness/value-added

- sustainability

- demand/usefulness/performance

- future scenarios

- contribution to overall project goals

Some possible facets or indicators which you might consider for each of these broad criteria are listed in Table 1 in the Guidelines on Evaluation. These should be supplemented by project's own operationalisation of evaluation criteria, relevant to their particular situation and view of what is important.

Feedback on the following area specific issues is also requested:

Electronic Journals: Effects on scholarly practice.

Access to Network Resources: Contribution to and emerging role in subject information retrieval infrastructure.

On Demand Publishing: Emerging models of HE library provision particularly as regards the provision of teaching resources.

Electronic Document Delivery: The elements of the business case for operational services.

Training and Awareness: Indications of impact on behaviour.

4. Future development

In this section you should set out any ideas you might have about future developments. It would be helpful to identify any changes in planned direction as well as any changes in proposed management and staffing arrangements.


Questions to be addressed:

- What are the main objectives for the next reporting period?

- What, if any, changes in overall direction are proposed?

- How do you now envisage the future scenario for the product or services you are developing, beyond the project timeframe?

Projects which are ending this year should present here, as appropriate;

- Post-project plans of the participating institutions, where possible expressed in terms of business plans/ specifications of follow-on projects or activities.

- Overall conclusions about the results and implications of the project.






If you have any queries about the Annual Reporting process or format please feel free to contact May Pettigrew at

The Tavistock Institute

Evaluation Development and Review Unit

30 Tabernacle Street

London EC2A 4DD

phone: 0171 417 0407

fax: 0171 417 0567

email: m.pettigrew@tavinstitute.org



Appendix 2: Project Reports Reviewed


ADAM: Art, Design, Architecture & Media Information Gateway

BIZ/ED: Business Education on the Internet

CAIN: Conflict Archive on the INternet


EEVL: Edinburgh Engineering Virtual Library


OMNI: Organising Medical Networked Information

ROADS: Resource Organisation and Discovery in Subject-based services

RUDI: Resources for Urban Design Information

SOSIG: Social Science Information Gateway


Internet library of Early Journals

DIAD: Digitisation in Art and Design

Higher Education Digitisation Service


EDDIS: Electronic Document Delivery

SEREN: Sharing of Educational Resources in an Electronic Network in Wales



CLIC: A parallel electronic version of an established journal - Chemical Communications.

Internet Archaeology: an international electronic journals for archaeology

PPT: Parallel Publishing for Transactions

The Superjournal project

Electronic support for Scholarly Communication

Electronic Seminars in History

Electronic Reviews in History

DeLiberations on Teaching and Learning in Higher Education

NewsAgent for libraries: a personalised current awareness service for library and information staff

JILT: The Journal of Information, Law and Technology

Open Journal: The integration of electronic journals with networked information resources

Sociological Research Online

Project Reports Reviewed (continued)


ACORN: Access to COurse Readings via Networks

ERCOMS: Electronic Reserve Copyright Management System

PATRON: Performing Arts Teaching Resources ONline


DIGIMAP: National on-line Access to Ordinance Survey Digital Map Data

HELIX: Higher Education Libraries Image Exchange

MIDRIB: Medical Images: Digitised Reference Information Bank



On-Demand Publishing in the Humanities (was Only Connect)

SCOPE: Scottish On Demand Publishing Enterprise

ERIMS: Electronic Readings in Management Studies

Eurotext: A Collaborative Resource Bank of Learning Materials on Europe


Ariadne: A parallel Web and print newsletter for librarians and information scientists

EduLib: Educational Development for Higher Education Library Staff

Netlinks: Networked Learner Support

Netskills: Network Skills Training for Users of the Electronic Library

TAPin: Training and Awareness Programme in networks