Comparative Evaluation of the Subject Based Gateways Approach to Providing Access to Network Resources

A report to JISC under the eLib Supporting Studies Programme

 

David Haynes, David Streatfield, Noeleen Cookman and Helen Wood

August 1998

 

David Haynes Associates

Signet House

49-51 Farringdon Road

London EC1M 3JP

Tel. 0171 242 4849

Fax 0171 242 4858

E-mail dhaynes1@compuserve.com

Information Management Associates

28 Albion Road

Twickenham

TW2 6QZ

 

Tel/Fax. 0181 755 0471

E-mail 100753.631@compuserve.com

Contents

Report Summary *

Introduction *

Background *

Terms of Reference *

Approach *

Looking to the future *

Reaching different audiences *

Going beyond established limits *

Literature review *

Programme of activity *

Preliminary investigation *

Focus groups *

Electronic discussion *

Structured interviews *

Delphi-style forecasting exercise *

Costing study *

Development of models *

Perceptions of the issues *

An ideal Internet information service *

The main issues and concerns *

Evaluation and quality assessment issues (system and information) *

Resources and finance *

Users *

Implementation of standards *

Access to and searching of information/systems and provision of information *

Structures and inter-relationships *

Technical issues *

Growth and development issues *

Providers *

Legal issues *

Other issues *

Service providers' views *

Future funding *

End-user charging *

Sponsorship *

Commercialisation *

Advertising *

Marketing *

Savings through collaboration or rationalisation *

delphi-style FORECASTING activity *

Likelihood and desirability scores *

Gaps between likelihood and desirability *

Cataloguing costs *

Limitations and problems of comparability *

Existing data on cataloguing costs *

University costs *

SBG provider costs *

Amount of cataloguing activity in the HE sector *

Potential market for catalogue records *

Models for Access to Network Resources *

Model 1 - JISC pays for continuation of SBGs *

Model 1A - Continuation of SBGs (No change) *

Model 1B - Continuation of SBGs (Some rationalisation) *

Model 1C - Continuation of SBGs (Centralised service) *

Organisational culture *

Model 2 - JISC pursues a market development path *

Model 2A - Rationalised services *

Model 2B - Fully centralised service *

Model 3 - JISC ceases to support SBGs at the end of current funding *

Alternative approaches *

Discussion *

Impact on the HE community *

Model 1 A - Continuation of SBGs - on current basis *

Rationalised services (Models 1 B and 2A) *

Centralised services (Models 1C and 2B) *

Model 2 - Market development *

Model 3 - Cease funding *

Financial implications *

Model 1 - Continuation *

Model 2 - Market development *

Model 3 - Cease funding *

Development of the subject-gateway approach *

International co-operation *

Commercialisation *

Integration with other resources *

Recommendations *

Strategic approach *

Rationalisation *

Matched funding of services *

Commercialisation of services *

Implementation *

Appendix A - Structured interview schedule *

Appendix B - List of interviewees *

Appendix C - Focus group participants *

Appendix D - The forecasting instrument *

List of respondents to whom the Delphi style instrument was sent *

Appendix E - Results of the focus group exercise to prioiritise issues *

Report Summary

An evaluation of the subject based gateway (SBG) approach to providing access to networked information resources was carried out. The emphasis of the study was on the general approach rather than on the performance of specific JISC-funded services. The purpose of the study was to help inform JISC's decision-making on the future of the current gateway projects.

A range of consultation techniques were used to identify issues across the HE community, among academic librarians, and with gateway service providers. A Delphi-style forecasting exercise was also carried out to identify possible future needs.

The responses emphasised the importance of quality of resources on the gateways, and the need for continuity of services. Issues such as standards and consistency of searching between services were also raised. Service providers were particularly concerned about continued funding.

An interesting aspect of the forecasting exercise was the prediction that technologies such as artificial intelligence would not replace the need for human intervention in evaluating the quality of other internet sites within the next seven years. Some anxieties were expressed about the commercialisation of the Internet, the growth of charged-for services and the inability of search engines to cope with the expansion of resources available via the Internet.

An investigation of the costs of cataloguing Internet resources was inconclusive. Estimated cataloguing costs varied from £3.20 to £34.00 per record. Further work is needed in this area.

Three models of access to network resources were presented as possible futures for Subject Based Gateways:

Model 1 - continued JISC funding -effectively turning the project into services and continuing to fund existing services on a similar basis to the present.

Model 2 - Market development - seeking sponsors to co-fund SBGs with JISC - reducing JISC's financial commitment by 50%.

Model 3 - Funding withdrawn and the SBGs left to fend for themselves.

Of these Model 2 was recommended, provided there were additional funds to hire a Managing Director with marketing skills to co-ordinate the services and seek matched funding on a performance-based contract. Further savings could be made by rationalising services so that marketing, standards development and training are provided centrally.

Opportunities for commercialising services should be explored. Examples include sale of records (or blocks of records) to HE institutions, and sale of the gateways to one of the large, general providers (e.g. Yahoo or Altavista).

A strategy for subject coverage should be developed to provide a framework for delivery of individual services. This would be informed by existing provision outside the UK HE sector and by exploring the scope for co-operation by means of mirroring sites in the US and Finland (for example) and joint development of services.

We estimate that the annual costs to JISC of running the existing services would fall to approximately £370,000 (from £850,000) because of matched funding and reduced running costs.

 

Introduction

Background

JISC has developed and supported several services that provide a means of gaining access to networked resources by their subject content. The services supported by JISC follow two models:

There are two further models that are used elsewhere but which have not been specifically supported by the JISC eLib programme:

Several of the existing projects for subject based gateways are coming up for renewal and this is a good opportunity to assess the approach and to compare its effectiveness against alternatives. The purpose of this exercise was not to evaluate individual services, but to assess the approach itself in terms of its effectiveness in supporting HE users.

Terms of Reference

The aims and objectives of this project were:

To investigate the key stakeholder views of the four models described above.

To evaluate, as a guide to policy, the relative technical, economic and scholarly merits of investments in different approaches to organising and accessing network resources. As part of this, to make a comparison between the value of the (largely) operational services BUBL and NISS (based on the first model) and SOSIG, OMNI, EEVL, and ADAM etc. (based on the second model and often using ROADS technology) with alternative local (third model) or established commercial (E.g. Alta Vista) and emerging approaches to accessing network resources.

To assess the need for compatibility/convergence between the subject gateways and other network bibliographic resources, especially in the light of the report on the proposed National Agency for Resource Discovery (NARD).

Examining the current and alternative organisational structures for the SBG approach, to recommend an approach which would provide the best combination of services, subject coverage, cost and cost recovery mechanisms.

 

Approach

Mapping the different methods of intervention is a prerequisite to a consultation of stakeholders in the provision of access to networked services. This provides a basis for analysis of the way in which different stakeholders interact with networked information. The provision of access to networked information was put into the context of overall provision of information for the HE sector.

Our overall approach is to provide sufficient evidence to enable JISC to decide on its strategy for funding support for access to networked information for the HE sector. This has focused on Subject-Based Gateways to Internet sites in the past. This project provided an opportunity to revisit this approach and to identify and consider alternatives. The likely impact of any decision about future support is likely to affect funding decisions for the next 5 - 10 years. However changes in the technology, the HE sector itself and the commercial environment make it impossible to predict what will work best in 10 years, or even 5 years time.

Looking to the future

In looking at the different models we considered the theoretical possibilities of the different interventions (i.e. how the models might work in an ideal world) as well as the practical reality where existing systems are constrained by commercial considerations, time lags in implementation of technology and limitations in the skills of different audiences. We also considered the impact of the following factors in assessing the future viability of the models:

We drew on our experience of administering Delphi instruments to take a peek into the future so that we can, in a qualitative sense, anticipate ways in which the future HE environment will affect the choices faced by JISC in enhancing access to networked information.

Reaching different audiences

Different audiences can be reached in different ways and that it is unlikely that any one method is going to give as complete a picture as a mix of techniques will.

For instance Web sites and mailbases provide an excellent way of reaching an international audience.

Key influencers, such as senior decision-makers and researchers who have actively published in the area often respond well to direct face-to-face interviews. This is very useful at the start of a project because they can draw attention to issues that need to be raised with other target groups during the investigation.

Focus groups are an excellent way of identifying issues and prioritising them. They generally work best if there is some commonality among the participants

Going beyond established limits

Finally, we felt that it was necessary to step outside the frame of existing services and consider alternatives to the current approaches. A project of this type needs to be able to challenge current received wisdom on issues such as funding, access and exploitation of resources. For instance, it may be possible to develop a system of sponsorship of gateways by publishers in return for advertising their products or services. There may be new ways of providing access to networked resources that fall outside the four models that are currently recognised.

 

 

Literature review

A literature search was undertaken using the following INFOSCI databases on Dialog:

File 1 ERIC

File 2 INSPEC

File 6 NTIS: National Technical Information Service

File 61 LISA (Library and Information Science Abstracts)

File 121 British Education Index

File 148 IAC™ (Trade & Industry Database)™

File 202 Information Science Abstracts

File 211 IAC™ (Newsearch)™

The following search terms were used:

Subject-based gateways

Information gateways

Search engines

Evaluation and subject-based gateways

These terms were also used as a basis for the Internet searches, using the following search engines:

InfoSeek

Lycos

Yahoo

Excite

Some web-sites acted as starting points for further searches because of their links to other relevant sites. For example, Searchenginewatch: http://searchenginewatch.com, Netskills: http://www.netskills.ac.uk and some of the electronic web newsletters, such as Ariadne: http://www.ariadne.co.uk, were investigated.

Authors such as Terry Gray, Ron Conte, Richard Peterson and the editor of PC Magazine Online compare the various Web search aids, referred to, in many cases, as search engines, catalogues, directories, indexes or Web databases. These studies appear to be very descriptive and comparisons are made about their search capabilities, such as Boolean Logic, truncation, field searching and word/phrase searching, and retrieval performances, such as precision and response time. Some studies include ranking charts.

The search engines/information directories described include:

AltaVista

Excite

WebCrawler

Lycos

Opentext

InfoSeek

Yahoo

NlightN

Internet Sleuth

PlanetSearch

LookSmart

HotBot

Search.com

Magellan

In many cases Alta Vista, Excite, Yahoo, HotBot, Lycos and InfoSeek are ranked as the top five. These choices are opinions based on different search strategies undertaken at different times. In most studies links are provided to the various search engines.

Chu and Rosenthal looked at three search engines (AltaVista, Excite and Lycos). They found that:

'AltaVista outperformed Excite and Lycos in both search facilities and retrieval performance although Lycos had the largest coverage of Web resources among the three Web search engines examined.'

This is by no means an exhaustive list. It is only an example of what has already been written in this area.

The project leaders for each of the subject-based gateways have also written descriptive articles about their services. These tend to give a brief history of the gateway and include links to sites of interest within the subject area. Examples can be found at the relevant subject-based gateway Web page, for example: ADAM, EEVL, SOSIG, and OMNI.

The Internet searches gave an overview of searching for information and provided pointers to useful sources. There are lists and links to individual search engines, "collections" of search engines; other collections of WWW search facilities; subject-specific Information Gateways; Resources for comparing various search engines, such as http://www.searchenginewatch.com/ and the DESIRE project; and General or Introductory sites, such as the NetLearn pages, Matrix Information and Directory Service (MIDS) and various Internet Magazines, such as Ariadne – http://www.ariadne.co.uk and Heriot-Watt's "Internet Resource Centre" pages and Newsletter.

Further Searching for Information on the Internet covered the more advanced search engines, searching library catalogues, searching for software, searching Wide Area Information Services (WAIS), general sources of interest to anyone working in the UK Higher Education community and some suggestions about searching for email addresses.

Lorcan Dempsey discusses the resource discovery services in the UK and suggests "the need for some co-ordinated development activity". He states that:

For many people the first ports of call are the major robot-based ‘vacuum-cleaner’ services such as Lycos and Alta Vista which provide access to web pages world-wide, or classified listings such as Yahoo. Within UK higher education, there is a variety of services: NISS, BUBL, the new eLib subject-based services (ADAM, EEVL, IHR-Info, OMNI, SOSIG, et al), various listings and sensitive maps, and multiple more or less useful specialist resources. In an interesting recent development there is some experiment with Harvest to create an index of academic sites at Hensa at the University of Kent….

He continues:

Within the subset that is supported by higher education funds there is some overlap and even competition. BUBL has a growing classified collection of thousands of links…. NISS operates an information gateway and populates it with records in a variety of ways. BUBL and NISS aim to cover all subject areas. The subject-based services are creating subject specific gateways. Some are now available; others will come on stream, but major subject areas are not covered and projects are not constrained to take the same technical or service approach…

In another article Dempsey includes a section where a number of the eLib project areas are described including the Subject-Based Information Gateways, which are in various stages of development.

Klaas Wierenga, project leader for the DESIRE project promoting the use of the World-Wide-Web (WWW) in the European research community states that the DESIRE project:

Aims to solve the problems, which currently hinder the use of the World Wide Web for researchers. One of its goals is to deliver a sustainable approach to making searching, finding, browsing, accessing and retrieving multimedia resources as easy as possible. Some users need to locate information in sets of identified categories. For this purpose Desire will provide the foundation for this kinds of services be developing subject-based information gateways (SBIGs) through which mediated, quality controlled and well-catalogued data sources may be accessed. An example of an existing service like this is the Social Sciences Information Gateway (SOSIG) in Bristol.

Chris Armstrong investigates the many ways of searching the Internet using search engines, such as Alta Vista or services, such as Excite Web Reviews or the Magellan Internet Guide. He also identifies the subject-based gateways, such as ADAM, EEVL, OMNI and SOSIG, and states that these:

Will direct users to selected and evaluated resources in their interest area.

He refers to the Anagnostelis, Cooke and McNabb review of the subject-based gateways where the services are described and offer a form of scoring mechanism. He states that:

No standard quality vocabulary has been developed and users are invariably unable to judge the strengths and weaknesses of sites.

Roddy MacLeod and Dave Bond have developed a site called 'PINAKES: A Subject Launchpad' since they found that the search engines were good for locating relevant sites on extremely specific topics, but not so good on finding resources on more general subjects. PINAKES points Internet users towards some of the subject-based gateways that are currently available. The Launchpad is reviewed in the Internet Resources Newsletter, Issue 42, March 1998,

(URL: http://www.hw.ac.uk/libWWW/irn/irn42/irn42.html).

The NISS Information Gateway Directory of Network Resources has taken the form of a subject-based Resource Guide. Internet searchers are guided through a hierarchical inverted subject "tree" to the topic they are interested in. Users can also browse the directory in alphabetical subject order, in UDC shelfmark order or search by keyword in title, keyword in description of a resource or UDC classmark. NISS recognises that the subject approach is important since almost all libraries are organised principally by some kind of subject classification system. NISS also offers a service for adding new resources to the NISS gateway - this appears to be quite simple and is merely a matter of filling in one of the templates found at the Website.

The KPMG Report on Infrastructure Services reviewed and compared some of the services that are offered as JISC funded projects including Mailbase, BUBL, NISS and HENSA. However its conclusions were controversial with very strong objections about its accuracy from one of the services reviewed. For this reason we have not included a detailed analysis of that report in this investigation.

 

Programme of activity

Preliminary investigation

An initial meeting was held with members of the JISC Steering Group on December 3rd 1997 at which a number of points relating to this project were agreed. Individual semi-structured interviews were then arranged with members of the Steering Group to identify issues for consideration in the project; these were completed in early January 1998 in time to provide context for the series of five focus group meetings with service providers, electronic publishers and users held in January and February 1998 in London (three), Bath and Edinburgh. (The Bath event was arranged to follow on from the Models 6 Workshop organised by UKOLN).

Focus groups

The focus group events were deliberately set in a broad context encompassing subject-based gateways; search engines; institutional gateways; and general/national gateways. The initial card-sort activity was designed to clarify perceptions of what an ideal electronic information service might encompass; then participants were asked to identify and prioritise the main issues and concerns to be addressed/resolved in providing access to network/internet resources (using the Nominal Group Technique).

Electronic discussion

An electronic debate was initiated in the same period to allow participants in the lis-eLib mailbase to comment on the same sort of issues.

Structured interviews

Key issues raised in the focus groups and electronic commentary were included in the schedule for a series of 10 structured interviews with representatives of services funded by JISC and projects funded through the eLib Programme. These interviews covered the nature of each service (subject coverage, selection and de-selection policies, audiences, size, collaboration, publicity, statistics of use, ethical issues), service costs and sources of income, future developments (co-operation, technical developments, information content, funding) and alternative ways of supporting access to high quality network resources (relating existing services to advances in search engines, commercial services, clumps of catalogues, institutional/local gateways, centralised sources, ‘kite-marking’ of sites or institutions), as well as integration with other services, partnerships and a views on a centralised model of service provision.

Delphi-style forecasting exercise

Throughout the interviews and focus groups, respondents were asked to respond to any questions posed about future developments by adopting a three to five year planning frame. However, we felt that it would also be useful to glean somewhat more speculative views about change in this general area, given the propensity for IT developments to emerge rapidly and to be adopted or rejected with little prior warning. Accordingly, 50 people (including several of the earlier respondents and 10 overseas experts in the area) were asked to respond to a Delphi-style forecasting instrument, in which various propositions about the year 2005 were judged on their likelihood, desirability and importance.

Costing study

A short investigation was undertaken to quantify the costs involved in cataloguing and supporting access to high-quality network resources on the Internet. An extensive search of the literature and of Internet sites was conducted before SBG providers and a selection of HE Institutions were approached for costings on cataloguing costs.

Development of models

A series of models were developed on the basis of the consultations and analysis of the possible scenarios for the future of Subject Based Gateways. These are described in detail later in this report.

 

 

Perceptions of the issues

An ideal Internet information service

Focus group participants were encouraged to establish common ground for the discussion and incidentally to show what services they regarded as most important by means of a card sort activity. A set of concepts describing an ideal Internet information service was borrowed from an article by Sheila Webber. Individuals assigned a rank order for topics dealt to them at random. Then any disagreement about the positions in the initial rank order were challenged by participants and some items were added. This process of challenge formed the basis of discussion to clarify differences in understanding about the terms used and in perception of their relative performance. It should be stressed that the positions shown below are an incidental outcome of the discussion which was primarily designed to establish common ground for the issues identification and prioritisation session that followed.

The results are presented in tabular form as Appendix E. One interesting feature is that, apart from the ‘top three’ items there was very little consistency amongst the rankings arrived at by the five groups. The usual result of this type of activity is strong consistency amongst the higher and lower levels of the rankings. The absence of any such consistency amongst the focus group participants suggests that the idea of systematic searching on the Internet, whether by means of gateways or search engines, is not yet sufficiently strongly established to invoke consistent expectations.

The three strongly supported elements in an ideal Internet information service were: quality assessment procedures in place to evaluate the resources provided; a specialist search engine (searching both descriptions of rated subject websites round the World and pages of sites listed on relevant bibliographical databases); and a flexible search engine for the whole site (allowing, for example, restriction by type of resource: articles, discussion lists etc.).

The main issues and concerns

Focus group participants were next asked "What are the main issues and concerns to be addressed/resolved in providing access to network/internet resources over the next three years?" Their replies have been regrouped under main headings below.

Evaluation and quality assessment issues (system and information)

There was widespread acceptance of the need for greater attention to evaluation of systems, services and information quality. More explicit quality criteria were seen as necessary in relation to the evaluation of sources, mechanisms and ‘intelligent agents’, indeed it was argued that the prime role for SBGs is to make critical assessments of resources. Several aspects of the information made available through gateways and search engines were seen as important, including its level and comprehensiveness, whether it was current and timely, accessible and academically credible. An oft iterated point was that the conventions of scholarly peer review had yet to be carried over to Internet publication and a specific concern was about establishing the authenticity and integrity of digital texts.

One minority view was that much evaluation of services is backward looking and unhelpful - "experience suggests that many services work perfectly well without."

Resources and finance

The prime concern here was with sustaining the efforts already being made and ensuring that adequate finance and other resources could be secured to provide the required infrastructure (whether the emphasis was to be on subject-based or national gateways) with the accelerating growth of Internet resources. A forceful case was made that funding is not currently sufficient to develop or provide an adequate service in any of the areas covered, expanding the more common view that "subject-based gateways cannot go on forever – there will not be enough money to pay for what is needed." There was seen to be a need for a national development strategy for taking this work to the marketplace but a danger was envisaged of commercial providers ‘cherry picking’ the most attractive resources, leaving less attractive services to be funded centrally.

The question of "the cost – and who pays?" centred partly on the perceived unwillingness or inability of Universities involved with current initiatives to take over responsibility for service management. (One person asked whether "there is mileage in getting groups of users to take on ownership of SBGs and raise revenues for this?") A related worry was about the cost of escalating technological change. At institution level the concerns were with securing enough resources to maintain both ‘manual’ and automated gateways, with funding for hardware, software and access to resources and with achieving adequate computer to student ratios to be able to benefit from current developments. Microcharging was not widely welcomed ("charging at the point of access would totally suppress use") but was seen as likely in some form. One optimist, however, anticipated that user perceptions will change so that people come to expect to pay for good information.

Several groups had reservations about the impact on services of any move towards commercialisation (or "creating a commercial infrastructure to control access to generate payment") although some people saw the move towards a business model as inevitable ("fewer free sites") and talked of merging free and commercial resources. The question of payment mechanisms or effective pricing models (including electronic payment) surfaced several times. A useful point made was that charging mechanisms need to be scalable in parallel with service scalability. Adoption of advertising as a source of revenue was not generally seen as more than a means of ancillary support, and one which gave rise to further anxieties about censorship of content and reduction in quality control in favour of volume of use.

Any more commercial approach clearly depends upon being able to demonstrate value-added services. This was seen as inherently difficult because "this is not clear until the user finds the resource", suggesting the need for users to be taken direct to useful resources rather than simply being presented with descriptions of potentially interesting sites or information. There was seen to be a need to show users a saving in time and effort through access to SBGs, but this was not yet seen as realistic.

Users

All five groups talked about user requirements: the division here was between the people who saw increased user knowledge as being primarily an issue of computerised registration and data collection (or "technical profiles of target groups") leading to better user interfaces and those who felt that services would not be adequately used until more was learnt about the information-related behaviour of different types of potential users so that appropriate training and tailored services could be provided.

Views on user education similarly moved through those who were primarily concerned with training people to use systems ("orienting the user") to those who emphasised the lack of recognition of information seeking as a skill by academics and beyond to others who emphasised user involvement and control in system development. These different perspectives translated into differing concerns with user assistance, making users aware of limitations of Internet resources, helping users to learn to evaluate resources (with one model offered being the critical appraisal skills workshops for hospital consultants and other health care professionals as part of evidence-based health care programmes), and community relevance of the services offered.

A specific concern raised was that increased information availability on the Internet was leading people towards unrealistic expectations of what the technology could deliver and gross underestimates of the time, energy and effort required to realise what is possible. It was also emphasised that people’s technical and information handling competence varied enormously "from no knowledge to vast amounts" and that IT familiarity amongst HE staff was often lacking. (Generic user profiles were reported to have been developed by SOSIG as a way of helping users.) It was felt that too little was known about JISC and similar developments, requiring greater attention to promoting the services to potential users. An optimist here anticipated that "some publishers will help with free publicity for promoting relevant gateways".

Particular user groups singled out for attention were lecturers and librarians. Both groups were seen as needing "total retraining to cope with Internet resources" and students were characterised as "often more adept than their lecturers". However, the seductive power of IT was such that "in three years time we may have to educate lecturers to use books."

An interesting suggestion was for centralisation of training services to prevent unnecessary duplication by service providers.

Implementation of standards

Again, all the groups raised the standards standard above the battlefield but there were few suggestions offered about what could be done and by whom to improve standardisation, beyond reference to the Dublin Core. The areas identified for standardisation were metadata (and problem of searching across different data types), delivery mechanisms ("you have to learn to access every site"), selection criteria amongst services and guidance for creation of quality resources. There was limited discussion about how to achieve the spread of agreed standards and the point was made that standards are slow-moving and tend to lag behind current practice.

Access to and searching of information/systems and provision of information

Given that the various approaches under consideration are all aimed at improving access to information it is not surprising that several people raised concerns about any future steps that might limit this access, including the charging issues already mentioned, any tendency towards institutions being proprietorial about information on gateways, and registration schemes, which were seen as potentially inhibiting. At a broader level, there was a strong suspicion that increased use of the Internet as ‘primary media’ would lead to increased recognition of its importance, leading in turn to increased risk of exclusion of potential users. Most groups referred to exacerbation of the division between information ‘haves’ and ‘have nots’, or to the disenfranchisement of non-University members and there was concern that subject-based gateways supported by public money should not become exclusive; this problem was presented as "bridging the gap between the HE sectors and the public".

With the growing complexity of the Internet and the independent creation of institutional and subject-based gateways, there was also concern about the difficulties of navigating the virtual environment and particularly about searching across different gateways. This difficulty was described in terms of "a constantly changing picture of network resources and of the information environment" and, more mundanely as "inconsistencies in terminology making string searching difficult." One emerging aid here was AGOG, the gateway of gateways being developed by Oxford University.

Naturally, this recognition led to various comments about the issue of searching the Internet and whether gateways would provide a viable alternative to major search engines (assuming that these would become increasingly sophisticated). (One approach discussed involved ‘shipping’ metadata into SBGs, which could then be evaluated by staff.) Much of the discussion here could be summarised as the requirement to design simple user interfaces to complex collections. Part of the answer was seen by various groups as involving the seeker setting limits on the subject, level and quality of searching required – and the systems being able to respond appropriately, better organisation and design of websites, better means of showing people "where they are on the network", multi-lingual support (at the user interface and metadata levels), and other ‘value-added’ services. This phase of discussion provoked a worry about "too much emphasis on searching and not enough on browsing" (the argument was that over-organised search strategies would eliminate the element of creative serendipity involved in "browsing the stacks").

The somewhat arcane jargon of granularity ("Does a gateway point to a database or to individual records on a database?"), "integrating third party descriptions with primary content", forward knowledge and push services was invoked in various discussions, or, at a more prosaic level "finding the stuff to link to".

Two specific difficulties rehearsed were on-line to off-line transition when people are working and an extension of selective dissemination of information services to include an ‘alarm system’ when relevant information is sent.

Structures and inter-relationships

The issues described here could be categorised as being primarily conceptual, strategic or operational.

Conceptual issues discussed included the hybrid library, covering such issues as the need to push non-Internet resources as well as electronic resources. It was stressed that a lot of information is not available on the Internet, but that (for example) "Searching for medical sources, nobody looks before 1966 when Medline started. This means that the pre-1966 literature is effectively lost to scholarship." Other topics covered were the desirability of striving towards open systems, developing structures in the face of increasing globalisation, and the difficulties of establishing any sense of community ownership of potentially very large systems.

At the strategic level there were several calls for a national strategy for development of the whole area (or even improving the web infrastructure) to be undertaken by JISC or by the Library and Information Commission. The contrary view was that things were happening too fast to do more than try to sort out the potential relationship between subject gateways and other search services and attempt to secure international co-operation in provision of SBGs and other services.

The Scottish focus group noted the huge national strategy potential for Scotland in the near future, based on greater likelihood of co-operation amongst the Scottish libraries and education sectors than in England, easier access to key politicians and a more co-operative tradition.

Operational issues raised included the desirability of moving towards a service culture (from the ‘development’ culture) by: establishing service agreements amongst users, operators and providers; eliminating duplicated effort; clarifying the relationships amongst SBGs and between these and other types of services; and rationalising the subject coverage (including filling the gaps where there are no subject gateways and not doing what is done better elsewhere).

Other issues included the effect upon the British Library of digital developments, including the extension of legal deposit; hard copy delivery choices; inclusion of other people’s local resources; the option of establishing a common user interface for gateway services; integration with other discovery services and with non-Internet resources; and electronic archiving.

Technical issues

Although there were no restrictions on the focus group agendas, participants tended not to dwell on technological issues; indeed there was some feeling that most requirements were technologically feasible providing that sufficient resources could be commandeered, and that IT advances ("especially search engines") would lead to greater possibilities [but at what cost?]. People were less sure about the likely impact of other developments such as JAVA.

Discussion of Internet capability centred mainly on the question of bandwidth (perhaps a shorthand for "we need greater capacity so that we can overload it"?). Mention was also made of the Millennium Bug. Other concerns related more to the information infrastructure than to the technology such as "too many common interfaces"; network congestion, reliable connection to the Internet, or a "browser that works for most sites".

A particular plea was to "Control the enthusiasm of the techies: lots of ‘bells and whistles’ take a long time to download, make systems crash, create compatibility problems and result in poorer users." It was asserted that "Lots of gloss should be on destination pages rather than passage pages."

Growth and development issues

The biggest issue in development terms was the likely impact on service development of a huge increase in the number of sites, seen as leading to increased need for good search engines; as well as a big increase in the number of users and concomitant pressure on resources.

Uncertainty about technological advances and about the direction and pace of Internet-based change was the other constant theme. Variations here included the effect of charging for international band-width; electronic legal deposit requirements; the likely pace of digitisation of archive resources; automation and standard metadata; the rate of commercialisation of the Internet; cacheing and mirroring strategies; and multi-media advances; as well as the effects of the Year 2000 glitch and of European adoption of monetary union.

Other issues were centred on the implications across all sectors of lifelong learning and of changing educational demands for distance learning, open learning and part-time learning; and the increasing dross on the Internet (an analogy was made with coral – "constant growth and dead in the middle").

Providers

Given the involvement of service and content providers in various focus groups there was surprisingly little attention focused on these people, apart from occasional recognition that they had training needs. One issue identified was how the enthusiasm of content providers (whether volunteer individuals or institutions) could be maintained if there was a move towards more commercial national or subject-based gateways. A suggested solution was through a shared cataloguing model, with members who contributed getting credits that could be spent as users. A training concern voiced was to encourage providers to use meaningful titles and good metadata.

Legal issues

The main legal issues cited were difficulties in securing, maintaining and circumventing copyright (and the shape and impact of likely legislation); concerns over maintaining privacy and security of information and over the inhibiting effects of measures aimed at achieving these; and, more specifically problems caused by people stealing frames.

An aspect of copyright that moved beyond legal concerns was that some academics were seen as anxious about plagiarism after Internet publication or fearful that releasing a working paper on the Internet could prejudice later publication in a refereed journal.

Other issues

The international character of the Internet gave rise to several related concerns (including the lack of a global strategy) and to the hope that gateways would in future have less Western bias (or US dominance) and become more international in character. There was also suspicion about the political motives of major players and about "control by ‘people in Authority’ over what people look at".

Questions were asked about the likely effects of Internet development and of gateway services on the scholarly journals and trade publishing industries, as well as about the future role of the information specialist.

 

 

Service providers' views

The views represented here are based on discussions with the service providers for the JISC funded subject based gateways. Contact with providers was established by a variety of means including face to face interviews, site visits, correspondence and telephone interviews.

As part of the original conditions for JISC funding, the service providers have prepared exit strategies. This was on the understanding that JISC funding was provided on a time-limited basis. A number of responses to possible future models for provision of access to network resources including:

Federation of Information Services (FIS)

Federation of Information Gateways (FIG)

Some scoping work has also been done for the development of a National Agency for Resource Discovery (NARD) for which access to networked resources would form a significant part.

Some of these proposals have provided some useful figures on possible budgets for future services.

Future funding

Most service providers see the likely future as involving some form of ‘mixed economy’ (or "intermediate level of support") entailing revenue from data gathering, providing services, sponsorship and advertising but according to most respondents "there will also be a requirement for an element of grants".

End-user charging

There is resistance to the idea of end-user charging because "charging reduces usage, whereas the teaching need is to encourage usage", or "the culture is not there for users to pay on an individual basis and confidence in credit card security is not there." A respondent anticipated that direct charging will be driven by banks and the entertainment industry, but until this happens it is unlikely to be successful. On the other hand although some people recognised the possibility of charging University Libraries for services, one insisted that "If there were a charge of say £500 per institution, there would be a problem of assigning the budget – especially for University Libraries with devolved budgeting".

Sponsorship

In looking at alternative funding sources, it should be noted that, in general potential donors or sponsors are not usually attracted by the idea of supporting the infrastructure of an organisation. (One provider of services observed that "it may be easier to get sponsorship for a particular machine than a resource officer". Similarly, it was reported that three quarters of NISS operating costs are covered by the JISC grant, but that 75% of the costs of the information are met from other sources such as sponsorship.) It is also relatively difficult to secure steady on-going revenue from sponsorship.

Again, a significant part of present funding is hidden in subsidies ranging from office space, through heat and light to unassigned senior staff time (EEVL, for example, estimated that financial support in kind amounted to almost 50% of funding).

Some gateways do have funding arrangements in place, for example with professional societies in their subject areas, but there is estimated to be only limited potential for charging for commercial access.

Commercialisation

A major factor in considering the way forward is that, with the exception of BUBL, the gateways reviewed in this assignment are all operating as development projects rather than as services. Some SBG providers feel that there is a market for commercial products from their services (both OMNI and EEVL are actively pursuing this). However, according to one commentator "most of the SBGs don’t have sufficient content to operate as stand-alone services. You need something more substantive than pointers and catalogue records to demonstrate added value and there is already substantive competition elsewhere".

Although one person volunteered the view that "Commercial organisations have the comfort factor of legal status and financial loading – so buying into a commercial product provides security" most respondents were cautious about venturing into the brave new world of commerce. A more typical view was that "Gateways will have to become more commercially orientated and provide cost effective services". This gave rise to a series of questions, such as "What does a cost effective national service mean?" ("Operating 15 – 20 gateways is not economically viable".) "What proportion of cost is taken up in producing the records and what does it cost to run the service?".

One respondent envisaged the future way of working as based on service contracts and with centralised service provision but with distributed evaluation of resources.

Advertising

Advertising is seen as a future source of revenue by some service providers. There will be issues of obtrusiveness (one provider sees advertising banners as likely to undermine the value speed and reliability of the service) and a need to develop both contacts with potential advertisers and a price policy. Better information about the users who may provide the ‘magnet’ for advertisers will also be necessary.

Marketing

Whatever services are developed, it will be important to concentrate attention heavily on marketing, and hence on appropriate service delivery. (For example, it was stressed that engineers as a user group have particular requirements for up-to-date and technologically orientated information (such as patent information), that the pertinent information is very fragmented, with lots of small, specialised trade publishers. Accordingly, it was argued that a centralised gateway is more important for this group than in other subject areas. Without accepting or disputing the merits of this argument, it will be important to weigh up such considerations in deciding how to move forward.)

Savings through collaboration or rationalisation

There were two broad views expressed by service providers in discussing the potential for reducing costs by rationalising aspects of service provision. One view was that any benefits are at best theoretical because "co-operation always costs money" or "we could make a centralised model work, but it would not be any cheaper – it would still need subject expertise and build a sense of community for each group - it would eventually look very much like the current situation". The other view was that operating as a more centralised service would provide economies of scale and savings through the elimination of duplicated effort and overlap ("Splitting up material into different subject gateways adds a cost burden either to users or to the funding body.")

Three points from interviews with service providers are offered here without comment:

"The main operating expenditure is on people - most money is spent on content and intellectual work."

"SBGs work best within a broad subject area."

"Smaller areas would benefit from merging."

 

 

delphi-style FORECASTING activity

Fifty-five people were identified through a literature scan and during the project as having a broad view of the field; they were asked to participate in a Delphi-style forecasting activity. Each potential respondent was presented with a series of 28 propositions about the ‘Internet world’ in the year 2005 (i.e. seven years hence) and was asked whether each was likely to occur, whether this was desirable, and how important it would be if they did occur. Their comments on the desirability and likelihood of the propositions were reported on six point scales (providing for 56 responses in all), with the scales ranging from ‘highly likely/desirable’ to ‘not at all likely/desirable’. They were then asked to select the five most important propositions on the assumption that these had happened.

The ‘most important propositions were:

Table 1 - Importance

No.

Proposition

Score

10.

There are now widely applied standards for describing the subject content of Internet sites.

18

25.

Almost all new scholarly information is directly available through the Internet.

12

19.

There is a world-wide network of co-operating subject-based gateways.

10

2.

All main academic areas are covered by subject-based gateways.

9

13.

All significant public, academic, scholarly and academic libraries in the UK are sharing resources through the Internet.

7

1.

There is now no free access on the Internet.

6

4.

Applications of artificial intelligence have largely replaced institutional and subject-based gateways.

6

26.

Most journal-based abstracting and indexing services have extended their coverage to include network resources.

6

21.

Search engines can find information even inside proprietary databases.

5

5.

Commercially-driven subject-based gateways are the norm.

4

6.

Scholarly networks have transformed the availability of authoritative information on the Internet.

4

18.

A new structure of metadata-based services is providing quality information services for a fee.

4

22.

Most higher education is ‘delivered’ off-site (in people’s homes/other designated centres).

4

24.

More scholarly articles are published electronically than through print-based journals.

4

28.

Most academics and students start their quest with library-based resource pages.

4

We will comment on the significance of these importance ratings below.

Likelihood and desirability scores

The results were weighed on scales from highly likely (+3) to highly unlikely (-3), with similar weightings for the desirability ratings. A mean weighed score was then calculated by dividing these scores by the numbers of responses to correct for occasional non-replies to particular propositions. The results are shows in Table 2 and Table 3.

Table 2 - Likelihood of propositions occurring

No.

Proposition

Score

2.

All main academic areas are covered by subject-based gateways.

+1.25

26.

Most journal-based abstracting and indexing services have extended their coverage to include network resources.

+1.19

18.

A new structure of metadata-based services is providing quality information services for a fee.

+1.06

27.

Most libraries now provide their own selections of quality-based resources.

+1.0

13.

All significant public, academic, scholarly and academic libraries in the UK are sharing resources through the Internet.

+0.97

14.

Commercial exploitation of the Internet has created major problems of resource access for academic libraries in the UK.

+0.97

16.

Librarians are willing to pay for quality subject-based gateways.

+0.88

24.

More scholarly articles are published electronically than through print-based journals.

+0.81

23.

Service delivery contracts now specify the range of networked resources to which students are entitled.

+0.75

9.

Search engines are unable to cope with the burgeoning size of the Internet.

+0.57

6.

Scholarly networks have transformed the availability of authoritative information on the Internet.

+0.53

25.

Almost all new scholarly information is directly available through the Internet.

+0.47

19.

There is a world-wide network of co-operating subject-based gateways.

+0.44

10.

There are now widely applied standards for describing the subject content of Internet sites.

+0.42

12.

Sophisticated subject-focused search engines are the normal method of gaining access to Internet resources.

+0.4

3.

Subject-based gateways are largely maintained through centrally-provided (academic/government) funding.

+0.17

17.

Subject-based gateways are only available on a subscription basis.

+0.11

15.

Most useful information on the Internet is available only in proprietary sites on a fee basis.

+0.11

28.

Most academics and students start their quest with library-based resource pages.

-0.19

21.

Search engines can find information even inside proprietary databases.

-0.22

5.

Commercially-driven subject-based gateways are the norm.

-0.25

11.

Closed networks (e.g. for business and for the HE community) have rendered current models of Internet use largely redundant

-0.39

20.

There is a single gateway for quality academic resources for the UK academic community.

-0.45

7.

Search engines are fully adequate to find high-quality academic information.

-0.53

22.

Most higher education is ‘delivered’ off-site (in people’s homes/other designated centres).

-0.70

4.

Applications of artificial intelligence have largely replaced institutional and subject-based gateways.

-0.83

1.

There is now no free access on the Internet.

-0.89

8.

All major UK Internet access projects (such as national and subject-based gateways) are supported by the same central computer system.

-1.29

In our experience it is very unusual for the most likely (+ 1.25) and most unlikely ( - 1.29) responses in a consultation of this kind to be within such a narrow range. Two possible explanations are that all 28 of the propositions are inherently doubtful, or, more likely, that there is an unusual level of uncertainty about Internet-related propositions, given its recent emergence and the rapid escalation in its use.

However, within these unusually large ‘parameters of doubt’ there is some confidence in the growing use of the Internet for academic purposes (indexing of network resources, metadata developments, selection and sharing of resources by libraries). As to subject-based gateways and their alternatives, they are seen as fairly likely to cover all main academic areas, but centralisation of provision down to a single gateway or computer system looks less likely. A future of gateways of at least seven years is suggested, since alternatives such as applications of artificial intelligence or high quality ‘academic’ search engines are seen as doubtful by then.

Table 3 - Desirability of propositions being realised

No.

Proposition

Score

13.

All significant public, academic, scholarly and academic libraries in the UK are sharing resources through the Internet.

+2.0

25.

Almost all new scholarly information is directly available through the Internet.

+1.89

19.

There is a world-wide network of co-operating subject-based gateways.

+1.83

6.

Scholarly networks have transformed the availability of authoritative information on the Internet.

+1.81

10.

There are now widely applied standards for describing the subject content of Internet sites.

+1.78

2.

All main academic areas are covered by subject-based gateways.

+1.77

26.

Most journal-based abstracting and indexing services have extended their coverage to include network resources.

+1.69

7.

Search engines are fully adequate to find high-quality academic information.

+1.42

24.

More scholarly articles are published electronically than through print-based journals.

+1.33

16.

Librarians are willing to pay for quality subject-based gateways.

+1.00

12.

Sophisticated subject-focused search engines are the normal method of gaining access to Internet resources.

+0.89

3.

Subject-based gateways are largely maintained through centrally-provided (academic/government) funding.

+0.83

21.

Search engines can find information even inside proprietary databases.

+0.83

18.

A new structure of metadata-based services is providing quality information services for a fee.

+0.66

28.

Most academics and students start their quest with library-based resource pages.

+0.61

27.

Most libraries now provide their own selections of quality-based resources.

+0.25

20.

There is a single gateway for quality academic resources for the UK academic community.

-0.04

23.

Service delivery contracts now specify the range of networked resources to which students are entitled.

-0.12

4.

Applications of artificial intelligence have largely replaced institutional and subject-based gateways.

-0.28

8.

All major UK Internet access projects (such as national and subject-based gateways) are supported by the same central computer system.

-0.74

11.

Closed networks (e.g. for business and for the HE community) have rendered current models of Internet use largely redundant.

-1.06

5.

Commercially-driven subject-based gateways are the norm.

-1.23

17.

Subject-based gateways are only available on a subscription basis.

-1.31

22.

Most higher education is ‘delivered’ off-site (in people’s homes/other designated centres).

-1.44

9.

Search engines are unable to cope with the burgeoning size of the Internet.

-1.63

15.

Most useful information on the Internet is available only in proprietary sites on a fee basis.

-1.72

1.

There is now no free access on the Internet.

-1.77

14.

Commercial exploitation of the Internet has created major problems of resource access for academic libraries in the UK.

-1.85

Here the gap between the brightest and gloomiest prognostications is substantially wider, and nearer to the differences usually encountered, with stronger and more consistent views expressed about what people wanted to happen. Interestingly, there is a close correlation between what is seen here as desirable and what people regarded as important. Five of the six ‘most desirable’ propositions (all except item 6) form the five ‘most important’ propositions assuming they were to happen (as shown in Table 1 above). Clearly, an organised network of libraries served by effective subject-based gateways is regarded as a good idea – suggesting that JISC is on the right track in funding gateways. Unsurprisingly, the cluster of ‘least popular’ propositions is focused round restriction of access to information on the Internet, whether by accident or commercial design.

Gaps between likelihood and desirability

Where there is a substantial mismatch between the perceived likelihood and desirability of any proposition this may be of particular interest to policy makers as suggesting necessary courses of action. Where likelihood and desirability coincide it is likely that the issue will resolve itself.

Table 4 - Gap - More likely than desirable

No.

Proposition

Score

14.

Commercial exploitation of the Internet has created major problems of resource access for academic libraries in the UK.

2.82

9.

Search engines are unable to cope with the burgeoning size of the Internet.

2.2

15.

Most useful information on the Internet is available only in proprietary sites on a fee basis.

1.83

6.

Scholarly networks have transformed the availability of authoritative information on the Internet.

1.28

17.

Subject-based gateways are only available on a subscription basis.

1.2

5.

Commercially-driven subject-based gateways are the norm.

0.98

1.

There is now no free access on the Internet.

0.88

23.

Service delivery contracts now specify the range of networked resources to which students are entitled.

0.87

27.

Most libraries now provide their own selections of quality-based resources.

0.75

22.

Most higher education is ‘delivered’ off-site (in people’s homes/other designated centres).

0.74

11.

Closed networks (e.g. for business and for the HE community) have rendered current models of Internet use largely redundant.

0.67

18.

A new structure of metadata-based services is providing quality information services for a fee.

0.4

16.

Librarians are willing to pay for quality subject-based gateways.

0.12

 

Table 5 - Gap - More desirable than likely

No.

Proposition

Score

7.

Search engines are fully adequate to find high-quality academic information.

1.95

19.

There is a world-wide network of co-operating subject-based gateways.

1.43

25.

Almost all new scholarly information is directly available through the Internet.

1.42

10.

There are now widely applied standards for describing the subject content of Internet sites.

1.36

21.

Search engines can find information even inside proprietary databases.

1.05

13.

All significant public, academic, scholarly and academic libraries in the UK are sharing resources through the Internet.

1.03

3.

Subject-based gateways are largely maintained through centrally-provided (academic/government) funding.

1.00

28.

Most academics and students start their quest with library-based resource pages.

0.80

4.

Applications of artificial intelligence have largely replaced institutional and subject-based gateways.

0.55

8.

All major UK Internet access projects (such as national and subject-based gateways) are supported by the same central computer system.

0.55

2.

All main academic areas are covered by subject-based gateways.

0.52

24.

More scholarly articles are published electronically than through print-based journals.

0.52

26.

Most journal-based abstracting and indexing services have extended their coverage to include network resources.

0.50

12.

Sophisticated subject-focused search engines are the normal method of gaining access to Internet resources.

0.49

20.

There is a single gateway for quality academic resources for the UK academic community.

0.41

Fully adequate search engines form a small aberration here, because although most people wanted these and didn’t expect to get them, their potential arrival was not adjudged important. Leaving this item aside, there is again a strong relationship between what is seen as important and what people see as more desirable than likely, with the same five most ‘important’ items featuring prominently on the ‘desirable but not likely’ list. Clearly people would like to see an organised network of libraries served by effective subject-based gateways, but are not optimistic. Will JISC be in a position to continue to help and, if not, are people right to be pessimistic about academic network developments?

 

Cataloguing costs

In order to understand the financial implications contained in the evaluation of Subject Based Gateways, we have attempted to quantify the costs of cataloguing and providing support for access to high quality networked resources on the Internet. There is possible duplication of effort between the Subject Based Gateway providers and many of the Higher Education Institutions who have their own institutional gateways. This duplication of effort could offer a possible opportunity for the sharing of resources and thus open a potential market for the Subject Based Gateway Providers. When discussing the value of access to networked resources, it is important to bear in mind that cataloguing and record creation is only part of the cost, and separating the cost from the whole set of services is difficult.

Limitations and problems of comparability

Any calculation of the costs involved in cataloguing networked resources needs to be handled with caution. There are a large number of variables that need to be taken into account during any comparison of costs, for both the Subject Based Gateway Providers and HEIs.

As one SBG provider put it:

"The (subject gateway) services differ widely in the number of value-added services they offer. Some offer a wide variety of primary data. Others have a substantial training and awareness element."

There is also the difference in the average length of descriptions, number of keywords assigned. Finding and assessing information on the Internet is a time consuming activity as the resources need to be discovered, the creators identified, the accuracy and relevance of the content evaluated (which can take time depending on the time of day, size of file and number of graphics).

Even with automation of some of the discovery processes, the human element of evaluating resources is still a major source of work. The time taken to catalogue a record can vary greatly - with some technical sites requiring half an hour or so to review properly. Only then can a record be created allowing access to the information. Again there are considerable differences between SBGs as to the depth of indexing and classification.

It is very difficult to factor into the calculation the time spent on maintenance, which is necessary to achieve the high standard of information the subject-based gateways achieve. As one of the providers suggested:

"with our current collections policy, content editing, style guidelines, cataloguing procedures and regular link checking, we feel confident that our catalogue records are of a high quality"

The services are usually part of a bigger set of services such as training, outreach, promotion, research and development.

The level and productivity of staff used to undertake the work has a large impact on the costs - automated time to harvest sites, support staff, cataloguers and subject specialists. Experienced staff tend to be more productive and this reduces costs. According to one provider this can vary from 20 to 45 minutes.

Many HEIs have set up their own institutional gateways. These range from collecting links of useful sites to the finding, assessing and cataloguing of sites in a similar manner to the subject based gateways. Direct comparisons are difficult. However we can get some impression of the time being spent on cataloguing and therefore the costs to HEIs.

Existing data on cataloguing costs

There appears to be very little published information on specific costs of cataloguing Internet resources. We did find a substantial amount of tangential information discussing practical and theoretical issues concerned with cataloguing Internet resources, such as discussions on the question of organising the networked resources; control of the catalogue in a networked environment; analysis of bibliographic data contained and bibliographic control of the resources; and access to their resources. The question of costs was not even mentioned as an issue in the material that was reviewed.

This based on an extensive literature and Internet search to identify any previous investigations in this area. The following search terms were used in the search strategies:

Acquisition

Catalogue

Cataloguing

Cost

Costs

Electronic

Digital

Internet

Resources

The following search engines were used: Altavista, Infoseek, Yahoo, and Metacrawler. LIS specific sites, electronic journals and the archives of some of the mailbases discussion lists were searched:

BUBL

PICK

UKOLN

IFLA

EARL

BL

NISS

We used BUBL's gateway to search the archives of all the "library" lists and the entire "Librarianship and Information Science" list. To double-check, we also searched the archives of specific lists like: lis-cigs; lis-eLib; lis-link; lis-lirg and lis-sconul-acopi.

University costs

None of the HEIs consulted had undertaken any costing of cataloguing resources on the Internet. A number believed it was an exercise that should be undertaken, but were hesitant to do so. There was an awareness that it can be an expensive undertaking, yet how does one measure a high quality support to students?

One gut reaction placed the figure at up to £25 per item, but this was a perception rather than a proven fact. The trend of HEIs appointing people with a specific remit to provide Internet support indicates the importance the institutions are placing on networked resources, and the cost of accessing these resources will increasingly come into consideration. The salary range of staff undertaking these roles appears to vary between £16,000 and £23,000 p.a.

One example included an institution producing evaluated links to support their curriculum, acting as a stepping stone for students to make better use of the Internet for information. These links are often provided and updated by the subject specialists. In addition to cataloguing the resources, the links will have to be checked and updated, and maintenance will play an increasingly important part in the costs of providing the service. One institution estimated it takes their staff approximately 1 hour to catalogue a record from discovery to completion of the record.

From these figures it appears that the cost of cataloguing an Internet resource for an institutional gateway ranges from £10 to £25.

SBG provider costs

In order to try and measure like with like, a group of SBG providers pulled together a set of agreed criteria to determine which activities should be included in their cost calculations, and which to exclude.

They included in the cost calculations the following tasks:

Checking and maintenance of records

Location and assessment (acquisition)

Record creation

Collection management

Development of subject specialised classification

Management of cataloguing staff (including, distributed teams, volunteers and paid)

Staff training and development

Technical support and maintenance

Tasks excluded from the calculations:

Provision of primary content

Thesaurus work

Database development

Training and awareness

Publicity and dissemination

Technical research and development

Management of non-cataloguing staff

Liaison with funders

Future strategy

International collaborations

During a six-month period the SBG providers added between 350 to 3,057 records, with three of the five providers surveyed adding between 948 to 1,326 records. Staff costs for the six-month period varied from £9,600 to £24,950. Based on the individual responses from the providers, the cataloguing costs varied, from £3.20 up to £34.00 per record.

There are a number of activities the subject based gateway providers undertake to add value to the quality and development of their service, these activities are not measurable, but should be taken into account when reviewing costs. This includes activities such as the further development of the gateway, improving the search facilities and developing selection criteria and cataloguing standards. All are involved in developing better access to information for their users, including thesaurus construction, and the development of subject headings and authority files. In addition the SBG providers offer a range of training covering their services. All have developed stringent cataloguing and selection criteria to support the selection and acquisition of records. In the creation of the records the role and value of the subject specialist to enhance the service is not to be underestimated The time taken to undertake maintenance varies between subject based gateway providers depending on their subject coverage, this again has implications for their costs. Although software products such as Link-checker assist the maintenance, the follow-up work needs to be undertaken to ensure that the gateway links are reliable. Maintenance costs will come increasingly into focus as the SBG databases increase in size and age.

Amount of cataloguing activity in the HE sector

The cataloguing activity within the subject gateways is high, enabling them to deliver value added services. Institutions are undertaking the cataloguing of electronic information on a number of different levels - from listing links, to cataloguing electronic publications along the same lines as the subject based gateway providers. Evidence of the importance of this activity can be seen in the increase in number of university posts for Web site information resource officers being advertised. Although the main emphasis of these posts is to manage the Web pages about the institution, some universities now also provide considerable subject-based gateway resources. This is often done by individual departments in direct support of their courses.

A rough calculation of cataloguing costs in HEIs can give a general order of magnitude of the resources being spent on Institutional Gateways in the UK. A more detailed survey would be necessary to verify these figures and to provide some raw data on which to base these calculations. In order to arrive at a basis for the calculation we have made the following assumptions (which are open to modification and refinement):

No. of HEIs in the UK (approx.)

200

Proportion of HEIs with institutional gateways

50%

Average no. of subject resources catalogued per institution

100

Average time take to catalogue a record

1 hour

Salary range for Internet cataloguers

£16,000-£23,000

Average staff costs per record

£10

Alternative estimate of staff costs per record

£25

Average no. of records added or amended per 'online' institution per year

30

A university or HE College with an institutional gateway of 100 records would have spent between £1,000 and £2,500 on cataloguing. If 50% of all HEIs have an institutional gateway this accounts for a total expenditure in the UK of between £100,000 and £250,000. If there were a 30% turnover of records a year the annual ongoing staff costs would amount to between £30,000 and £75,000 for cataloguing.

Potential market for catalogue records

One possibility, previously mentioned is the sale of records to universities. The SBG providers could offer HEIs the option of rationalising their costs by selling them centrally created catalogue records.

One of the benefits would be the provision of a high standard of cataloguing, greater consistency, wider coverage and greater comprehensiveness within any chosen subject.

However a further consideration is that one of the conditions of JISC funding for the SBGs is that the services should be made freely available to the HE community. Whether this applies to an ongoing service based on regular updates of the catalogue records (or indeed the sale of the catalogue records themselves) is not clear.

 

Models for Access to Network Resources

One current model being suggested for the future of SBGs is the development of a central co-ordinating agency or network centre. The central agency idea is based on the AHDS (Arts and Humanities Data Service) model of a central co-ordinating body for a distributed services. The distributed services are provided by third parties (in this case the HEIs could bid for this). JISC's Committee on Electronic Information recently put out a call for tenders for parties interested in providing a network centre of this type to start from April 1999. In the meantime funding for the existing SBGs has been extended to July 1999.

International co-operation plays an important part in this study because it allows for wider co-ordination of subject coverage with very little increase in the level of resources needed.

Following our review of the current situation, the technical developments that are going on and our understanding of users' requirements we have developed three possible alternatives for JISC support of access to network resources:

Model 1 - JISC pays of the continuation of SBGs

Model 2 - JISC pursues a market development path for SBGs

Model 3 - JISC ceases to fund SBGs

Each of these models is described in more detail before we go on to consider some of the likely effects of following each.

Model 1 - JISC pays for continuation of SBGs

In Model 1 JISC effectively turns the development projects into services. There would be some rationalisation as some aspects of the service would be centralised. There are three sub-models:

1A - Continued separate services

1B - Some rationalisation of services

1C - Highly centralised service

 

Model 1A - Continuation of SBGs (No change)

Model 1A represents continuation of subject based gateway services by JISC without any major changes in direction. Some change is inevitable however because of the continued development of alternative services in other countries and development of common search interfaces and standards between services. Mirroring agreements and development of common cataloguing standards and search interfaces would help to extend these services and more them more widely available to the HE community.

Model 1B - Continuation of SBGs (Some rationalisation)

Under sub-Model 1B, the support activities would be centralised and there would be some rationalisation of staff and accommodation costs. There would also be some saving by centralising the computer hardware and software support. The individual subject gateways would continue to be managed by the current partners who would be responsible for the content and organisation of material on the gateways. This option provides some of the savings associated with lower hardware and accommodation costs whilst avoiding the costs associated with increased co-ordination of the gateway content. Marketing and promotional activity would be pooled and there should be benefits associated with a single marketing strategy for all the JISC funded SBGs.

Model 1C - Continuation of SBGs (Centralised service)

With sub-Model 1C, the services would be highly centralised with effectively one body responsible for all the subject areas covered by the subject-based gateway. The gateways would be available via a single interface. One of the general gateway providers could form the basis for a centralised service.

Organisational culture

It is now important for the SBGs to move from the research and development culture to a more service-oriented culture. The skills needed to conceptualise and set up a service are not the same as those required to continue running a service. As part of this review, job descriptions would need to be revisited to ensure that the appropriate balance of skills was available for the services.

A key component of the service-oriented culture would be the development of service level agreements with JISC (on behalf of the HE sector in the UK) and some targets for productivity in terms of the number of resources covered and the promotional activities. A better knowledge of users would be needed and we suggest that JISC should require service providers to develop generic user profiles as a means of ensuring that they focus on their core users.

It is important to recognise that a shift from a development to a service culture is likely to have a significant impact on staffing. As an example, when the Education Management Information Exchange (based at the National Foundation for Educational Research) moved from development mode after four years in 1984, two-thirds of the staff were replaced in Year 1 and overall staffing had doubled within four years.

Model 2 - JISC pursues a market development path

In Model 2 we envisage a similar process of refocusing the SBGs towards a service based approach but with a strong emphasis on commercialisation. However in order to become more self-supporting, JISC would invest resources to help bring the SBGs to the marketplace.

A Managing Director with a primary marketing role would be appointed to run a co-ordinating and marketing organisation on behalf of the SBGs. The remit of that person would be to prepare the individual services and market them to sponsors in the public and private sectors with a view to raising funding for 50% of the SBG requirements. Under this scenario JISC would be committed to providing 50% funding for the services on a matched funding basis. Until a market for these services has been fully developed it will be difficult to predict the long-term viability of fully self-supporting services. The target market for funds or sponsors might include the following:

Research councils

Learned societies

Academic publishers

Electronic service providers

Other business sources

Individual institutions

Public libraries

DCMS

The primary market for sale of catalogue records would be Higher Education Institutions in the UK.

This success of this option would depend on the calibre of Managing Director appointed and we envisage a package with a competitive salary for 18 months and additional incentives in the form of a bonus based on the level of matched funding obtained and payable in the two years after the funding has been secured, possibly with some element of recognition for the number of gateways whose future has been secured in this way.

If the appropriate funding is found, the service will be maintained after 18 months. If it fails the service will be wound down during the 6 months after the end of the Managing Director's initial 18 month contract.

As part of the preparation for market commercialisation, the services would have to demonstrate their viability by setting productivity targets. In this way they can demonstrate their market potential.

Model 2 depends on some degree of centralisation, so that the branding and marketing of the services can be co-ordinated. There are two sub-models which models which explore different levels of centralisation of services:

Model 2A - Market based approach (rationalised services)

Model 2B - Market based approach (fully centralised service)

Model 2A - Rationalised services

Under Model 2A there would be strong rationalisation of the services so that there is effective branding and co-ordination of access as well as a single marketing function. This could be extended to the development of a single search interface for all the services, possibly ROADS technology. The existing services providers would continue to concentrate on resource discovery and description (cataloguing) and would be responsible for maintaining their own computer hosts and databases.

Model 2B - Fully centralised service

The alternative option under this scenario is to fully centralise management and control of the service with a single organisation responsible for development and delivery of an integrated service. The Managing Director would have a more immediate, interventionist role in this. This model would allow for contracts to be placed with service providers or for centralising all the activities.

Model 3 - JISC ceases to support SBGs at the end of current funding

The third model is for JISC to cease funding the subject gateways when the current extension to funding runs out in July 1999. This means that there is about a year to complete an orderly wind-down of services. Those services that are able to secure funding from other sources will be able to continue. Our concern with this option, is that unless the services are snapped up by a generous funder (possibly in the commercial sector) they are unlikely to survive. The existing staff have been selected for their research and development expertise and not necessarily for their promotional or commercial experience. (We recognise that a great deal has been achieved by existing providers in this area). It is unlikely that without additional support and expertise they would be able to secure future funding to guarantee the continuity of their services.

Alternative approaches

The original remit of the project suggested that there were a number of alternatives available for locating resources on the Internet. The key issue that the SBGs attempted to address was the need to identify and describe high-quality resources. The original models that were put forward were:

It was always acknowledged that there was some overlap between these alternatives and that they were not necessarily mutually exclusive. This has been borne out by our discussions with service providers and with users and their representatives in the focus groups. For instance there is a commonly held view that institutional gateways have a role in directing students to course-specific material (in much the same ways as lecturers provide their own reading lists for each course) whereas the subject gateways are more useful for general project work and research - where a wider perspective is required. There is the added factor that individuals providing input to the institutional gateways are often also providing input to the national subject-based gateways.

There has been discussion for some time (probably since the creation of the SBGs) about the future model for retrieval of high quality resources on the Internet.

Other options that were initially considered in this project are:

Search engines

Institutional gateways

Commercial services

Search engines

The current state of search engine technology does not provide access to high-quality information resources. Even moves towards weighted searches and relevance ranking have not come anywhere near addressing the problem of evaluating resources and filtering out low-quality or irrelevant resources. Some search engine-providers are beginning to develop a more structured approach to locating resources, Yahoo! is an example, although the quality issue has still not been resolved. Some providers are moving towards selected human intervention to provide some level of site evaluation.

This provides a possible avenue for securing future funding for the Subject Based Gateways. For instance Internet resource directories and search engine providers such as Yahoo or Alta Vista might consider acquiring and funding the content of SB services in order to provide added facilities for their users.

Institutional gateways

Although there is considerable activity in this area and there is some duplication of effort, without some means of co-ordination (i.e. by means of national SBGs) there was little prospect of the level of provision or of consistency of quality that is provided by the SBGs. The quality and depth of coverage of the institutional services is very patchy. They could be enhanced by incorporating catalogue records provided by the existing SBG projects.

Commercial services

JISC has no control over commercial services, although it can purchase existing services if it chooses. The commercial services tend to be proprietary which limits the scope for cross-database searching. The pointers on most commercial services are to individual bibliographic records, or increasingly to electronic publications.

There is a consensus among the librarians and users surveyed that there is an on-going need for subject based gateways. Even those individuals who are sceptical about the long-term future of SBGs (because they believe that technology will deliver solutions that bypass the need for SBGs eventually) acknowledge that in the immediate future (3-5 years hence) SBGs should continue to be provided. The question that this raises is what is the future scope of subject gateways and how does JISC address subject coverage?

 

Discussion

Impact on the HE community

Each of the scenarios put forward in the three models will have an impact on the academic community. The impact will not be uniform throughout the HE sector because of variations in the degree to which each subject is covered and the different levels of usage of subject based gateways. In some disciplines JISC funded SBGs will have had a major impact on the range of resources available to students, lecturers and researchers in HE institutions. For instance students in selected institutions and in specific disciplines have benefited from the promotional and training activities of the SBG providers and have been able to make direct use of high quality Internet resources. They have also had an indirect effect by being available to lecturers preparing course materials and to subject librarians compiling their own institutional gateways.

However we must also acknowledge that there is a large academic community that has not benefited from SBGs for a variety of reasons:

SBGs are a part of the range of electronic and physical information resources that are available to the academic community and to the wider public. As such they are an important component in the array of information search tools available to users.

In this section of the report we consider the impact of the different funding scenarios for SBGs on the academic community.

Model 1 A - Continuation of SBGs - on current basis

Funding of SBGs on the same basis as at present will help ensure that the use of the services will continue to increase. The SBG providers have developed marketing strategies to inform target audiences about the services and to continue the growth in uptake of the services. Although there are widely acknowledged problems in measuring levels of usage, the general trends revealed demonstrate a steady increase in the level of activity resulting from use of the SBGs.

From the academic point of view this means continuity of service and the prospect of building on their familiarity with resources. On the negative side, it means that the patchiness of coverage will also remain and it limits the scope for development of individual services and the creation of new services. Our consultations and investigations of the existing services reveal major gaps in coverage of subjects such as: physics, agriculture, mathematics, and chemistry in the UK. To some extent this may be addressed by the development of agreements to mirror other European sites and sites in North America and Australia.

Rationalised services (Models 1 B and 2A)

Rationalisation of existing services and moving towards a common infrastructure for delivery of the services will give users a more recognisable brand with greater consistency between services. Working with a common search engine on a single computer allows for cross-disciplinary searches. Leaving the control over the content, and to a certain extent the appearance of the individual gateways to a distributed group of providers will help to ensure that close links that have been established with the subject-based communities is maintained. The valued access to the specialist knowledge of providers and their network of workers and volunteers would continue under this scenario.

Centralised services (Models 1C and 2B)

A centralised service, possibly based on one of the existing providers would allow for stronger branding of the service and closer integration of the individual subject domains. Access via a single classification framework would allow greater flexibility in searching across existing subject domains. There would also be a mechanism for ensuring greater consistency between subjects in terms of appearance and treatment of resources. One comment that has come through strongly from our consultations is that the granularity of existing SBGs varies considerably. A general service could attempt to cover the full range of disciplines to some degree, with the main effort being concentrated on the established subject areas.

The centralised service, with an integrated subject coverage policy could adapt its acquisition policy to reflect changing priorities and needs within the HE community. In this way it could be more responsive and adaptable than the existing framework. However there is a danger of some loss of continuity of the existing, separate services.

It may be possible to initiate more ambitious promotional activities. With a single identity there are greater opportunities for the service to build on the exposure that individual SBGs have in universities and colleges.

On the down side there is the loss of the individual character of the SBGs and a danger of greater remoteness from the subject communities being served. One of the perceived benefits of the existing SBGs is that they have developed close links with the subject communities that they serve. Some of them have been able to offer additional services to complement their own coverage. For example OMNI provides access to the Medline database and EEVL provides access to electronic journals in engineering as well as to selected databases.

Model 2 - Market development

This model has all the benefits of Model 1B, because of rationalisation of the support services and infrastructure. As in Model 1B, there is better integration of services without loss of the individual identities of the services. There is the possibility of cross-discipline searching and there is also the possibility (using ROADS technology) of better integration with external resources such as the Finland Virtual Library.

A higher level of support can be provided for users with a single point of contact for all the subjects. Training could be co-ordinated so that specific institutions are targeted across several subjects at once, rather than on a subject by subject basis.

With the diversification of funding sources there is a unique opportunity for the services to forge closer links with the subject community. This is especially the case if the research councils or the learned societies become involved. As well as the possibilities for integration with other services (such as OPACs and electronic journals), the services could provide access to commercial services including online bibliographic databases.

This option does not address the problem of continued patchiness of subject coverage. There is also the potential loss of services if matched funding is not found for some of the SBGs exacerbating the patchy coverage.

Development of future services will become more market-driven and will depend on users buying into the process. The biggest risk of the model is that there is no guarantee that an adequate level of additional funding will be secured by this method.

Model 3 - Cease funding

The main advantage of Model 3 from the users' perspective is that the money currently spent on SBGs could be re-deployed to support the development of other services. Many parts of the academic community are not currently covered by SBGs and of those that are, many potential users are unaware of their existence or choose not to access them. It may be that alternative developments are of more general benefit. For instance loss of SBGs may increase academic pressure to improve search engines (by more customisation of Silver Platter etc.). Group purchase of commercial resources and services by academic institutions could be brought forward in the way that BIDS has developed to provide access to commercial bibliographic databases. This could also result in diversion of resources to other activities such as training end-users to use search engines more effectively and to develop critical evaluative skills applied to Internet resources.

There are existing gateways in other countries, notably in Finland, the United States and Australia. Some effort would have to be spent on re-educating existing users of JISC-funded SBGs to use the alternatives.

A great deal of voluntary effort goes into the development of SBGs and this energy and commitment could be re-directed into other potentially more productive areas such as the development of institutional gateways, development of academic search engines and other strategies for providing access to Internet resources.

It became clear from our consultations, that alternatives to SBGs are speculative in nature and that there is no immediate replacement for them at the moment. It would inevitably take some time to develop and establish a new service or services to replace the SBGs during which time the users of existing services would be left in the lurch. Loss of specific services to dependent users could be damaging and the loss of confidence in this type of resource (SBGs and national gateways) could be irreparable, creating the damaging view that Internet services are inherently unreliable. The uptake of Internet resources by academics and students could be affected by withdrawal of funding, especially when the existing promotional activity stops.

Financial implications

The financial models developed in Table 6 below are based on current estimates of JISC expenditure on the subject based gateways plus BUBL. The full list of services covered in this evaluation is:

ADAM

Biz/Ed

BUBL

CATRIONA

EEVL

IHR-Info

OMNI

ROADS

RUDI

SOSIG

NISS has been excluded from this model because it is already funded as a service and is not currently under review. CAIN is a content based service rather than a subject gateway and it was not considered appropriate to include it either.

Although we anticipate that the services may develop further we have not incorporated this into Models 1 and 2. The models provide a mechanism for testing different funding approaches for the SBGs and we have done this by fixing as many variables as we can, so that a clear view emerges of the financial implications of the different models.

Table 6 below can be used as a dynamic tool to test different scenarios. Formulae have been built into the 'Total' cells for ease of updating the model. Our intention is to make this available to JISC as a spreadsheet at the end of this project. The timeline starts from the end of the current extended funding.

Table 6 - Financial implications of each scenario

Option/Cost Heading

Year 1

Year 2

Year 3

Year 4

Year 5

           

Option 1A - Continued, no change

         

Present level of JISC funding

£850,000

£850,000

£850,000

£850,000

£850,000

TOTAL JISC funding

           

Option 1B - Continued, rationalised

         

Present level of JISC funding

£850,000

£850,000

£850,000

£850,000

£850,000

Managing change

£50,000

£30,000

£0

£0

£0

Savings from rationalisation

£0

-£100,000

-£100,000

-£100,000

-£100,000

TOTAL JISC funding

           

Option 1C - Continued, centralised

         

Present level of JISC funding

£850,000

£850,000

£850,000

£850,000

£850,000

Managing change

£50,000

£50,000

£0

£0

£0

Savings from centralisation

£0

-£150,000

-£150,000

-£150,000

-£150,000

TOTAL JISC funding

           

Option 2 - Marketing (successful)

         

Present level of JISC funding

£850,000

£850,000

£850,000

£850,000

£850,000

Managing change

£50,000

£30,000

£0

£0

£0

Managing Director

£50,000

£25,000

£0

£0

£0

Marketing bonus

£0

£10,000

£15,000

£0

£0

Savings from rationalisation

£0

-£100,000

-£100,000

-£100,000

-£100,000

Matched funding

£0

-£165,000

-£380,000

-£380,000

-£380,000

TOTAL JISC funding

           

Option 2 - Marketing (unsuccessful)

         

Present level of JISC funding

£850,000

£850,000

£850,000

£850,000

£850,000

Managing change

£50,000

£30,000

£0

£0

£0

Managing Director

£75,000

£38,000

£0

£0

£0

Savings from rationalisation

£0

-£100,000

£0

£0

£0

Savings from closedown

£0

£0

-£850,000

-£850,000

-£850,000

TOTAL JISC funding

           

Option 3 - Cease funding

         

Present level of JISC funding

£850,000

£850,000

£850,000

£850,000

£850,000

Savings from closedown

£0

-£850,000

-£850,000

-£850,000

-£850,000

TOTAL JISC funding

These financial models are based on 1998 constant £ and do not take inflation into account. We have used consolidated figures on the current annual costs to JISC of funding the Subject Based Gateways. The figures for savings from rationalisation are based on the anticipated savings identified by service providers during the interviews. The Year 5 figures would apply to all subsequent years.

These figures do not take into account the possibility of developing the services, by extending subject coverage for instance. If the subject based gateways continue (whether funded by JISC or not), it is inevitable that they will have to develop in order to cope with an expanding universe of information available on the Internet. Under this scenario any additional development costs would be shared between JISC and the matched funding partners.

Model 1 - Continuation

Under Model 1 we considered three sub-options: no change, rationalisation and centralisation. The rationalisation option could generate savings in the following areas:

Promotional and marketing activities

Hardware and software support infrastructure

Project management

The rationalised service model includes expenditure to upgrade one of the existing systems to run all the SBG services funded by JISC. There would also be the costs associated with the organisational and management changes. There would be some loss of support in kind currently provided by the host institutions and this limits the benefit of the rationalisation.

Eventually, by Year 3 the service would be running at a lower cost level than at present (in 1998 constant pounds) and these savings would be made year on year.

In Model 1C there are greater potential savings arising from centralisation of the services. The overheads associated with co-ordinating several centres would disappear. It would probably cost more to implement, and this is reflected in the higher 'managing change' costs in Year 2 (compared with Model 1B). However the ongoing savings would be greater from Year 3 onwards.

Model 2 - Market development

Model 2 requires an on-going funding commitment from JISC (at a reduced level). For this approach to work, it will be necessary for JISC to invest in the marketing of the services to potential funders. This would need to be done at a senior level by a persuasive and skilled marketing professional. This model can be summarised as "spending money to save money". It depends on a degree of rationalisation, as in Model 2A and this is reflected in the managing change costs and the savings from rationalisation. We have concentrated on the rationalisation model rather than the more radical centralisation model (Model 2B) in the financial analysis. The great financial benefit of this option is that provided the Managing Director meets his or her targets JISC would reduce its financial exposure. Even in scenario where all the existing services continue (Model 2 successful) the JISC funding is reduced by 50%. If some or all the services fail to obtain matched funding the JISC exposure would be reduced even more.

We have split Model 2 into two scenarios - successful marketing of all the SBG services, and unsuccessful marketing of the services and eventual withdrawal of funding.

In the successful scenario, matched funding is secured for all the services and they continue to develop at the current levels. In practice some may expand and others may contract or cease to operate altogether.

Potential savings from rationalisation of the services (based on feedback from service providers) would be of the order of £100,000 per year. However there are costs associated with managing the required changes and upgrading one of the host services as well as transfer of systems and records to a centralised facility. An alternative is to put out the hosting of individual services to contract (with existing providers initially) and provide an integrated front-end to allow searching across all the services. Development of integrated searching facilities using ROADS for instance might provide one avenue for this. In the first year the costs would increase because of the costs of employing a Managing Director and the costs associated with implementing the rationalisation of services. In the second year costs would be less than the current level of funding because of the savings arising from rationalisation and the start of some matched funding. We would expect that other revenue generating activities such as sale of advertising space and sale of cataloguing records could contribute to the finances of the service. These have not been specifically identified in these cost models because of lack of data to support the market size estimates. From the third year onwards the costs to JISC of providing the current level of service would be substantially less than 50% of current costs because of savings due to rationalisation and the 50% matched funding.

If the marketing scenario (Model 2) is unsuccessful the ongoing JISC liability very quickly runs down after the second year. In the first year the costs increase, in the second year they decline moderately and in the final year there are no costs to JISC associated with support of SBGs.

The existing services depend on considerable goodwill from volunteers with subject expertise who submit resources for consideration by the SBG providers and from host institutions that often provide accommodation and other overheads either a minimal cost or free of charge. These invisible contributions could be lost during the commercialisation of the SBGs.

Model 3 - Cease funding

The obvious financial benefit of this option is that JISC can draw a line under its financial support of SBGs at the end of current funding and there is no further financial commitment on its part.

Development of the subject-gateway approach

International co-operation

There is already considerable effort being spent on co-operation with international partners, for example the DESIRE project and the ROADS collaboration with the Finland Virtual Library. Both of these efforts provide models for the way forward for co-operation between service providers to ensure more comprehensive coverage. A series of subject views can be developed which allow users to search several subject resources simultaneously and obtain a listing of resources by service. A refinement of this process would ensure that duplicate references were removed from any consolidated listings.

Another avenue being explored is reciprocal agreements for mirroring sites. This offers the advantage of reducing international traffic, and especially trans-Atlantic traffic which is likely to become restricted as institutions are being asked to pay for the costs of international bandwidth. Mirroring offers scope for extending the subject coverage of Subject Based Gateways in the UK. It has been suggested that some additional effort to tailor the mirrored site to the UK's needs would provide a valuable adjunct to the existing services, by extending the subject coverage. However mirroring should not be seen as a replacement for UK-based gateways, because of the on-going requirement to reflect the needs of the UK HE community.

The UK has developed a leading role in the development of the subject-based gateway approach with the emphasis on high-quality, authoritative resources. There is a strong feeling that the UK can only continue to benefit for the work that has been done by maintaining and developing existing resources. Individual providers have developed co-operative arrangements with partners in other countries as part of an international network of collaboration. The development of ROADS has been a significant factor in stimulating collaboration between JISC funded services and those outside JISC's remit.

Commercialisation

There has been considerable speculation about commercialisation of services as a way of generating revenue to provide ongoing support. Individual service providers have explored different options and in some cases are generating significant amounts of revenue. However there may be conflicts between sponsors' interests and the selection policies of the providers.

Integration with other resources

As part of its long-term strategy JISC is exploring the development of a Resources Discovery Network Centre. In its Circular 10/98, JISC states:

The network Centre will have a mandate to provide a high-quality resource discover service across all disciplines covered by UK HE, where economically viable. It will achieve this by developing partnership and co-funding arrangements with institutions or consortia in the UK or overseas to provide resource discovery services, to agreed standards, across as wide a range of subjects as possible.

This approach allows for integration of different types of networked resources (from resource descriptions to bibliographic records, electronic publications and finally to raw data in digital form) under a common framework. The existing hybrid library developments are an example of this approach. OPACs are being made available via the Internet and some groups of academic and other libraries are beginning to develop union catalogues with cross-catalogue searching facilities. Digital data resources are also being developed, for instance the AHDS provides access to arts and humanities data. The SBGs provide a key element in this framework of access to digital information.

Another development is the work at the University of Wisconsin-Madison to develop a distributed architecture for Resource Discovery. Project Isaac links metadata from selected, reputable, authoritative sources to provide a single, virtual metadata collection. The team working on this have identified some of the JISC funded developments as possibilities.

 

Recommendations

Strategic approach

We recommend that JISC adopt a strategy for on-going support for the development of access to network resources with continued support for Subject Based Gateways as a component of this. This strategy should be expressed in terms of:

The SBGs have grown beyond the experimental phase into full-blown services. The previous approach of inviting bids and allowing a degree of freedom for institutions to develop their own approaches has worked well and there is now a good corpus of experience for JISC to build on. However it is now necessary to put this into an overall strategic framework. As well as co-ordination of subject coverage further work is needed on integration of the user interface. Some institutions (e.g. Heriot-Watt) have developed a single point of entry to these gateways by means of a menu of services. The next stage is to develop common protocols and standards for records on the different services so that there is the possibility of searching across more than one gateway at a time. This is particularly important for interdisciplinary topics.

Since the early draft of this report was circulated JISC has issued a document (Circular 10/98) inviting proposals from institutions to provide a central co-ordinating agency or Network Centre for services provision. This is part of a proposed strategy to move from a project basis to a service basis for the Subject Based Information Gateways. The circular does not specifically map the proposed subject coverage or prepare a strategy for this. This would be part of the role of the Network Centre. We strongly recommend development of a subject framework which is published so that bidders (consortia or individual institutions) can focus their bids in areas where needs or priorities have been identified. We understand that political nature of establishing priorities and the difficulties involved and suggest that a process of consultation would be necessary to ensure effective coverage. These priorities should take into account existing provision in other countries. There are already a number of well-developed subject gateways overseas, notably in the United States and in Finland and mirroring suitable sites (supplemented with some UK-oriented material) rather than establishing parallel UK-based gateways would be a cost-effective way forward. An alternative is to enter into collaborative arrangements with other organisations overseas to jointly develop resources.

Rationalisation

We recommend that JISC explore the detailed costs associated with rationalising the central support functions of the SBG services. The editorial control of the services should continue to be devolved to centres of excellence with the required subject expertise. However support functions such as host computer services, marketing and project management should be centralised. There is a significant level of duplication of effort and of resources to maintain and provide the different SBG services. Centralisation of hardware and software support will also allow for greater eventual integration of services.

It is important to ensure that the editorial control is kept close to the user communities so that they are able to reflect their users' needs more accurately and anticipate developing requirements. Detailed subject knowledge is also an important part of the quality of service provided by the Subject Based Gateway services and suggests that a distributed model for data gathering is appropriate.

Matched funding of services

We recommend that JISC seek to market the SBG services to potential funders with the objective of obtaining matched funding. For this approach to work, JISC would need to invest money in the marketing activity during the first 18 months. Thereafter savings of up to 50% of existing commitments would be realised.

The advantage of this approach is that if individual services fail to obtain matched funding, JISC is not exposed to the ongoing expenditure. The disadvantage is that the uncertainty hanging over these services will continue for up to a further 18 months. We consider the 50% funding target is achievable for the SBGs and indeed some are already on their way to meeting this target.

Commercialisation of services

Some of the gateway providers have identified opportunities for commercialising their services. Sale of records, subscriptions to closed areas of the site, and sale of advertising have all been considered at some point. One of the roles of the Managing Director will be to explore opportunities for generating revenue from these services. We recommend that this is built into the matched funding targets so that there is some flexibility in exploring ways of reducing costs to JISC. Our initial work on the costs of cataloguing suggests that there could be considerable savings to individual HE institutions if they were able to buy records from the SBG providers. The benefit of this to JISC is that this could provide a significant revenue stream for the services.

A possibility raised during one of the interviews was selling the SBG databases to one of the existing search engine owners. Some of the search engine services are looking for ways of adding value to their services and for ways of setting themselves apart from the competition. In order for this added-value service to be viable there would have to be a commitment to maintaining and updating the information. This is a further service that could be supplied by the SBG providers. The respondent who raised this possibility went on to suggest that care would be needed in the choice of potential purchaser. Search engine providers were seen as being relatively neutral because they are not selling information services directly. Online database hosts such as Dialog may be less desirable because they have considerable electronic resources and there is the danger that they might either squash the gateways or use them as a means of marketing their own products.

Implementation

The recommendations in this report have had limited circulation. We suggest that the consultation is broadened by publishing the proposed models on the Web and inviting comments from interested parties. As well as the providers, we would suggest that representatives of the user community (students and academic staff) and their intermediaries (librarians and information officers) should be targeted during this consultation.

The next step would be to organise a meeting between the central co-ordinator (i.e. JISC task group or the proposed Managing Director depending on the model chosen) and the senior mangers of the services to work on developing an acceptable version of the model propounded here. For this event senior mangers of the services would be asked to address key questions such as:

Focus groups with potential funders (learned societies, research councils, electronic publishers etc.) would be used to tease out funding options before a detailed plan is produced for the implementation the service development.

Market research and planning will be necessary to explore some of the options for sale of commercial services. Because this is a relatively new market there are not established principles or corpus of experience to call on when trying to estimate the likely market for services. We suggest that there should be a particular focus on sale of catalogue records (metadata) to HE institutions in the UK. This could be based on a survey of the costs of cataloguing electronic records and the level of activity in HEIs. Selection of an appropriate sample of available material on institutional gateways could be compared to the coverage in the SBGs to estimate the level of overlap between the services.

 

Appendix A - Structured interview schedule

Question areas for interviews with service providers on Access to Network Resources

Introduction

We are reviewing different approaches to access to high-quality network resources. This study was commissioned by JISC to help them develop a strategy for the next 3-5 years. As part of this exercise we wish to talk to existing service providers.

This interview will take about 1.5 hours.

Nature of service

First of all, we would like to ask a few questions about the nature of your services and its target audience.

1 What is the subject coverage of your service?

Request any relevant literature.

2 Do you have a written selection policy for resources that are included on your gateway? If so, may we have a copy please?

3 What criteria do you apply to deselect resources on your gateway?

  1. Who is the service primarily targeted at?

Prompt: Level of student or academic, subject interests, geographical location

5 What is the estimated size of your target audience? Do not include potential audiences which are not a primary target.

6 Do you have any formal co-operation agreements with other services (including non-UK based services)? If so, which ones?

7 Are there any services that you feel are trying to cover similar ground as you are (i.e. competitors)?

  1. How to you publicise the service that you provide?

Prompt: via the Internet, traditional leaflets, training of academic librarians

9 Do you have any statistics on levels of use?

10 Do you apply any particular operating principles to deal with ethical considerations?

Prompt: privacy etc.

 

Costs

We are interested in the costs of operating SBGs and in their potential for generating income. My next few questions are aimed at building up the overall picture - not at finding ammunition to make cuts

11 Is there any readily available information about the costs of your services? If so, is this information broken down under general headings? Could we have a copy please?

Prompt: such as staff and hardware costs

12 What is the likely effect on your operating costs of co-operating with other JISC funded services or other gateway operators? If savings were made, what order of savings could be achieved in this way?

13 Does the service receive any financial support or attract any income other than the main JISC grant? If so, what?

14 Does the service pay full overheads to your University? [If appropriate] Are there any other potential funding sources for the service apart from JISC? What are they?

15 Is there any potential for commercial sponsorship of your service? In what ways?

16 What are your views on making charges for services to other institutions where staff use the service, or at the point of use? How would you view this?

Future developments

We are interested in your views on how SBGs may develop in future.

17 Do you envisage any co-operation amongst SBGs across subject boundaries or with services in other countries in the next 3-5 years? In what ways?

Prompt: Do you think that clumping of subject gateways is likely to occur?

18 What technical developments do you envisage occurring in the next 3-5 years and how do you think they will affect service provision?

Prompt: search engines, Metadata processing /handling

19 Do you envisage any changes in how SBGs will handle their information content (or allow users to do so) in the next 3-5 years? What changes?

Prompt: Filling gaps, continuity, Different types of searching

20 Do you think that changes in the general financial environment are likely to have an impact on SBGs in the next 3-5 years? What changes? What effects?

Prompt: Charging and microcharging, limiting user access

 

 

Alternatives

We have been asked by JISC to evaluate alternative ways of supporting access to high quality network resources. We would like to explore these with you.

21 What relationship do you see (or envisage) between your service and these scenarios:

Search engines

Commercial services (e.g. Silverplatter/Dialog)

Clumps of catalogues

Institutional gateways/local gateways

Centralised resources

Kitemarking of sites or institutions

22 What opportunities to you envisage for integrating with other services (e.g. international initiatives, institutional approaches)?

23 Do you have any formal partnerships with local or regional bodies? Are you planning any partnership? If so can you give us details?

24 What do you think of a centralised model, as opposed to the existing distributed model?

Thank you for your time. We shall be feeding this into our preliminary report to JISC at the beginning of March. All individual responses will be anonymous. JISC has indicated that they will publish the final report of this study, as part of their supporting studies series.

 

 

Appendix B - List of interviewees

Simon Bains

Cranfield University

Iain Baird

 

Michael Breaks

EEVL

Dan Brickley

Biz/Ed

Alan Dawson

BUBL

Lorcan Dempsey

UKOLN

Gordon Dunsire

Napier University

Nicky Ferguson

SOSIG

Tony Gill

ADAM

Geir Granum

EEVL

Daniel Greenstein

AHDS

Agnes Guyon

EEVL

John Harrington

University of Cranfield

Richard Heseltine

University of Hull

Paul Hofman

ROADS

Mike Johnson

CHEST/NISS

Linda Kerr

EEVL

Ian Lovecy

University of Wales, Bangor

Patrick McAndrew

EEVL

Roddy McLeod

EEVL

Alison McNab

Loughborough University

Dennis Nicholson

BUBL

Dawn Patrick

City University, London

Stephen Pinfield

BUILDER Information Services

Dave Price

Bodleian Library, University of Oxford

Mark Resmer

Sonoma State University, USA

Chris Rusbridge

eLib Programme

Debra Shapiro

Signpost Cataloguer

Leigh Sharpington

IHR Data

Catherine Sladen

Biz/Ed

Steven Smith

IHR Data

Tracey Stanley

University of Leeds

Sue Welsh

OMNI

Chris Willis

OPEL Project

Ian Winship

University of Northumbria at Newcastle

Bridget Wynstanley

HASSET

Ed Zedlewski

NISS

 

 

 

Appendix C - Focus group participants

Celia Ayres

University of Reading

Michael Breaks

EEVL

Dan Brickley

Biz/Ed

David Campbell

CTI Clues

Colin Campbell

Gloucestershire Library Service

Sheila Cannell

University of Edinburgh

Dave Cook

JISC

Alan Dawson

BUBL Information Service Manager

Mark Denham

University of Glasgow

Jenny Flemington

University of Edinburgh

Michael Fraser

CTI centre

Richard Gedye

Oxford University Press

Alan Gilchrist

Independent consultant

Peter Gooch

Independent consultant

William Hann

Independent consultant

Frances Hendrix

LASER

Debra Hiom

Institute for Learning & Research Technology

Paul Hofman

ROADS

Robert Kiley

Wellcome Trust

Lesley Kumiega

National Perinatal Epidemiology Unit

Steve Lay

ICRD Group, University of Cambridge

Carol Lunau

National Library of Canada

Fiona McLean

British Library

Caroline Moss-Gibbons

Institute of Grassland & Environmental Research

Dennis Nicholson

BUBL

Una O’Sullivan

Open University

Clare Powne

Edinburgh University Library

Dave Price

Bodleian Library, University of Oxford

Angela Sabin

SOAS

Pauline Sowry

ADAM

Chris Stephen

HUMBUL

Geraldine Turpie

Bowker-Saur

Sheila Webber

University of Strathclyde

Ian Winship

University of Northumbria at Newcastle

Norman Wiseman

JISC

Astrid Wissenberg

Hybrid Library Project Manager

Emma Worsford

SOSIG

Janice Yeadon

Imperial College

Robin Yeates

South Bank University

 

 

Appendix D - The forecasting instrument

Subject-based Gateways and their alternatives

FORECASTING ACTIVITY

Name:

Organisation:

Please rate the likelihood, desirability and importance of the following propositions about Internet–related Information Services in seven years' time. After each proposition you are asked to indicate your views on whether it is likely to happen and whether you would like to see it happen by ticking one of the boxes on the scale provided in each row. If you feel unable to offer an opinion on any proposition please leave all the boxes for that proposition blank. You are then asked to pick out what you consider to be the five most important of these propositions, on the assumption that they have happened.

Except where otherwise indicated, these propositions related to world-wide developments.

It is now 2005 AD and:

1. There is now no free access on the Internet.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

2. All main academic areas are covered by subject-based gateways.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

3. Subject-based gateways are largely maintained through centrally-provided (academic/government) funding.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

4. Applications of artificial intelligence have largely replaced institutional and subject-based gateways.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

5. Commercially-driven subject-based gateways are the norm.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

6. Scholarly networks have transformed the availability of authoritative information on the Internet.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

7. Search engines are fully adequate to find high-quality academic information.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

8. All major UK Internet access projects (such as national and subject-based gateways) are supported by the same central computer system.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

9. Search engines are unable to cope with the burgeoning size of the Internet.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

10. There are now widely applied standards for describing the subject content of Internet sites.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

11. Closed networks (e.g. for business and for the HE community) have rendered current models of Internet use largely redundant.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

12. Sophisticated subject-focused search engines are the normal method of gaining access to Internet resources.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

13. All significant public, academic, scholarly and academic libraries in the UK are sharing resources through the Internet.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

14. Commercial exploitation of the Internet has created major problems of resource access for academic libraries in the UK.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

15. Most useful information on the Internet is available only in proprietary sites on a fee basis.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

16. Librarians are willing to pay for quality subject-based gateways.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

17. Subject-based gateways are only available on a subscription basis.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

18. A new structure of metadata-based services is providing quality information services for a fee.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

19. There is a world-wide network of co-operating subject-based gateways.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

20. There is a single gateway for quality academic resources for the UK academic community.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

21. Search engines can find information even inside proprietary databases.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

22. Most higher education is ‘delivered’ off-site (in people’s homes/other designated centres).

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

23. Service delivery contracts now specify the range of networked resources to which students are entitled.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

24. More scholarly articles are published electronically than through print-based journals.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

25. Almost all new scholarly information is directly available through the Internet.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

26. Most journal-based abstracting and indexing services have extended their coverage to include network resources.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

27. Most libraries now provide their own selections of quality-based resources.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

28. Most academics and students start their quest with library-based resource pages.

Highly likely ˇ ˇ ˇ ˇ ˇ ˇ Highly unlikely

Highly desirable ˇ ˇ ˇ ˇ ˇ ˇ Highly undesirable

 

Please tick one box only in each row

 

Please list the numbers of the five propositions that would be most important for Internet–related Information Services if they came about within the suggested time scale.

 

 

 

 

 

 

 

Please return the completed questionnaire as soon as possible and by 25 March 1998 at the latest to:

Information Management Associates

28 Albion Road

Twickenham TW2 6QZ

United Kingdom

Alternatively you can fax this to: +44 171 242 4858 or email it to:

100753.631@compuserve.com

 

List of respondents to whom the Delphi style instrument was sent

Klaus Adelhaud

Munich University, Germany

Barbara Allen

UIUC, USA

Mary Auckland

SOAS

Anne Barker

University of Aberystwyth

Tony Barry

Australian National University

Andrew Bevan

ADIS

Michael Breaks

EEVL

Joseph Cadieux

 

David Campbell

University of Aberdeen

Geoffrey Clare

University of Warwick

Helge Clausen

Royal National Library Denmark

Brian Clifford

Manchester Business School

Myles Clough

 

Dave Cook

HEFCE

Steve Cramond

University of Adelaide, Australia

Emma Cusworth

Aston University

Alan Dawson

BUBL

Lorcan Dempsey

UKOLN

Greg Newton-Ingham

University of East Anglia

Nicky Ferguson

SOSIG

Richard Gedye

Oxford University Press

Alan Gilchrist

Cura Consortium

Daniel Greenstein

AHDS

Hazel Hall

Queen Margaret College, Edinburgh

William Hann

Free Pint

Frances Hendrix

LASER

Richard Heseltine

University of Hull

Maggie Jones

National Library of Australia

John Kelleher

Tavistock Institute

Robert Kiley

 

Steve Lay

University of Cambridge

Joe Luis Borbinha

IP, Portugal

Carol Lunau

National Library of Canada

Fiona McLean

British Library

Alison McNab

University of Loughborough

Oliver Obst

University of Münster, Germany

Clare Paine

 

David Price

Bodleian Library, University of Oxford

John Price-Wilkin

University of Michigan USA

Albert Prior

Swets, Netherlands

Rowena Rouse

Oxford Brookes University

Chris Rusbridge

eLib Programme

Kelly Russell

eLib

Sam Saunders

University of Leeds

Rachael Stone

Department of Health

John Sumsion

University of Loughborough

Graham Thomas

University of East London

Geraldine Turpie

Bowker-Saur

Sheila Webber

Strathclyde University

Tom Wilson

University of Sheffield

Ian Winship

University of Northumbria at Newcastle

Astrid Wissenberg

Kings College London

Ed Zedlewski

University of Bath

 

 

Appendix E - Results of the focus group exercise to prioiritise issues

 

FG1

FG2

FG3

FG4

FG5

Quality assessment procedures in place to evaluate the resources provided.

[3]

3

1

2

3

Specialist search engine (searching both descriptions of rated subject websites round the World and pages of sites listed on relevant bibliographical databases).

5

1

2

2

2

A flexible search engine for the whole site (allowing, for example, restriction by type of resource: articles, discussion lists etc.).

5

1

2

2

9

Topic areas, where key resources on the theme are linked together intelligently.

8

1

1

1

8

Quality assessment procedures in place to evaluate the performance of the system.

[3]

2

6

2

7

User surveys conducted to assess requirements and satisfaction.

3

2

3

9

4

News of developments in the subject field.

8

6

4

3

6

Teaching and learning materials related to the subject – text.

7

4

6

6

6

Options for customising the site (e.g. adding link to University catalogue; choosing material from one or more parallel versions aimed at different countries).

2

5

10

10

2

Frequently asked questions lists – with answers.

7

3

11

9

12

Bibliographical databases with links to full text.

7

3

1

-

6

Reference tools – dictionaries, glossaries, directories (associations, databases, suppliers etc.).

8

2

8

7

5

Lots of current awareness and document supply options.

8

6

4

4

8

User registration system designed to provide profile of users. °

[3]

6

10

1

12

Users able to suggest additional links and to evaluate/comment on sites.#

6

5

7

8

6

Options for including delivery of print newsletters and journals as part of subscription¤.

7

2

3

11

9

Teaching and learning materials related to the subject – any medium.

7

4

4

7

10

Bibliographical databases with citations, abstracts and proper indexing.

8

2

5

-

6

An interactive events calendar for the subject field.

8

4

4

6

11

Encyclopaedias of the subject.

8

5

9

7

5

Contact with subject experts.

4

2

11

9

9

Dedicated conference and discussion areas*.

7

7

12

11

7

° seen as involving heavy administration but as important for ‘value-added push technology user profiling’.

¤ A key source for one participant.

# Seen as useful by some gateway providers but very difficult to manage.

* Variation here was between perception as "unimportant" and as "The main means of communication within a subject group".

Elements added by later focus groups:

Short descriptions/assessments of each resource.

   

2

2

5

Very wide range of sites checked by providers.

   

6

2

2

Human contact point.

   

2

5

5

Rapid loading and access.

   

6

3

4

Usable with a basic browser.

   

5

8

3

Open systems.

   

7

6

 

Research database and archival collections.

     

3

 

Subject focus for a subject ‘community’.

     

1

 

Integration at gateway level.

     

1

 

Contact between users (forums).

       

8

Search facilities.

       

2

Browsable directory of resources.

       

1

Link to existing printed material.

       

3

Responsiveness to users.

       

1

Suggestions at first group not added in this form later:

Quality evaluated and selected resources of known scope and known evaluation criteria.

1

       

Providers with resources.

4