Selection criteria for quality controlled information gateways
Work Package 3 of Telematics for Research project DESIRE (RE 1004)
Title page
Table of Contents

Previous - Next

Appendix III: Quality / selection definitions, models and methods in use


Quality can be applied to many products and processes. Traditionally it has been applied by management scientists to refer to product quality and, more recently, service quality. Although it can be difficult to adequately define "quality", current approaches to quality stress that processes are as important as tangible results, as is the case with TQM. Standards like ISO 9000 and BS5750 provide a framework for the implementation of quality management in an organisation, but do not otherwise address the issues of product or service quality. The information industry has recently turned its attention to the notion of product quality with regard to online databases and CD-ROM and progress has been made in co-operation with the library community and online user groups. Quality is usually defined in relation to a set of guidelines or criteria. The same broad approach is currently being applied to information provided over the Internet.

1. Introduction

Internet subject gateways have mostly defined "quality" with relation to carefully chosen lists of selection criteria. Quality has, however, been a subject of serious study in industry and management science since the Second World War. It might be useful to describe some of the concepts developed by management theorists and practitioners and to investigate their relevance to the quality selection issue. Finally, work being done on databases and information quality will be described.

2. Concepts of quality in industry and management science

Quality has been analysed as a factor in the management process since the 1930s, but it was not until after the Second World War that it became important. North American managers brought in to advise Japanese companies on restructuring after the war devised new concepts of quality which began to be accepted as being of universal application. The important pioneers in this field were W. Edwards Deming, Joseph M. Juran and Kaoru Ishikawa.

A universally agreed definition of quality still does not exist. Juran (1988, p. 2.8) suggested that quality should be seen as "fitness for use". Another short definition views quality as "conformance to requirements" rather than "goodness, or luxury, or shininess, or weight" (Crosby 1979, p. 17). These definitions from the management literature make it clear that quality cannot just be defined in relation to some abstract concept of "excellence", but should be seen in relation to the demands of the user of the final product. A recent working definition of "quality" has been provided by Clark, Money and Tynan (1990, cited in Clark 1992):

"How consistently the product or service delivered, meets or exceeds the customers' (external and internal) expectations and needs".

In the management context, quality processes can be applied to any product. A product can be defined as the "output of any process", consisting mainly of "goods, software, and services" (Juran 1988, p. 2.2).

2.1 Product quality

Product quality is usually defined with specific relation to the product, whether it is a good or service. For goods, important aspects might be reliability, durability, performance characteristics, aesthetics, etc. These dimensions will differ according to the product type: the most important factor being whether it meets the end requirements of the customer (Bergman and Klefsjö 1994, p. 19).

2.2 Service quality

Early quality models concentrated on goods. The enormous growth of the service sector in Western economies since the Second World War has resulted in a growing literature on service quality. Defining and modelling the quality of services is generally acknowledged to be more difficult than modelling the quality of goods due to the intangible nature of services themselves (Bergman & Klefsjö, 1994, pp. 266-267). There are two popular models of service quality in use.

1. Grönroos's service quality model

The model created by Grönroos (1984b) attempts to understand how the quality of a given service is perceived by customers. It divides the customer's perception of any particular service into two dimensions:

  1. Technical quality - What the consumer receives, the technical outcome of the process.
  2. Functional quality - How the consumer receives the technical outcome, what Grönroos calls the "expressive performance of a service" (Grönroos 1984b, p. 39).

Grönroos (1984b, p. 41) suggested that, in the context of services, functional quality is generally perceived to be more important than technical quality, assuming that the service is provided at a technically satisfactory level. He also points out that the functional quality dimension can be perceived in a very subjective manner (Fig. 1).

Fig. 1. Grönroos's Service Quality Model

Source: Grönroos (1984b, p. 40)

Grönroos's model is important because it reminds us that service quality must include the manner in which it is delivered.

2. The 'Gap' model

The 'Gap' model (Parasuraman et al. 1985; Zeithaml et al. 1990) is a means of describing customer dissatisfaction in the context of service quality. A team from Texas A&M University carried out some interviews with executives in U.S. firms and with consumers. A series of five 'gaps' regarding service quality were then identified:

"A set of key discrepancies or gaps exists regarding executive perceptions of service quality and the tasks associated with service delivery to consumers. These gaps can be major hurdles in attempting to deliver a service which consumers would perceive as being of high quality" (Parasuraman et al. 1985, p. 44).

Five gaps were identified:

  1. Between customers' expectation and management's perceptions of those expectations, i.e. not knowing what customers expect
  2. Between management's perceptions of customers' expectations and service quality specifications, i.e. the wrong service-quality standards.
  3. Between service quality specifications and service delivery, i.e. the service performance gap.
  4. Between service delivery and external communications to customers about service delivery, i.e. when promises do not match delivery.
  5. Between customers' expectation and perceived service (the total of the other four gaps).

It is this last 'gap' which has the most significance. The 'Gap' model keeps a clear focus on the perceptions of the customer, and these are seen as paramount

As part of this research, criteria for evaluating service quality were gathered. Ten key categories were identified which they called "Service Quality Determinants", and noted that despite the different types of service analysed, consumers used fairly similar criteria. The ten Service Quality Determinants listed by Zeithaml et al. (1990, pp. 21-22) were the following:

It is possible that these criteria could provide an initial framework for the development of quality criteria in other contexts

The ten determinants of service quality interact in the minds of customers with other factors, namely past experience, word of mouth and external communications to create a view of what service is expected. The diagram (Fig. 2) gives an indication of other factors which might impact on consumer expectations and thereby consumer perceptions of quality. Personal word of mouth communications are still important and still exist in a network environment. Electronic mailing-lists frequently get messages of the type "I've looked at this WWW site, and found it useful / not very useful / amusing; here is the URL". This factor in particular brings another level of subjectivity into the model, and leaves service quality definitions vulnerable to aspects of human behaviour, for example: the desire to emulate other people's choices associated with and exploited by the fashion industry. This could create potential inefficiencies (Anand et al. 1993).

The work on determinants led to the development of a scale for measuring customer perceptions of service quality called SERVQUAL (Parasuraman et al. 1988; Zeithaml et al. 1990, pp. 175-186; Parasuraman et al. 1991). This scale has been subject to criticism and refinement and there is a continuing debate about the measurement of service quality and the determinants which should be used (Mathews 1995).

Fig. 2. Parasuraman, et al.'s Determinants of Perceived Service Quality

Source: Parasuraman, et al. (1985, p. 48)

Companies are constantly encouraged to develop an improved emphasis on service quality. Schlesinger and Heskett (1991), for example, argue that organisations should abandon the industrial approach to services - the mass-production techniques used in supermarkets, fast-food restaurants and airports - and adopt a "new model" of service based around customers' requirements. Additionally, the service quality debate is connected with the debates on "excellence" initiated by the management guru Tom Peters (Peters and Waterman 1982) and other concepts like market orientation (Caruana and Pitt 1994; Caruana et al. 1994).

2.3 Total Quality Management

The development of the quality concept in industry has created a requirement for an organisational structure which can include quality concepts at every stage in the planning and delivery of a product or service. The process is called Total Quality Management (TQM). The essence of a TQM strategy is described by Bergman and Klefsjö (1994, p. 22).

The important insight is that quality becomes a continuous process. This is especially important for service industries, where customer perceptions of quality are constantly changing. Quality becomes a process of continuous feedback and improvement. This process is known as a "quality system". Bullivant (1994, p. 14) defines TQM as "a commitment to a company-wide culture where everyone is clear of the direction and objectives of the organisation and work in support of each other to achieve these goals". Rowley (1996), however, notes that while there is general agreement about the theoretical aspect of TQM, the practical aspects of implementation are more problematic.

2.4 International standards

National and international standards authorities have turned their attention to quality in recent years. In the UK the British Standards Institution (BSI) first published BS 5750 in 1979 for quality systems and the standard was further developed by the International Organisation for Standardisation (ISO) into the ISO 9000 series.

A useful definition of quality is found in ISO 8402: "quality is the totality of features and characteristics of a product or service that bears on its ability to satisfy stated or implied needs" (ISO 8402: 1986). This has since been refined to "the totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs" (ISO 8402: 1994; EN ISO 8402; BS EN ISO 8402: 1995).

The ISO 9000 approach has been applied in many different organisations and obtaining certification can be essential in some industries (Rothery 1993; Harvard 1994). Although services can be included in the certification process (ISO 9004-2: 1991; EN 29004-2: 1993; BS 5750: Part 8: 1991), ISO 9000 is seen primarily as a product-orientated rather than process-orientated model.

2.5 Benchmarking

Benchmarking is another recent approach to ensuring improvements in quality. Bullivant (1994, p. 1) defines benchmarking as "the continuous process of measuring products, services and practices against leaders, allowing the identification of best practices which will lead to sustained and superior performance". Benchmarking can be carried out with relation to different types of organisation (Bendell et al. 1993, pp. 69-70):

The purpose of the benchmarking process is not just to understand the processes carried out by other organisations, but to enable a considered self-assessment to be made of your own organisation.

3. Database quality

It is only comparatively recently that the library and information community have taken an interest in quality assessment and analysis (Morrison 1994). In the UK a guide to implementing BS 5750 and ISO 9000 in libraries was published in 1993 (Norton and Ellis 1993) and there has been a increasing interest in quality issues in both Germany and the Nordic countries (IMO 1995, Johannsen 1995). This interest is largely concerned with issues of quality management (Brophy, Coulling and Melling 1993; Kinnell 1995; Pilling 1996; Seay, Seaman and Cohen 1996; White and Abels 1995) or TQM (Brockman 1992; Martin 1993; Rowley 1996) and has led to the development of a set of IFLA guidelines for performance measurement in academic libraries (Boekhorst 1995) and a draft international standard on library performance indicators (Carbone 1995). In addition to this emphasis, however, there has been an additional focus on defining information quality both with regard to bibliographic records and with electronic databases. It was the advent of shared computerised cataloguing that made the quality of bibliographic records an issue and studies of this have been carried out both in the UK and USA (Chapman 1994; Thomas1996).

Information quality was not a serious concern for libraries before the advent of electronic databases. Libraries would select books and journals according to their own criteria, which would usually have some reference to users' needs or the requirements of the host organisation. This general approach was later applied to other information formats when they came available, (microforms, video cassettes, LP records, CD-ROMs, etc.), within the financial constraints of the organisation. Information quality itself began to be a serious concern with the increasing use of electronic databases, both online and on CD-ROM.

Early online databases were regarded with respect by their librarian users. Basch (1992, p. 85) notes that they were regarded as "nothing short of miraculous" as research tools. For this reason, it was only by the late 1980s that database users began to make suggestions as to how they could be improved. The 1989 Annual Retreat of the Southern California Online Users Group (SCOUG) developed a "user wish list" (Basch 1990a), and the following year, a rating list for database quality (Basch 1990c). The rating list was arranged in a set of ten categories:

  1. Consistency
  2. Coverage and Scope
  3. Timeliness
  4. Error rate/Accuracy
  5. Ease of use
  6. Integration
  7. Output
  8. Documentation
  9. Customer Support and Training
  10. Value to cost ratio

4. The Internet and information quality

SCOUG dedicated their 1995 Annual Retreat to the subject of quality on the Internet (SCOUG 1995). The participants noted that the Internet was quite different to the database industry as most information providers were not motivated primarily by financial considerations. Information providers therefore had little or no incentive to improve the quality of their "product". SCOUG also noted that whilst certain technical standards were in place, primarily HTML and the work being carried out by the WWW Consortium, the Internet Engineering Task Force (IETF) and commercial organisations like Netscape, there were no content standards. They considered that people might be prepared to pay for a service which gave access to "charted, safe, quality areas" of the Internet (Ebbinghouse 1995b). They identified quality issues under the following general headings (Ebbinghouse 1995a):

  1. Credibility
  2. Authority
  3. Indexing
  4. Registration
  5. Reviews / Ratings
  6. Technical
  7. Security and privacy
  8. Feedback / Maintenance / Customer service
  9. Time-size-format pre-alert warnings
  10. Help files
  11. Copyright / Intellectual property
  12. Finding tools
  13. Downloading reliable, transparent and standardised
  14. Online billing
  15. Reliable browsable directory of addresses of sites
  16. Censorship issues, self-censorship
  17. Search engines
  18. Advertising
  19. Payment mechanisms
  20. Able to track usage of your site
  21. Maintenance

Because of the nature of the meeting, the quality issues raised by SCOUG do not all refer specifically to information quality but to the Internet generally.

An Information Market Observatory report on The quality of electronic information products and services was published in September 1995 (IMO 1995). It concentrated on quality issues raised by commercial databases, but did make mention of the Internet. The report identified the main problems as:

  1. Too much information - often redundant and inaccurate,
  2. The lack of centralised control - no editorial function or refereeing.

It also made mention of the World Wide Web (WWW).

"…there is much duplication between sites. Sites and resources can appear, move or disappear very quickly. Web sites contain information that ranges from the highly significant through to the trivial and obscene, and because there are no quality controls or any guide to quality, it is difficult for searchers to take information retrieved from the Internet at face value" (IMO 1995).

It concluded that the Internet "will not become a serious tool for professional searchers until the quality issues are resolved".

Ciolek (1996a, p. 107) has argued that if the WWW is to continue to be of use to scholars and the research community quality issues will have to be confronted:

"Our greatest folly seems to be our willingness to cultivate this global communication system, open to all and sundry, without first ensuring that we have enough useful and trustworthy, accurate and timely information to be circulated across such a networked behemoth".

Ciolek is the editor of the Information Quality WWW Virtual Library (Ciolek, ed. 1996b) based at the Coombs Computing Unit of the Australian National University in Canberra. This WWW site gives access to information on quality issues and the Internet, and the unit administers a electronic mailing list called Information-Quality-L which acts as a centre for the world-wide exchange of ideas on Internet quality.

5. Conclusions

Quality is a diverse concept and has been applied to many things. In management science it is applied to products, services and the process of management itself in TQM. With information, we are mostly concerned with product quality - ensuring that a product, whether it be a journal, electronic database or WWW page, fulfil an agreed set of criteria. Additionally, however, information providers might have some interest in service quality with its emphasis on the expectations and requirements of the customer or user and their fulfilment.

Next Table of Contents

Page maintained by: UKOLN Metadata Group
Last update: 25-Mar-1998