Quarter 4, 2002:  Volume 3, Issue 3

Reliability Edge Home

Les Warrington
Senior Fellow (Quality and Reliability)
University of Warwick (United Kingdom)

The University of Warwick (United Kingdom) is currently conducting a detailed and far-reaching survey of reliability practices, the "Benchmark Survey of Timely and Cost- Effective Application of Reliability Techniques." Companies world-wide and across all industries are called upon to participate. The survey has been designed to investigate the contextual factors that lead a project to choose specific reliability techniques. It attempts to benchmark reliability activity according to the speed that reliability techniques deliver useful output, the quantity and cost-effectiveness of that output, its authority and accuracy and the uses to which the output contributes. 

This article presents a discussion of some of the issues that were considered during the development of the University of Warwick’s survey and a general overview of the final structure and implementation strategy. Readers may wish to incorporate this information into a framework for examining their own reliability practices and are also encouraged to participate in the survey. 

Background 
Product reliability, be it expressed in terms such as durability, endurance or absence of warranty claims and in-service faults, is not achieved by conduct of a deterministic set of techniques. Indeed, over recent years, prescriptive standards have been viewed by many as being obsolete. Therefore, companies are much more required to use judgment and experience to choose the likely, most cost-effective techniques to be their reliability programme plan. 

A reliability programme plan is influenced by many factors. Simple benchmark comparison of reliability techniques is unlikely to yield meaningful results because projects are so diverse. Of course, benchmark between comparable industries, companies and projects may be possible where there are sufficient similarities, but a universal benchmark requires an understanding of internal and external influences as well as measuring the performance achieved. 

Benchmark Model 
The model chosen for this benchmark survey focuses on benefits achievable from reliability effort, namely to: 

• Assess risk or provide assurance.
  
• Contribute to design.
  
• Provide improved project planning.
  
• Facilitate sustainable process improvement.

Each reliability technique derives knowledge and applies it to current and future projects (as demonstrated in the benchmark model diagram displayed in Figure 1). The cost-effectiveness of any particular benefit is measured in terms of: 

• Timescale.
  
• Resources, skill and cost requirements.
  
• Magnitude, accuracy and validity of contribution.
  

Figure 1: Benchmark Model [Click to Enlarge]

These are the primary benchmark metrics. This model has been validated through workshop critique and individual review by several UK companies. 

Benchmark Development 
The survey aims to do more than measure benchmark performance. Through the process of workshop and individual company discussions, a number of additional objectives have been identified. These objectives come from the perspective of wishing to introduce greater authority to the process of selecting and scoping an optimally cost-effective reliability project plan. Such activity is usually undertaken early in a project and has a major impact on costs and risk. Accordingly, the survey additionally aims to understand how the following factors influence the benchmark metrics: 

Corporate influences

• Culture and commitment
  
• R&D finance
  
• Project finance
  
• Organisation (reliability as a separate or organic function) 

Influences deriving from written requirements 

• Requirements stability (frequency and scope of change)
  
• Contractual requirements
  
• Justification required 

Customer influences 

• Type of customer
  
• Data requirements 

Project related factors

• Industry and technology
  
• Knowledge-base (experience and learning curve)
  
• Complexity
  
• Lead times

Benchmark Layout
In order to incorporate suitable survey questions covering the primary benchmark metrics and all associated influencing factors, it was immediately apparent that careful layout design would be necessary. The layout should: 

• Be easy to understand.
  
• Minimise textual answers.
  
• Avoid ambiguity.
  
• Avoid repetition.

Accordingly, the benchmark survey is split into four sections, grouping common data and reducing repetition: 

• Part 1: Corporate information
  
• Part 2: Project information
  
• Part 3: Project phase information
  
• Part 4: Reliability technique information 

Each part maximises the use of check boxes and all questions have help reference to both clarify and avoid ambiguity. The linkage between sections is illustrated by Figure 2.

Figure 2: Benchmark Survey Structure [Click to Enlarge]

The result is a powerful survey requiring minimal time to complete. Provided that appropriate fact-holders are available, the survey may be completed accurately (with two reliability techniques) in less than two hours. All completed surveys will be treated confidentially. 

Analysis and Distribution of Results
The statistical analysis objectives to be achieved through the survey include: 

• Identify the strength of influencing factors.
  
• Benchmark each reliability technique.
  
• Benchmark each company in relation to comparable returns, using the influencing factors to extend the benchmark. 

In addition, however, many of the influencing factors are controllable, and hence specific guidance will be available to respondents, highlighting those factors currently limiting the impact of reliability techniques and their benchmark performance. 

All respondents will receive aggregate and analysed results, together with a report highlighting the manner in which corporate, customer, project and requirements factors hinder or enhance the cost-effectiveness of reliability techniques. 

Timescale
This survey is open-ended. However, an initial "critical mass" of returns is hoped to be received by mid-2003. Returns will continue to be encouraged and will lead to analysis updates to all participants. 

Publication
Summarised results will be published in journals and on the Web. However, only participants will receive aggregate results that will allow them to conduct more detailed analysis pertinent to their own projects and planning processes.