Ebook: Incident Investigation and Root Cause Analysis

Overview

The thorough investigation and analysis of incidents (both actual events and near misses), along with the appropriate follow-up, provides one of the most effective means of improving the safety and reliability of process facilities. Other risk management programs, such as hazards analysis and management of change, are directed toward anticipating problems so that corrective actions can be taken before an event occurs. Yet, in spite of their undoubted value, these predictive techniques have the following limitations: 

1. The analyses are, of necessity, theoretical and speculative; there can be no assurance that all plausible events have actually been identified. Indeed, it is more than likely that important failure mechanisms will be overlooked.
2. It is difficult to predict the true level of risk associated with each identified event because estimated values of both consequence and likelihood are usually very approximate. In particular, predictions as to what might happen are invariably colored by the personal experiences of the persons carrying out the analysis.
3. Most serious events have multiple causes, some of which appear to be totally implausible or even weird ahead of time (which is why serious accidents so often seem to come out of the blue). Even the best qualified hazards analysis team will have trouble identifying such multiple-contingency events.
4. It is very difficult to predict and quantify human error - yet most events involve such error.

Actual incidents, on the other hand, provide hard information as to how things can go wrong, thus helping to cut through wishful thinking, prejudice, ignorance and misunderstandings. The root cause analysis that follows an incident investigation will help identify weaknesses and limitations in a facility's management system, thereby reducing the chance of recurrence of similar incidents.

Another reason for emphasizing the importance of incident investigation in the process industries is that process safety management (PSM) systems - of which Incident Investigation and Analysis constitutes one element - have been in place in many cases for more than fifteen years. Many of these facilities have made good progress in meeting regulatory requirements. However, the fact that such systems can "survive an audit" and are working well on paper does not mean that they are as effective at actually improving safety as they might be. Incident investigations help identify how the elements of PSM really are functioning, and can provide management with insights as to how the systems can be improved.

Incident Investigation and Analysis Philosophy

Publications in the field of incident investigation and analysis often promote a particular methodology with the implicit claim that their approach is better than the methods promulgated by other organizations. Such publications are often commercial in their approach, thus tending to create a concern in the mind of the reader as to the objectivity of the materials that are presented.

This ebook does not advocate or promote any particular methodology. Indeed, it is suggested here that an effective incident investigation and analysis requires much more than the mere application of a particular investigation technique. Equally important - maybe more so - is the ability on the part of the investigators to inculcate an atmosphere of trust and confidence with everyone with whom they work - not only those involved in the incident itself, but also the managers who will be charged with taking appropriate corrective actions. Each analytical technique has its strengths and weaknesses - an effective investigation will use a judicious mix of approaches as circumstances dictate.

Therefore, rather than stress the use of just one particular analytical method this ebook suggests that a successful investigation should be conducted through use of the six strategies and techniques listed below and also shown in Figure 1.

1. Establish trust, and thereby encourage candid discourse from those involved in the event and also from the managers responsible for follow-up.
2. Listen to what people actually say, base all findings on verifiable facts, and be thorough in all phases of the investigation.
3. Establish a clear cause and effect chart - backed up with solid evidence - integrated into a timeline.
4. Use technical experts to assist with understanding specialized issues.
5. Develop an understanding of root causes and systemic issues at different management levels.
6. Manage an incident investigation and analysis as a project with its own schedule, budget and deliverables.

Figure 1
Elements of Successful Incident Investigation and Analysis

Contents

Chapter 1 - Incident Investigation and Analysis
Introduction
Incident Investigation and Analysis Philosophy
   Trust and Candor
   Listen to the Facts
   Cause and Effect
   Technical Expertise
   Root Cause Analysis
   Project Management
Communications
Definitions
   Incident
   Accident
   Near Miss / Hit
   Potential Incident
   High Potential Incident
   Root Cause
   Safe Limits
Standard Example
Six-Step Investigation and Analysis Process
   Step 1 - Initial Investigation
   Step 2 - Evaluation and Team Formation
   Step 3 - Information Gathering
   Step 4 - Timeline Development
   Step 5 - Root Cause Analysis
   Step 6 - Report and Recommendations
Building Trust
   Blame and Fault-Finding
   Management Trust
   Early Reporting of Bad News
Regulatory Investigations
   Incident Investigation Standard
   OSHA PSM Guidance
   Analysis of the OSHA PSM Regulation and Guidance
      (1) Investigation
      (2) Timing
      (3) Team
      (4) Report
      (5) Follow Up
      (6) Participation
Industry Issues
   Shortage of Qualified Workers
   Aging Infrastructure
   Training and Education
   Contractor Management
   Alarm Flooding
   Management Pressure
   No Good Deed Goes Unpunished
Conclusions

Chapter 2 - Initial Investigation
Introduction
The Go Team
   Immediate Actions
   Team Preparation
   Drug and Alcohol Testing
Incident Report Form
   Incident Number
   Title
   Location, Date and Time of Event
   Duration of Event
   Date and Time of Report
   How Observed
   Person(s) Reporting
   Preliminary Ranking
   Incident Type
   Incident Flags
   First Description of Event
   Immediate Corrective Actions Taken
   Witnesses
   Contractor Involvement
   Detailed Location
   Consequences
   Emergency Response
   Security Issues
   System Alert
   Incident Owner / Department
   Notes and Attachments
First Management Report
Conclusions

Chapter 3 - Evaluation and Team Formation
Introduction
Evaluation
The Incident Pyramid
Consequence Ranking
   Worker Safety
   Public Safety and Health
   Environmental Impact
   Economic Loss
   Depth of Analysis
Team Formation
   Outside Investigators
   Corporate Support
Team Members
   Sponsor
   Incident Owner
   Facility Manager
   Lead Investigator
   Administrator
   Area Supervisor
   HSE Representative
   Process Safety Management Coordinator
   Union Representative
   Process / Facilities Engineer
   Maintenance Technicians
   Subject Matter Experts
   Contractors / Vendors
   Emergency Response Specialists
   Attorneys
Charter / Terms of Reference
Team Member Qualifications
   Objectivity
      Common Sense
      Jumping to Conclusions
   Haughtiness and Empathy
   Understand Incident Investigation Methodology
   You Do Know What You Don't Know
   Understand Process Technology
   Logical Thinking
   Painstaking
   Understand Process Safety Management
Conclusions

Chapter 4 - Information Gathering
Introduction
Interviews
   Interview Guidelines
      Regulatory / Legal Interviews
   Witness Interviews
   Rapport and Trust
   Interviewer Attributes
      Interpersonal Skills
      Technical Skills
      Critical Factors Recognition
      Objective
      Effective Note Taking
      Management Interviews
Documentation
   Engineering Information
      Process Flow Diagrams
      P&IDs
      MSDS
      Layout Drawings
      One Line Diagrams
   Operating Information
      Log Books, Maintenance Records and JSAs
      Hazards Analysis Reports
      Management of Change Records
      Operating Manuals / Procedures
      Incident Investigations and Audits
   Vendor Data
Field Information
   Damage Assessment
   Photographs and DVDs
   Closed Circuit Television
Instrument Records
Testing / Lab Analysis
Conclusions

Chapter 5 - Timeline Development
Introduction
Timeline Steps
   Section 1 - Events Prior to the Incident
   Section 2 - The Incident
   Section 3 - Post-Incident Response
Timeline Construction
   Conditions
   Multiple Timelines
Timeline Table
Background Information
Conclusions

Chapter 6 - Root Cause Analysis
Introduction
Levels of Root Cause
   Single Incidents
   Multiple Incidents
Types of Root Cause Analysis
Argument by Analogy: Story Telling
   False Extrapolation
   Linearity
   World Views
Barrier Analysis
   Safeguards
   Safeguard Level 1: Normal Operations
   Safeguard Level 2: Procedural Safeguards
   Safeguard Level 3: Safety Instrumented Systems
   Safeguard Level 4: Mechanical Safeguards
      Check Valves
      Pressure Safety Relief Valves
   Safeguard Level 5: Passive Safeguards
   Safeguard Level 6: Emergency Response
Categorization
   Equipment Failure
   Human Error
      Mistakes
      Slips
      Fixation
      Violations
      Error in an Emergency
   Human Error as a Root Cause
Process Safety Management
System Analysis
Why Trees
   Single Chain of Events
   Wrong Chain
Fault Tree Analysis
   Incident Fault Tree
   Fault Tree Development
   Addition of Safeguards
Linkage of Fault Trees to the Timeline
Common Cause Events
Incident Data Bases
   National Response Center (NRC)
   Accidental Release Information Program (ARIP) Database
   CFOI (Census of Fatal Occupational Injuries)
   Major Accident Reporting System (MARS)
   Marsh & McLennan Reviews
   Annual Loss Prevention Symposia
   Process Safety Beacon
   Government Agencies
Conclusions

Chapter 7 - Report and Recommendations
Introduction
Levels of Recommendation
   Short Term Recommendations
   Intermediate Recommendations
   Long Term Recommendations
   Industry Guidance
Report Structure
   Executive Summary
      What Happened?
      What Could Have Happened?
      What Was the Cause?
      What Actions Should Be Taken?
      Recognition
   Terms of Reference
   Reason for Selection
   Sequence of Events
   Consequences
   Root Causes
   Other Hazards
   Recommendations
   Attachments
      Attachment A - Regulations and Standards
      Attachment B - Root Cause Analysis
      Attachment C - Organization Chart
      Attachment D - Review of Similar Events
      Attachment E - Investigation Team
      Attachment F - Review of Modern Designs
      Attachment G - Index to Pictures and Documents
      Attachment H - Detailed Timeline
Issuing the Report
   Writing the Report
   Presenting the Report
   Follow Up and Recommendations Tracking
Conclusions

Chapter 8 - Project Management
Introduction
Phase / Gate Method
Organization
   File Systems
   Incident Register
   Information Security and Chain of Custody
   Record Retention
   Removing Evidence
Contingency Plans
   Inadequate Planning
   Resources Availability
   Vaguely Defined Scope
   Scope Changes
Legal Issues
   Security of Information
   Attorney/Client Privilege
   Injudicious Written Statements
   Personal Information
   Anecdotes
Feedback
Conclusions