Authors: Jeffrey S. Brown, Process Engineer at Eli Lilly,  and Ken Storey, Freelance Writer for Reliability Center, Inc.

Every major manufacturing company has had what could be labeled a near miss or serious incident without injury. It was one such incident at an Eli Lilly pharmaceutical plant in Clinton, Indiana that caused the investigating engineer to use a new software program to try and determine the root cause of the incident and avoid a future repeat.

A materials-basket that is used in a four-story tank was being moved to a new location when the cable holding the basket suddenly snapped. The basked fell and edged against the side of the tank. Fortunately, nothing was in the basket at the time and no injury occurred. It would be easy to simply replace the broken cable, chalk it up to cable fatigue, and keep on working, but that is exactly where many companies make a major mistake.

In an effort to “get going again”, many firms overlook the long-term negatives of not taking the time and energy to get to the root cause of a failure to prevent a future failure from occurring. Do you simply fix the “broken part” or do you find the root cause of the problem and avoid future failure repeats?

The path Eli Lilly and the investigator Jeffrey S. Brown took involved the latter route. Root Cause Failure Analysis (RCFA) methodology and a relatively new software program PROACT^® were used to get to the incident’s root cause. As Brown says, “If you are serious about avoiding breakdowns, failures, and potential accidents, you have to use a systematic approach to find the root cause of the problem if you really want to avoid having it happen again”.

In the materials-basket, cable-breaking example that Brown investigated, the operators had not seen anything unusual, just the basket falling with the cable failure. However, using the RCFA methodology involved in the software and with support from the vendor, Reliability Center, Inc., he and his team were able to determine that failed safety switches had actually been a major cause to the cable failure. The cable’s block pulley system had come into contact with the cable drum that resulted in the cable snapping. The assumed cause by most people of the cable fatigue from use was not even a factor as tests showed the cable’s strength still met specifications.

The process of root cause failure analysis involves preserving the data, analyzing the data collected, using hypothesis testing for possible causes, and then tracking results for a final conclusion. The data collected is organized around five P’s™: parts, position, paper, people, and paradigms.

There is a lot of information to be collected and analyzed for the process to work properly so a team was organized with each member assigned a specific task connected to the five P’s™.

Each of the five P’s™ needs to be approached in a thorough manner. In the materials-basket case, the Parts aspect involved collecting the broken parts as well as the parts that did not appear affected, but were connected to the incident. Being able to preserve all the parts involved in the incident before they are discarded or replaced is a key element of the root cause process.

The photos aspect of the process involves taking pictures of the exact Position the equipment was in at the time of the incident. The difference between what was the root cause can be a matter of small distances and recreating the situation for photography purposes helps to accurately access the situation at the time of failure. It’s worth noting that one of the first things the National Transportation Safety Board did in its recent investigation of the Amtrak multi-fatality collision with a truck was to recreate the position of the truck and the train when they reportedly first saw each other, and then take photos of their positions.

The Paper aspect of the methodology as applied to the case involved getting the manual for the hoist, a recent inspection report, and a Job Safety Analysis (JSA) that had been done on the materials-basket and hoist.

The People part of this RCFA methodology for the case involved interviewing the operators and others at the scene of the incident. It also involved interviewing the engineer and engineering technician who had recently checked the safety switches before using the materials-basket.

The Paradigms aspect in our case involved an emergency switch on the cable that would activate to reverse the direction on the pulley block to prevent it from coming into contact with the cable drum. This paradigm also involved the thought process of “the safety switches had never failed before – so they must not have failed now”.

When Brown’s team had collected the information to satisfy the five P’s™ of the RCFA/PROACT^® process, the information was systematically fed into a computer using the software as a guide. All of this information is preserved and used to help develop various failure hypotheses. The hypothesis we tested was that somehow the pulley block had come into contact with the cable drum. Looking at the top of the pulley block, indentation marks of the cable could be found. These are the types of “discoveries” that are easily overlooked without following a RCFA methodology.

A further investigation of our hypothesis showed that the cable hoist motor would go into what is called a locked-rotor-condition and increase torque if it encountered an immovable load such as the block contacting the drum. This increased torque was enough to snap the cable.

An analysis of the safety switches determined they had activated, but not in time to stop the cable failure. As a result of getting to the root cause of this failure the clearance between the hoist’s cable-drum and the pulley block has been increased to provide a greater “safety area” of cable once the safety switches activate. In addition, the Standard Operating Procedure for the materials-basket also has been modified to hopefully prevent a similar incident in the future.

The bottom line to the RCFA methodology and the accompanying software that made managing the process easier is that it solves problems instead of fixing a broken part. After all, if you don’t get to the root cause, you’re just setting yourself up for failure again.

Watch the video to learn more about the PROACT^® Root Cause Analysis Mobile Management System.

About the Authors

Jeffrey S. Brown, PE is a Process Engineer at the Eli Lilly pharmaceutical plant in Clinton, Indiana. Ken Storey is a freelance writer in Richmond, Virginia.