Author: Ronny Johnston, Maintenance Planner, Alabama Power Co.

Alabama Power Company (A Southern Company) in Parrish, Alabama, experienced a process interruption on their Unit 10 Electric Fire Pump apparently due to an out board bearing failure. When investigated further, they found that the pump had actually failed five times in a 6- month period and each time it was the outboard bearing that had failed. Before the pump was repaired and put back in service they put together a Root Cause Analysis (RCA) team to utilize the PROACT® Approach by Reliability Center, Inc. in Hopewell, VA.

The RCA Team was comprised of; Paul Cooner, the mechanic on the pump, Chris Curow, the Maintenance team leader, Harold Dobbins, CBM specialist and myself as the facilitator of the PROACT® process.

After a team review of how the PROACT® process worked, a logic tree was developed to graphically represent the cause and effect relationships that could have lead to the event occurring. All of the hypotheses that could be eliminated based on hard facts, were ruled out and some team assignments were made for those hypotheses that still required verification. During the team meetings it was brought out that the outboard bearing was the thrust bearing. IT was also explained to the other team members how they could identify a roller bearing that was a thrust bearing. Team members were assigned to retrieve the failed bearings and start the process of finding out why it failed. Team members were also assigned to locate an instruction manual and any other documentation possible.

It turned out that the old bearing was not a thrust bearing after all. The mechanics had been replacing the burned up bearings with the same type they had removed from the pump, which was the wrong bearing. Further investigation revealed that the CMMS (computerized maintenance management system) listed only one bearing for this fire pump and its description did not indicate it was the inboard bearing. The storeroom did have the correct bearing for the outboard but it was not listed for this pump.

The new bearing was installed and the pump was returned to service but was immediately removed because of high vibration. The team was called back together and continued the process to find the cause of this high vibration. A pump expert with Southern Company was on the plant site and agreed to meet with the RCA team. After the team listed the things that could cause this high vibration the pump expert did an excellent job of explaining the proper process for building this pump and things we should look for.

The CBM (condition-based maintenance) Team Leader joined the RCA team meeting and gave a good account of the type vibration found. The team identified shaft run out as a possibility because the shaft had been welded on when a new sleeve had been installed. The packing gland fit had not been as good as it should have been so it was added as a possibility. It was also decided that the impeller run out should be checked and it was noted that the impeller was not perfectly centered in the case. The coupling was thought to have been set up without the motor being at magnetic center. If this were to be accomplished a spacer would have to be installed. A team member identified this as the first thing to check. This was found to one of the root causes. A spacer was installed, the coupling was set up properly and the pump has run with excellent results ever since.

Root Cause Analysis Recommendations:

1. Have the description for the inboard bearing changed in the CMMS to read inboard bearing

2. Have the out board bearing set up in CMMS with an improved description.

3. Purchase or develop an accurate instruction manual for the fire pump.

4. Add a parts list to CMMS for the fire pump.

5. Add a procedure to CMMS with an accurate and user-friendly checklist for the fire pump.

6. Have some training for Mechanics and E&I on proper alignment and procedures for checking magnetic center.

Cost Overview and ROI of Root Cause Analysis

The cost of repairs was taken from the CMMS. These cost are not considered completely accurate and are considered conservative. The cost overview is as follows:

A. Work Order #6 on Failure #6, description: Broken shaft – cost $17,059

B. Work Order #5 on Failure #5, description: Rotating assembly repaired & returned to storeroom – Cost $2,365

C. Work Order #4 on Failure #4, description: outboard bearing failure- cost $577.39

D. Work order #3 on Failure #3, description: outboard bearing failure- cost $ 390.10

E. Work Order #2 on Failure #2, description: outboard bearing failure- cost $130.16

F. Work Order #1 on Failure #1, description: Pump locked up- cost $5,295

The grand total Parts and Labor is $ 25,816.65. If the rebuild cost is subtracted out from Work Order #6, which appears to be when the incorrect bearing was installed the first time, the cost will be $8,757.65. Not having the pump available has no cost associated with it in this case, but had it been needed for its intended purpose, it could have proven to be disastrous.

The cost of the RCA was about two hours for four people and about one hour for two others. If they each made $30 per hour it would only be $300 and the recommendations, if all are done, should be less than $500. This estimate shows the RCA costs to be about a tenth of the conservative cost of the repairs. Without the rebuild costs included this equates to around a 1000% ROI (return on investment). If the rebuild costs are added in as well, the ROI grows to around 3125%.

Click here to learn more about the PROACTOnDemand® RCA Management System.