Modular power protection in industrial applications
Modular power protection and conversion technology, particularly in the form of UPSs, has long been used in commercial applications but take-up in industrial applications has, to date, been relatively slow. This relatively slow uptake is due, in part, to a limited understanding of the “ilities” (“Availability”, “Reliability”, “Scalability”, “Flexibility” and “Maintainability”) commonly associated with modular technology and how the various “ilities” complement each other.
This final article within our 'ility' series focuses on the important subject of system maintainability and how the various UPS topologies influence servicing methodology, site operative training needs and spare parts holdings. We will also consider how system maintainability affects both system availability and total cost of ownership.
Different levels of maintenance
Companies purchase a UPS because they have a critical load that requires clean, continuous power; so, regardless of the UPS topology purchased, the UPS must always be in peak operational condition. However, because all UPS systems contain both electrical and mechanical components that have a statistical reliability, and consequently a statistical probability of unexpectedly failing at any time, it is essential that they receive routine preventative maintenance inspections.
Indeed, some components (e.g. capacitors, cooling fans, batteries) have a known finite useful working life and will definitely require proactive replacement if the UPS system’s availability and reliability are to be maximised.
In the “Availability v Reliability” article in this series (Power News 08/2017), maximising system availability was considered to be the primary objective of almost all UPS systems. To maximise system availability the reliability (MTBF) must be maximised and the repair time (MTTR) must be minimised; to achieve this every UPS system must be:
- Properly maintained to minimise the probability of an unexpected failure (i.e. maximise MTBF);
- Rapidly repaired when an unexpected failure occurs (i.e. minimise MTTR).
With the above in mind we can define a UPS system’s maintainability as how easy it is to properly maintain without risk to the critical load and how quickly the system can be returned to full operation following a system failure. There are generally three levels of maintenance. These are often referred to as first line, second line and third line, and for the benefit of this article we will define these types of maintenance as:
1 First line (“day-to-day” routine checks)
This level of maintenance is typically visual where operational conditions such as alarm conditions and cooling fan operation, etc., are noted and reported. First line maintenance can and should be routinely carried out by site operatives and, depending upon the site’s maintenance regime, first line maintenance will typically take place daily, weekly or monthly.
Some systems have their alarms connected to a building management system (BMS) or some other form of remote monitoring system to provide immediate notification of any alarms. Such remote monitoring enhances the effectiveness of first line maintenance.
First line maintenance tasks are the same regardless of the UPS topology in use, however, it is essential for site operatives to have access to expert technical support if a system problem is encountered.
2 Second line (“open-the-door” routine preventative maintenance)
This level of maintenance is proactive in nature and is intended to prevent problems occurring (i.e. to maximise MTBF). Regardless of the UPS topology, all routine preventative maintenance works must be performed by manufacturer-trained engineers using the latest revision of the maintenance procedures, service software and manuals. System firmware should also be upgraded as part of preventative maintenance.
Some routine preventative maintenance tasks will require system modification or component replacement. When this is required, modular UPS topology provides significant benefits as, regardless of the complexity of the modification or component replacement works, all works can be quickly and successfully completed on-site by module exchange.
Non-modular topologies may require the UPS to be offline for several hours whilst such modification/component replacement works are carried out, which exposes the critical load to raw mains.
3 Third line (“open-the-door” system fault investigation/repair)
Non-modular topologies require the site attendance of experienced manufacturer-trained engineers to undertake any fault investigation and repair activities. These engineers must use the latest manufacturer supplied service manuals and software, whilst having access to a complete set of manufacturer supplied spare parts.
The call-out time and the time needed to diagnose the problem, to carry out the repair and to return the system to full operation means that the system could be offline anywhere from many hours up to several days, depending upon the availability of the trained engineer and the nature of the fault.
Modular topology systems allow a much simpler third line fault investigation and repair. System faults can be corrected by simply replacing the faulty module. “Hot swap” modular systems in particular present third line maintenance benefits as module exchanges can be performed by trained site operatives with access to telephone support from manufacturer-trained engineers. The “hot swap” nature of the modules means that a faulty module can be safely removed from the UPS system and a new module fitted into the system within minutes. Furthermore, if the UPS system has been correctly rightsized with N+1 parallel redundancy (see Scalability article) then the system could be returned to full operation without the critical load having ever been exposed to raw mains.
Site operative training
It is often desirable, and in some cases essential, for a site to have one or more of its operatives trained by the manufacturer to undertake certain maintenance tasks on the system. If the site is remote and/or there are a large number of the manufacturer’s systems on a site then the default position may well be to have manufacturer-trained operatives permanently on-site.
However, training site operatives on specific UPS and power protection systems presents some problems:
- Manufacturer product training is expensive;
- Trained operatives can change jobs or sites with little notice;
- Trained operatives can be absent from site because of shift patterns, holidays or sickness;
- If an operative is not working regularly on a system then the training is soon forgotten;
- The system documentation, modification status and maintenance procedures will have been the latest available at the time of training but are unlikely to have been updated over time;
- Site operatives are typically “generalist” in nature (i.e. they are expected to have a reasonable knowledge of a large number of different systems);
- UPS/power protection systems require specialist knowledge.
There is a very significant difference in the training, skill and experience needed to repair systems using “hot swap” true modular topology and other topologies. “Hot swap” true modular topology systems can be quickly repaired by the exchange of a faulty module by a trained site operative with telephone support.
A non-modular topology system must be repaired by a highly skilled and trained operative who is capable of replacing individual PCBs and components.
Spare parts and module rotation
It is not uncommon for industrial sites to purchase and hold on-site a set of spare parts for their UPS and power protection systems. This is good practice as it ensures that spares are readily available should there ever be a system failure. However, this may not be suitable for all components or for all systems. For example:
- Electrolytic capacitors have a finite shelf life and after an extended period of storage (e.g. two years) they might need to be reformed, otherwise they will fail to operate and could cause further system damage when operating voltage is applied to them;
- Printed circuit boards and firmware held as spares might no longer comply with the latest modification status;
- There are no guarantees that spare parts that have been stored for a number of years will still work.
As an alternative to holding spare parts on-site, users of modular topology systems have the option of holding spare modules on-site. These “spare” modules can be put into service to add redundancy into a system or can be placed in spare parts stock and rotated into service as part of routine maintenance. Such routine spare module rotation ensures that all modules on-site will receive the same amount of duty, prevents component damage caused by the module sitting unused for several years, and all but guarantees that the spare modules are known to be working.
A potential downside to putting “spare” modules into service and supporting the critical load is that system “right-sizing” (see “Scalability” article) could be affected and the system might no longer be operating at an optimum point on the efficiency curve, which will impact total cost of ownership.
To overcome this potential problem some modular systems have inbuilt intelligence (see “Flexibility” article) that places any modules in excess of the number needed to protect the critical load into a “sleep mode”.
All UPS and power protection systems need to be properly maintained if the twin goals of highest availability and lowest total cost of ownership are to be achieved. It is false economy to “save money” by not having a power protection system properly maintained.
First line maintenance tasks, regardless of the UPS topology used, should be performed by site operatives and can be enhanced by using remote monitoring to provide early identification of actual and potential problems.
Routine (second line) proactive preventative maintenance maximises system reliability (MTBF) and, regardless of the UPS topology used, must be carried out by manufacturertrained operatives using the latest maintenance procedures, technical manuals and service software.
If “hot swap” true modular topology is used, emergency (third line) maintenance and repair tasks can be performed by trained site operatives with access to telephone support, if spare modules are available on-site. This minimises repair times (MTTR) and therefore maximises system availability.
If non-modular UPS topology is used, third line maintenance and repair tasks must be performed by highly skilled manufacturer-trained operatives.
Both site operative training and spare parts holding and storage are simplified if “hot swap” true modular topology is used.