Disaster planning for DC operations

Your company may be prepared for fires, floods, and labor strikes. But does it have a plan for dealing with material handling system breakdowns?

By David Maloney

Although you wouldn't know it from the headlines, the biggest threats to global supply chains aren't necessarily the most sensational: pirates, for instance, or hurricanes or terrorist attacks. Far more serious are the threats posed by mundane occurrences like material handling system breakdowns or DC power outages. For one thing, they have a higher likelihood of occurrence. For another, they have the potential to bring order fulfillment operations to a halt, with potentially devastating effects on customer service.

To minimize these types of risks, smart facility managers make it a point to develop detailed contingency plans for dealing with disruption. But how do you go about that task? DC Velocity asked several leading facility designers and systems integrators for advice on making operations more resilient and creating a bulletproof business recovery plan.

Resiliency, not redundancy
The focus of a contingency plan should be resiliency, not necessarily redundancy, according to the experts. The main point of the exercise is to ensure that the facility can weather a supply chain storm. While that may include duplication of some processes and equipment, it can also be as simple as outlining alternative methods to keep product flowing.

Decisions regarding how much and what kind of redundant equipment is needed will be dictated by cost, the likelihood of disruption, and the company's tolerance for risk. What can a facility do without? How long can its supply chain afford an outage? Is the product being stored there going to spoil or deteriorate over time (like perishable food and certain pharmaceuticals)?

"One thing I ask my clients is, 'What is the cost of downtime to you?' It can be hundreds of thousands of dollars in labor costs, missed shipments, and lost customer service," says Bob Carver, vice president of Vargo Adaptive Software.

"Do you want to stay in business as if nothing happened?" asks Lou Cerny, vice president at Sedlak Management Consultants. If that's the case, the best course may be to invest in extensive backup systems. But that can be costly, he says. After weighing the expense, companies sometimes decide to back up only those functions that are critical to their supply chains, he adds.

Ready for the worst
One of the functions companies commonly target for backup is the facility's electrical system. Power outages are the number one cause of downtime in distribution centers, especially during summer months when the threat of thunderstorms is highest. Many facilities counter this with on-site power generators. Some choose to power the entire facility, while others restore power only to critical areas, including freezers and coolers, or to select equipment, like lighting, conveyors, and sorters.

Like electrical systems, software systems are likely to be high on the list of systems to be restored quickly following a shutdown. Ideally, software systems should be designed to assure near-continuous operation. When systems do go down, the design should allow for a swift recovery.

One way to boost reliability within a facility's control system is to minimize the number of control points. "Instead of 10 PLC [programmable logic controller] systems, we centralize the control," says Jacques Hasbani, vice president of software development for Fortna. He explains that most of the facilities his firm designs have a cold backup control box that is plug-and-play ready, so if there is a failure with a centralized control box, it can be easily swapped out.

Hasbani also suggests a clustered design for the servers that operate the application software and associated databases. Using multiple servers to run the system allows for higher processing speeds and provides needed redundancy in case of drive failure, he says. "We can provide an uptime of 99.82 percent with clustering," Hasbani adds.

Even with clustering, facilities should have a backup of their software and data somewhere— whether it's on site on another server, stored at a service provider's location, or at a third-party data storage facility.

Lead your league in rebounding
One of the most important parts of the contingency planning process is outlining the procedures to follow in case of equipment failure. For companies that don't have backup systems and equipment in place, that will likely mean devising temporary workarounds. For instance, if an RF system or other type of automated picking system were to break down, the workaround might be to use the facility's warehouse management system to print out paper pick lists until the systems can be restored.

Some types of automated equipment come with resiliency features built in. For example, many storage and retrieval systems are designed with retrieval machines that can change aisles so that if one machine is down for repairs, another can still gather products from the affected aisle.

Another way to boost resiliency is to store products of the same SKU in several locations throughout the building so that orders can still be filled if the primary storage location cannot be reached. SKUs stored in carousels, for instance, should be placed in more than one pod so that if one unit is out of service, the product can still be picked from another one.

For the same reason, pick modules should be configured so that if conveyor in one section is out of commission, orders can be diverted to other zones. (Again, this requires that SKUs be slotted to more than one area of the module.)

Some experts recommend keeping a few wheeled carts on hand to gather products in the event of a conveyor shutdown. Other low-cost backup options include non-powered conveyors. "One sortation system we put in had some non-powered, flexible conveyor nearby that could be wheeled into place to sort packages in an emergency," notes Marvin Logan, systems consultant with Bastian Material Handling. "That was a very inexpensive solution."

In designing pack stations, the easiest way to create resiliency is to simply plan more lanes than are currently needed. Most facilities will eventually grow into them.

Don't forget maintenance
What companies don't always realize is that a big part of disaster planning is preventing disruptions in the first place. That's where maintenance comes in. Establishing and following a well-designed preventative maintenance and spare parts program is one of the best ways to reduce the risk of equipment breakdowns and the associated disruptions, according to the experts. "We stress that anything mechanical is going to eventually break down," explains Cerny of Sedlak. "Good preventative maintenance solves a lot of issues proactively."

"While you look for reliability [when choosing equipment], you still need to repair equipment when it breaks. You need to maintain a critical parts inventory and an on-site repair crew or have contracts with local companies to perform maintenance quickly," says Bob Frye, senior account executive with Peach State Integrated Technologies.

"Break things out into critical spare parts and not-so-critical parts," adds Carlos Ysasi, vice president at Vargo. "If you can get it within 24 hours, then you might not want to keep it on the shelf. But if it is something like a cross-belt from Italy that you cannot get quickly, you might want a backup part on hand and people trained to switch it out."

Many integration specialists and equipment suppliers offer remote monitoring for the systems they install. This helps head off problems, or at the very least, alerts them immediately when something does go wrong.

Finally, test the plan, advises Justin O'Toole, vice president of supply chain for Fortna. "See if you have the right equipment and flexible processes within your design," he says.

More articles by David Maloney

For more DC Velocity, become a fan on Facebook and follow us on Twitter.

Feedback: What did you think of this article? We'd like to hear from you. DC VELOCITY is committed to accuracy and clarity in the delivery of important and useful logistics and supply chain news and information. If you find anything in DC VELOCITY you feel is inaccurate or warrants further explanation, please ?Subject=Feedback - : Disaster planning for DC operations">contact Editorial Director Peter Bradley. All comments are eligible for publication in the letters section of DC VELOCITY magazine. Please include you name and the name of the company or organization your work for.