TGS resilience in action: Our Miami Data Center withstands hurricane Milton

Madrid, 16 October, 2024 -Milton made landfall on Wednesday night, October 9, off the central Florida coast as a Force 3 hurricane, with winds reaching 208 km/h (120 mph). Due to its large size, our Data Center received the Tropical Storm winds with a lot of rain, leaving more than 10 inches of water and numerous tornadoes, which despite their uniqueness, since they are not usual in Florida, left extensive damage.

In the words of our Service Manager in Miami, Mario Oranges:

“We managed to maintain uninterrupted service for all our customers, without incident or downtime, from our own mission-critical equipment operations to our internal network teams, to our customer management.”

But how did they do it?

First of all, thanks to our team, 100% TGS, professionals who stood guard during the whole night that Milton took to cross the Miami area, and who solved several emergencies derived from the unsafe electrical network, since there were numerous blackouts during those hours.

Secondly, the excellent condition of the equipment and facilities. The constant quality inspections, the updates to the millimeter, the care for each of the innovations and protection measures implemented, are rewarded at times like these.

But let’s go into detail, because it’s worth reflecting on the great work that has gone into the planning and the disposition of our team.

LOCKDOWN PLAN

Miami, and Florida in general, is at the epicentre of the Caribbean hurricane season. It receives so many extreme weather conditions that any infrastructure built there must be designed to withstand any possible disaster.

Because it could not be otherwise, our Miami Data Center has an ad hoc contingency plan called the Lockdown Plan.

This plan involves closing the building, like the bunker it is, with a team of experts inside, able to work 24/7 monitoring the Data Center and SMC (Service Management Center) operations.

Of course, our team has time prior to the storm’s arrival to get their family and property to safety and be able to devote their attention, without fear of what might happen out there, to the proper operation of our Data Center. On this occasion, a team of 8 people stayed there from 9 p.m. on Wednesday, October 9, until 8 a.m. on Thursday, October 10.

Once inside, the team has an area perfectly equipped for rest, food and recreation.

But why is the building so tightly sealed?

There are two reasons: safety and structural risk, because during a hurricane, the air pressure inside and outside the building is so different that there is a serious structural risk of cracks or even the roof blowing off.

Guaranteed resources

A Data Center needs three things from the outside that are vital:

• Power.
• Water.
• Internet connectivity.

What happens if they are cut off? All three lifelines are guaranteed not only by your security systems, but also by their redundant design. Let’s look at them in detail:

Electrical power

Our Miami Data Center has two different power sources, one underground and one overhead, each with 9MW of power, ensuring a constant flow of power thanks to this redundancy.

It also has six 2MW generators, backed up by 6000-gallon reserve diesel tanks and an additional 12,000-gallon tank (for a total of 88,000 gallons), ensuring 100% operation for more than 12 days in the event of a total power failure.

Above all, there is the UPS or Uninterrupted Power Supply, a battery system that really came into its own during Milton, given the problems of flicker or blackouts that the supplier’s grid failure caused in many places in the area. Flickers are power outages or fluctuations of less than 60 seconds. These flickers automatically activate our generators, although they give the supplier a 90 second window to recover before supplying power to the system. These batteries can maintain uninterrupted power for 15 minutes, enough time for the supplier to recover.

Water Supply

Server rooms generate a lot of heat, and we all know what happens when a computer gets too hot. The air conditioning in our Miami Data Center is cooled by a circular system of air and chilled water called ‘chillers’.

To do this, the building is connected to Miami’s public water supply, but in the event it is cut off, the facility has its own well where the water supply is guaranteed.

For more security, the cooling system only needs one chiller, that’s how powerful it is, but our building has 3 to give an n+2 redundancy and more security for our operations.

Internet Connection

In the Data Center we have up to 13 different Internet providers, so there is unlikely to be a connection failure, including Telefónica’s own network. 

Our people: it would not be possible without them

The lockdown plan requires personnel to be on the move at all times, checking all critical systems at all times. Manual handling is essential as any alarm has to be dealt with immediately. On two occasions during the storm, our fire detection systems were activated due to the magnitude of the lightning strikes, which caused small electrical discharges that exceeded established limits, forcing the team present to check the entire building as per protocol.

Thanks to their dedication and professionalism, they were up all morning from the night the storm hit. Fortunately, there were no major flickers or real problems, thanks to the excellent maintenance of the equipment, the preventive plan to which they are regularly subjected and, finally, the fact that the storm did not hit them with full force.

Once again, we echo Mario’s words:

“The constant preparation and diligence of our team prove that we can guarantee the safety and reliability of our operations even in the face of formidable forces. Many thanks to our team for their dedication and hard work!”