[Photo: "Sky during the black out. Yes, there are stars above San Diego". COURTESY OF Dave Maass, SAN DIEGO CITY BEAT]

SAN DIEGO, CALIFORNIA, SEPTEMBER 13, 2011 (COMPUTERWORLD) Officials say outage was caused by human error By Jaikumar Vijayan September 9, 2011 05:30 PM ET3 Comments Computerworld - Thursday's massive power outage across parts of Arizona and southern California serves as another reminder of the vulnerabilities in the nation's power infrastructure.

The outage appears to have been caused by human error. But that fact is unlikely to comfort the growing number of people concerned about blackouts that could be triggered by cyber attacks.

Just this week for instance, the Bipartisan Policy Commission, a Washington-based think tank, expressed concern in a report about attacks against the nation's power infrastructure. The report said that attacks capable of triggering "cascading disruptions and damage" to U.S. power infrastructure are not just theoretical threats, but a very real danger.

The southern California outage started around 5.30 p.m ET Thursday and left close to 1.5 million customers of San Diego Gas & Electric (SDG&E) without power for nearly 12 hours. As of this afternoon, power had been restored to all affected areas.

In a news conference streamed live by local TV stations this afternoon, SDG&E president Mike Niggli said the problem started when an employee with APS, Arizona's largest utility company, was working on a capacitor, at a substation near Yuma.

"It was a human error that was the initiating event that took the transmission line out," Niggli said, nothing that the incident caused power flows to be redistributed throughout large portions of the western United States. "We know what the initiating event was. The question is, how did that ripple through the rest of the systems?"

In a statement APS said the outage was related to a 'procedure' at the North Gila substation northeast of Yuma. The error resulted in the 500 kV transmission line near Yuma tripping offline. Existing measures should have isolated the resulting outage to the Yuma area, the statement said.

"The reason that did not occur in this case will be the focal point of the investigation into the event," APS said. That review is already under way.

In a cascading blackout, problems in one section of a power gird ripple out over the entire gird. Similar, larger blackouts have happened elsewhere.

In 2008 for instance, a fire in a substation near Miami triggered a cascading blackout across a large swath of Florida, leaving three million people without power for hours. In 2003, a similar blackout in the northeast affected close to 15 million people in New York, Connecticut and even parts of Canada and the Midwest.

Investigators later determined the problem started when an engineer with Midwest Independent Transmission System Operator disabled a software function that allowed the utility to determine the real-time status of the power grid in its region. That problem was later exacerbated by a software failure at FirstEnergy Corp., which contributed significantly to the problem.

It's too early to say for sure what happened with yesterday's blackout, said Joseph Weiss, managing partner at Applied Control Solutions LLC and author of the book Protecting Industrial Control Systems from Electronic Threats.

But the key takeaway is that a cascading blackout can just as easily be triggered by a malicious act as by human error, he said. "The only way you can tell the difference is the intent of the individual," he said.

Often cyber security analysts tend to view threats to the power grid in the same way they view threats to information networks, he said. Any incident that results in an industrial control system being taken offline because of something happening upstream is, in a sense, a cyber incident.

In his book, Weiss says that there have been at least 170 known cyber-related outages in the U.S., including three that caused widespread regional outages. The relative lack of forensics-gathering capabilities in the utility business makes it hard to determin whether any of them might have been the result of a malicious act, he said.

"Because we have so little control systems forensics, it is very difficult to determine what happened" with many of these incidents, Weiss said.

Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld. Follow Jaikumar on Twitter at @jaivijayan or subscribe to Jaikumar's RSS feed . His e-mail address is


How the Blackout Affected San Diego, California
By Gisela Sommer Epoch Times Staff

[PHOTO - BLACKOUT: An airplane over downtown San Diego gets ready for landing at Lindbergh Field, San Diego's main airport. (Gisela Sommer/The Epoch Times)]

A massive power outage that started at around 3:30 p.m. on Thursday, Sept. 8 in Arizona, shut down the electricity grid in large parts of southern California, Arizona and northern Mexico, all the way up to Orange county, affecting 1.4 million customers in San Diego county. About 5 million people lost power in the region. Employee error and high heat were to blame, according to officials.

San Diego was hit the hardest. Several people got stuck in elevators in high-rise buildings downtown and elsewhere, with service personnel prying elevator doors open with crowbars to get air to the trapped people, KOGO Clearchannel radio station reported.

With no air-conditioning, lights or computers functioning, everyone left work at the same time, creating huge traffic congestion all over the county. Gas stations were shut off too. People were told to conserve gas and not use their cell phones or landline phones except in extreme emergencies. San Diego Gas & Electric (SDGE), was unavailable for comment.

Emergency 911 phone lines were working on generator power, but inundated. Elderly people with oxygen problems filled emergency rooms. Palomar Hospital stayed open with generators but canceled surgeries.

In San Diego, the Highway Patrol said there were several collisions at intersections because street lights were also out and people didnít stop before proceeding. Roadside assistance trucks were not responding because they were saving their own gas.

Some electric trolleys got stuck on roads, some on overpasses, with passengers trapped inside.

Crossing the border from Tijuana, Mexico into the U.S. took three hours.

At San Diegoís International Lindbergh Field airport all outgoing flights were canceled at 5:30 pm, and some incoming ones were diverted.


Stores were out of ice and closing down.

About three million people in San Diego County without electricity; nothing like this has ever happened, according to KOGO.

Shortly before 10 pm an announcement was made that some areas had gotten their power back. Cox Communication said their workers would be going to work as usual on Friday to restore hi-speed internet access.

San Diego Gas & Electric Co announced Friday that the event was purely an employee error and that the system was coming back online, with most power expected to be restored during the day Friday. President Mike Niggli asked at a late night press conference that people conserve energy and turn off their air conditioners to prevent surge damage when the power comes on.

All public schools throughout the county will stay closed on Friday, and people were asked to stay off roads. Some outbound flights were still cancelled as of Friday morning.

Chief News Editor: Sol Jose Vanzi

All rights reserved