The failure of a key federal safety system Wednesday led to widespread disruptions in domestic air travel for the second time in two weeks, prompting a fresh round of scrutiny from lawmakers amid continued breakdowns in technology.
The Federal Aviation Administration said a preliminary examination traced the outage to a damaged database file, but the agency is continuing work to pinpoint the cause of the issue.
The White House and Transportation Secretary Pete Buttigieg said they do not suspect a cyberattack or other external activity. Buttigieg said the FAA made the rare decision to shut down flight departures for about 90 minutes—a decision that wreaked havoc on the system much of the day—out of an abundance of caution.
“It’s been another challenging day for U.S. aviation,” Buttigieg told those who gathered at a transportation research conference Wednesday in Washington. Though the problems had been resolved, the nation continued to see effects “rippling through the system,” he said.
The failure of the FAA’s Notice to Air Missions system, or NOTAMs, came days after a meltdown at Southwest Airlines before Christmas that crippled flight operations, raising more questions about whether airlines and the agency that oversees them are doing enough to invest in and upgrade their technology infrastructure. Lawmakers pledged to probe the latest disruption as they begin work this year on a major package of legislation tied to FAA funding.
The nationwide flight stoppage was the first of its kind since the terrorist attacks of Sept. 11, 2001, said Michael McCormick, a former agency official.
“This is unheard of, and then the action that the FAA had to take in grounding all the flights makes it even more significant,” said McCormick, now a professor at Embry-Riddle Aeronautical University.
The NOTAMs system distributes warnings about potential safety hazards, such as closed runways, that an aircraft might face. Crews are required to consult the notices before taking off.
According to an FAA bulletin, the outage of the NOTAMs service began at 3:28 p.m. Eastern time on Tuesday. A backup system kicked in, then the main system resumed before problems reappeared, Buttigieg said.
Shortly before 8 p.m. Tuesday, the FAA issued a bulletin saying it was activating a hotline to manage the problem, inviting airlines to join.
In the middle of the night Tuesday, “it became clear that there were still issues with the accuracy of the information that was moving through the NOTAM system,” Buttigieg said.
About 5 a.m. Wednesday, the FAA conducted a “complete reboot” of the system, Buttigieg said. That attempt at a fix was “not sufficiently validated to feel comfortable” that the issues were totally resolved, Buttigieg said.
“At that point, the move was made to institute a ground stop until FAA could completely validate not only that the NOTAMs were populating correctly, but that they were actually getting out to the aircraft,” Buttigieg said.
That rare nationwide ground stop was issued at 7:21 a.m., halting most commercial air travel in the country for about 90 minutes, although airports and airlines struggled for hours through the backlog.
President Biden, who was briefed on the failures of the FAA system, directed the Transportation Department to investigate its causes, White House press secretary Karine Jean-Pierre said Wednesday. Congress also vowed to examine such questions as it begins hearings on funding the FAA.
Wednesday’s issues came amid efforts to modernize the system and address other concerns that have arisen over the years. The FAA is well into a years-long effort to improve the pilot alert system, saying it has consolidated information into one place and facilitated the process for computers to ingest data.
“In short, no NOTAMs, no flight,” the Aircraft Owners and Pilots Association said in a statement. “Everyone involved in this issue understands that systems and technology must be updated.”
Buttigieg said FAA safety systems constantly need to be upgraded and refreshed.
“There are a number of processes underway right now at the FAA to make sure that those systems stay up to date,” he said. “It’s been a big topic, certainly before, and since I arrived at this role.”
Sen. Maria Cantwell, (D-Wash.), chair of the Senate Commerce, Science and Transportation Committee, had previously announced plans to hold hearings into failures at Southwest Airlines that forced the carrier to cancel more than 16,000 flights between Dec. 21 and Dec. 31. Cantwell said Wednesday the committee also will review what caused the failure at the FAA.
“We will be looking into what caused this outage and how redundancy plays a role in preventing future outages,” she said.
Rep. Rick Larsen (D-Wash.), his party’s leader on the House Transportation Committee, said he spoke with Buttigieg on Wednesday, and would “continue to monitor this disruption to our air travel system until it is resolved.”
Two key Republican lawmakers were vowing to seek accountability and changes at the FAA.
Sen. Ted Cruz (R-Texas), the incoming top Republican on the Senate Commerce, Science and Transportation Committee, which oversees the FAA, said the agency’s “inability to keep an important safety system up and running is completely unacceptable and just the latest example of dysfunction within the Department of Transportation.”
Rep. Sam Graves (R-Mo.), chairman of the House Transportation and Infrastructure Committee also vowed to hold those responsible accountable.
Robert Mann, an aviation consultant, said a key question for Congress as it works on FAA legislation this year—a process typically conducted every several years—is what the government can do to modernize its systems and handle the nation’s growing volume of air traffic. He said the FAA is too dependent on aging technology—an issue also blamed in the Southwest debacle.
“They just can’t continue to do what they’ve always done,” Mann said. “A lot of these systems are decades old, hardware and software.”
NAV CANADA, that country’s air traffic control provider, said it also had an outage affecting newly issued NOTAMs for about three hours, beginning at 10:20 a.m. Eastern time. Spokeswoman Vanessa Adams said the cause is under investigation, but they do not believe it is related to the FAA problems.
“Mitigations were in place to support continued operations,” Adams said, noting that Canada did not issue an order barring flights from taking off.
Even as U.S. flights resumed Wednesday, delays continued to reverberate throughout the system. More than 1,3o0 flights into, within or out of the U.S. were canceled, according to the flight-tracking website FlightAware, while nearly 10,000 were delayed.
American Airlines said the carrier canceled nearly 400 flights and delayed 850 flights as a result of the FAA problem.
According to a memo circulated by American Airlines’ flight operations director Wednesday, the outage meant the airline could not issue flight plans or fueling paperwork. A memo sent later in the day noted it also faced challenges in booking accommodations for crew members.
The failure of the FAA’s notification system also comes when the agency has been without a Senate-confirmed leader for nearly a year.
President Biden nominated Phillip Washington, chief executive of Denver International Airport, to run the agency after its former administrator retired last spring, partway through his five-year term. The Senate Commerce Committee did not hold a hearing or vote on Washington’s nomination last year.
He has faced scrutiny over reports of a possible connection to a Los Angeles investigation linked to that city’s transit agency, which he previously led, and questions about whether he had enough aviation experience after a career spent largely in transit.
Wednesday’s problems could also exacerbate tension between airlines and the Department of Transportation, which disputed the causes of delays and cancellations last summer amid debate over how much responsibility air traffic controllers should bear. McCormick said the outage would prompt airlines to further question the reliability of the FAA’s infrastructure.
Some industry leaders pointed to the system failure as another example of the need to modernize the agency that regulates and oversees the nation’s airspace.
“Today’s FAA catastrophic system failure is a clear sign that America’s transportation network desperately needs significant upgrades,” said Geoff Freeman, president and chief executive of the U.S. Travel Association. “Americans deserve an end-to-end travel experience that is seamless and secure.”
The problems made for a difficult start to the day for many travelers.
Don Cleary, president of Marriott Hotels of Canada, was supposed to be on a 9:30 a.m. Air Canada flight from Washington to Toronto for an afternoon of back-to-back meetings. Instead, he was working on his laptop at Reagan National Airport and checking airline apps to see whether the Air Canada flight would leave Toronto before an American flight left Upstate New York. That would signal which one was likely to arrive in and leave Washington—with him aboard—first.
As delays on his Air Canada flight grew, he had booked another ticket on American as a backup. Meanwhile, at 9:30 a.m., he kept an eye on the terminal window, watching the runway as jets began to taxi out. It was a promising sign, but he said he wouldn’t become hopeful until one of his two flights had taken off. His afternoon meetings already were rescheduled for Thursday.
“I need to get there today, but I’m fully anticipating things will continue to get delayed,” said Cleary, a Bethesda resident who flies to Canada almost weekly. “It’s a mess . . . This is my first trip of the year. It’s not off to a good start.”
Doug and Lynn Fuchs, both professors at Vanderbilt University, took a seat on a dormant luggage conveyor belt at National for Lynn to work on her laptop. They had just rebooked their 11:35 a.m. Southwest flight, already delayed about 15 minutes, to Nashville for 6 a.m. Thursday. They said they assumed there was a good chance their original flight would be canceled as delays mounted across the country.
“We decided to just cut our losses,” said Doug Fuchs, as they prepared to head back to their Washington, D.C., home. “We didn’t want to spend all day in the airport.”
Those whose trips were disrupted may find themselves out of luck if they seek compensation beyond a ticket refund.
Carriers said cancellations and delays tied to Wednesday’s outage could spill into Thursday, but barring any additional problems, they expect normal operations on Friday.
Unlike Southwest’s flight disruptions, which were caused in large part by a breakdown of the carrier’s software, Wednesday’s canceled and delayed flights weren’t the fault of any airline. As a result, carriers are only required to get customers to their final destination or offer a refund if they opt not to take the rebooked flight.
As the end of the ground stop approached at Baltimore-Washington International Marshall Airport, pilots began asking air traffic controllers for guidance. A pilot asked at 9:02 a.m. if the situation had been resolved, according to a feed from LiveATC.net. “No, all is not good, but we are letting some people go,” the controller replied.
A few minutes later another pilot came on the radio: “What a morning.”