Boeing’s 737 Max flight tests underestimated risks, U.S. investigators say

Boeing Co.’s evaluation of the 737 Max system during development used an oversimplified test that didn’t anticipate the cacophony of alarms and alerts that actually occurred during a pair of deadly crashes, U.S. investigators concluded.

In the first official finding from a U.S. government review of the crashes that grounded Boeing’s top-selling airliner, the National Transportation Safety Board on Thursday issued seven recommendations calling on the Federal Aviation Administration to update how it assumes pilots will react in emergencies and to make aircraft more intuitive when things go wrong.

“We want them to step up how they certify these airplanes with regard to the human interface,� said Dana Schulze, the director of NTSB’s Office of Aviation Safety.

The NTSB — which is assisting inquiries in Indonesia and Ethiopia where the two crashes occurred — included some of the first details of how Boeing and the FAA certified the system that led to both crashes. Final reports listing the accident causes in those nations haven’t been completed. Under U.S. law, the NTSB has no regulatory authority and relies on its recommendations to improve safety.

The agency’s 737 Max recommendations don’t call for any specific updates to the plane or other aircraft, but they could lead to sweeping and costly changes.

The way Boeing designed its flight tests was permitted under existing rules. But the NTSB concluded that those standards, dating back to 1988, weren’t adequate. The agency called on the FAA to require more realistic assessments of complex emergencies during certification testing. Schulze said that should be done before the Max flies again.

The FAA, regulators in other nations and Boeing have made similar conclusions in recent months about the need to ensure that average pilots can respond to complex emergencies.

The FAA said in a statement Thursday that it would review the NTSB recommendations as it considers the proposed changes to the 737 Max: “The lessons learned from the investigations into the tragic accidents of Lion Air Flight 610 and Ethiopian Airlines Flight 302 will be a springboard to an even greater level of safety.�

Boeing said safety is its top priority. “We value the role of the NTSB in promoting aviation safety. We are committed to working with the FAA in reviewing the NTSB recommendations,� it said in a statement.

In the 13-page report accompanying the recommendations, NTSB acknowledged Boeing is revamping the plane to prevent the same malfunction from causing another crash.

“However, we are concerned that the process used to evaluate the original design needs improvement because that process is still in use to certify current and future aircraft and system designs,� the safety board said.

Boeing is fine-tuning the plane’s flight control software and it is aiming to have the grounding lifted early in the fourth quarter.

The NTSB recommendations also ask the FAA to review all aircraft models to ensure that they don’t have similar safety issues lurking in the background, and to urge other nations to conduct similar reviews.

In addition, the NTSB wants the FAA to find more scientific methods to assess how pilots will perform in crises and to explore how to make aircraft warning systems more intuitive.

Both the Lion Air crash Oct. 29 and the Ethiopian Airlines crash March 10 occurred after a malfunction prompted a safety feature known as the Maneuvering Characteristics Augmentation System to begin automatically and repeatedly pushing the planes into dives.

The crashes killed everyone aboard the planes, a total of 346 people.

Although news reports since the crashes have contained some information on Boeing’s flight tests and safety assumptions about MCAS, the NTSB report is the first official account of how it was certified.

Boeing assumed that an MCAS failure had a safety risk that was “major,� which was less severe than the “catastrophic� category, the NTSB said. The company assumed pilots would easily recognize an MCAS failure and counteract it.

Boeing’s 737s, including the Max models as well as earlier versions, have a relatively simple procedure for shutting off the motor that drives down the nose. Yet pilots in the Indonesia crash didn’t perform it and the crew in Ethiopia started the procedure but then reversed course, according to preliminary investigation reports. A third crew — on a Lion Air flight the night before that airline’s crash — was able to recover but also had difficulty figuring out how to respond, the NTSB said.

“In all three flights, the pilot responses differed and did not match the assumptions of pilot responses to unintended MCAS operation on which Boeing based its hazard classifications within the safety assessment and that the FAA approved and used to ensure the design safely accommodates failures,� the NTSB said.

At least part of the reason for those miscues is that the underlying failure — faulty sensors that measure the angle of a plane’s nose relative to the oncoming air — triggered multiple loud alarms that were potentially confusing, the NTSB found.

In all three cases, pilots faced a thumping warning of an aerodynamic stall, indications that their airspeed and altitude gauges weren’t accurate, as well as other alerts. And that was before MCAS began activating.

However, when Boeing assessed whether the system was safe during certification, test pilots flew far simpler simulator runs. They were tested on whether they could recover if MCAS began lowering the nose without any of the additional emergencies, according to the NTSB.

“While Boeing considered the possibility of uncommanded MCAS operation as part of its functional hazard assessment, it did not evaluate all the potential alerts and indications that could accompany a failure that also resulted in uncommanded MCAS operation,� the report said.

Years of research shows that pilots can be overwhelmed during complex emergencies with multiple alarms whooping accompanied by confusing, even contradictory, cockpit indications, the report found.

Boeing hasn’t yet responded to the NTSB’s recommendations, but the initial indications are that the company is receptive, Schulze said. A special review by Boeing’s board of directors Wednesday said the company should work with airlines to “reexamine assumptions around flight-deck design and operation,� particularly given shifts in demographics and “future pilot populations.�

The FAA this week also urged the International Civil Aviation Organization, an arm of the United Nations, to consider stronger training requirements for similar emergencies.

One of the NTSB recommendations calls on the FAA to review how emergencies are displayed to pilots in an attempt to make it clearer how to respond.

More modern planes, such as the Airbus SE A350 or the Boeing 787, have computerized systems and automated checklists to help flight crews in emergencies. It might be more difficult to equip the 737 with such systems because that plane uses older technology.

The recommendations urge the FAA to “develop more robust tools and methods� to assess how pilots will react in emergencies. Currently, test pilots try to determine reactions of average cockpit crews, but that may not be sufficient, according to the NTSB.

If the FAA follows the recommendation, one solution might be to begin using randomly selected pools of pilots from around the world to get a better sense of how they will behave, Schulze said.

While the Max’s design has been central to the investigations of the two crashes, the NTSB recommendations highlight how actions by the flight crews also played an important role in sending their aircraft into steep dives.

Human failures in such emergencies have been a recurring theme in accident investigations for decades, said Evan Byrne, the chief of NTSB’s Human Performance & Survival Factors Division.

“Through these investigations and with these recommendations, we’ve identified a gap with human-machine or human-airplane interface,� Byrne said. “What we’re trying to do is close that gap as it relates to multiple alerts going off simultaneously.�