Sunday, June 04, 1995
Terry McDermott
(Copyright, 1995, The Seattle Times Co.)
– —————————————–
With
150 computers controlling more than 3 million parts, all of it built to
move at 600 miles an hour and withstand collisions with everything from
flocks of geese and lighting bolts to human error, the Boeing 777 is
the most complex machine ever built.
While
praising it as a shining example of American industrial prowess, some
critics also see the 777 as an illustration of contradictions and
inadequacies in the country’s air-safety system.
In
particular, they worry that modern aircraft manufacturers such as
Boeing have outstripped the financial and technical capacity of the main
air-safety agency, the Federal Aviation Administration.
In
scores of interviews with industry executives, government regulators
and safety experts, and in thousands of pages of government records and
industry documents, the portrait of the FAA that emerges is
one of an agency dominated by the industry it regulates, and nearly
overwhelmed by the scope and difficulty of its many jobs.
There is a wary consensus that the FAA stands aside and watches as industry charges ahead. Unresolved is whether the FAA is standing so far away that it can no longer tell when something goes wrong and the safety of airplanes is compromised.
While strongly disagreeing with this portrayal, FAA officials acknowledge the agency over time has delegated ever more of its responsibility to private industry.
The
FAA’s own staff is stretched thin and does little more than randomly
audit the safety tests new airplanes must pass. An overwhelming majority
of the work certifying the safety of the 777 has been done by Boeing
employees, not the FAA. Some tests were changed from the original test plan. Some were never done. Most were never even reviewed by the FAA.
The
agency inspects many more pieces of paper than it does airplanes. It
has, for example, more economists doing cost-benefit analyses of
proposed safety rules than it has inspectors in Boeing’s factories.
FAA
and U.S. Department of Transportation officials argue that commercial
aviation remains safe. Their position is supported by almost all
available data. Contemporary air travel remains one of the safest means
of transportation ever devised.
The
flying public has been protected largely because the overriding
economic interests of companies and personal pride of individuals in
commercial aviation both are biased toward safety.
The
danger inherent in this is that safety concerns can be overridden if
the need for profits becomes too great, development becomes too hasty,
or personal pride too weak.
Some
of the agency’s staunchest supporters say the industry ought to be
responsible for safety. Asked who should run the test program for the
777, an executive in the FAA’s Seattle Aircraft Certification Office,
said:
“It is their program. I didn’t spend a dime on it.”
Boeing
retained control of the 777 testing regime from beginning to end and
shielded it from public review by claiming the need to protect trade
secrets. The FAA honors these claims, agreeing with
Boeing’s refusal even to release a full list of tests performed on the
airplane. Boeing declined repeated requests for interviews for this
story.
Agency staff frequently liken their role to that of police.
“You know the Texas Ranger approach?” one test engineer said. “There’s only one riot so you send one marshal, right.”
This
metaphor might provide little comfort to taxpayers, who perhaps expect
more than riot-control from 48,000 federal employees charged with
keeping airplanes in the air.
UNDER SIEGE
This spring, following the Oklahoma City
bombing, security was increased at all federal offices. Additional
guards and metal detectors were placed in their lobbies, and tank traps
in their plazas.
The extra precautions imparted a sense of embattlement to Washington, D.C. This is nothing new at the FAA, which has been under siege for years.
While not as large as the largest federal agencies, the FAA
is of considerable heft. The agency spends $8 billion a year,
three-quarters of it on air-traffic control. The rest is used to
regulate virtually all other aspects of American aviation, everything
from engine design to a flight attendant’s pre-flight announcements.
Under the FAA’s control are:
— 48,000 employees, one for every six airplanes in the U.S.
— 18,000 airports.
— 420 FAA-staffed control towers.
— 90 district offices.
— 6,500 commercial airliners.
— 275,000 private airplanes.
— 4,400 licensed repair stations.
— 500 pilot-training schools.
—
650,000 pilots, all of whom must be licensed, as must all mechanics,
factories, parts, hot-air balloons, helicopters and airliners.
These people, things and responsibilities are scattered throughout the United States and, in the case of manufacturers, the world. Given this breadth of purpose and place, the FAA is foremost a bureaucracy, and without question suffers from some classic bureaucratic flaws.
The FAA
has been repeatedly and at times harshly criticized by Congress, which
has characterized it as recalcitrant; by the aviation industry, which
sometimes regards its regulators as incompetent and overbearing, at
other turns meek; by the Department of Transportation’s inspector
general, which has accused the FAA of laxity; and most cuttingly by the National Transportation Safety Board (NTSB), which has at times accused the FAA of almost wanton disregard for safety.
There is much about the agency that just doesn’t seem to work:
— Staff levels and expertise fluctuate.
The FAA
is fodder in the constant Beltway war among parochial, political and
ideological interests. It is forever being pushed this way by political
pressures, pulled that way by budget economics. Funding fluctuates and
causes cyclic waves in the work force. This inconsistency limits the
agency’s ability to hire technical experts and drives some of the its
best people into early retirement or private industry. A program to hire
“at least one world-class expert” in every technical specialty was
begun in 1979. It has never been fully staffed.
— Training is inadequate.
No one ever seems to have enough money to train the employees who are hired. In one recent two-year period, only 1 percent of FAA
engineers responsible for approving aircraft software attended a
software-training course, even though software technology is changing
rapidly.
— Record-keeping is haphazard.
Much
of the voluminous data collected from airlines, manufacturers and the
agency’s own employees is never analyzed. Inspector-general audits find
databases poorly maintained and sometimes rife with error. FAA databases “are inaccurate, inconsistent, and often incompatible,” the General Accounting Office has testified to Congress.
— Research and development of new technology is lamentable.
Developing
a new automated air-traffic-control system has become such a nightmare
that agency officials, after a decade of screw-ups, now come to news
conferences equipped with flow-charts and diagrams of decision matrices
to explain where the process went off-track this time.
In
1992 testimony to Congress, the accounting office analyzed the FAA’s
attempt to develop a congressionally mandated statistical program for
analyzing safety.
The FAA
spent four years and $7 million and had made “little progress,” the GAO
said. In that time, the agency developed a concept, published a plan,
then, when a new FAA administrator arrived, started over from scratch. Completion of the program is “still years away,” the GAO concluded.
“You
can’t acquire and improve the kind of technology in this business on
seven- to 10-year cycles. The technology only has about a three-year
life cycle. By the time you’ve installed it, it’s obsolete,” said Joseph
Del Balzo, a former acting FAA administrator.
— Decision-making is diffuse.
Roger
Fleming, senior vice president of operations for the Air Transport
Association, a trade association for airlines, said the FAA’s emphasis
on consensus management created an almost complete lack of individual
responsibility for decisions, with timidity and uncertainty pervasive.
— Oversight is weak.
In a 1991 letter from NTSB Chairman James Kolstad to FAA Administrator James Busey after a crash, the board virtually accused the FAA of killing people.
“The
Safety Board questions the FAA’s depth of commitment to provide
effective quality assurance and safety oversight of the Air Traffic
Control system,” the letter said. “The fatal accident, which might have
been prevented if FAA . . . had identified that mandatory
redundancies were not present, demonstrates conclusively an inadequate
and ineffective quality-assurance and safety-oversight program . . . .
(The FAA’s) Office of System Effectiveness is . . . in effect evaluating
itself. It is organized in such a way that no actual oversight exists.”
ACTORS AND THEIR ROLES
Conversation with government managers invariably includes discussion of agency tables of organization. With the FAA,
almost every such conversation includes the caveat that the table of
organization you are being shown is out of date, sometimes by as many as
two cycles of reorganization.
One recent conversation with an FAA
manager included so many references to managers who were no longer the
people named on the chart he was using that he quickly switched to a
shorthand description of who, in fact, held what position.
“That’s
an actor,” he said, crossing one name out of a key position and writing
in the name of a temporary replacement. This one’s gone, that’s an
actor, he said, crossing out two more.
The pencil continued across the sheet.
“She’s moved here. Actor, actor.”
By the time he was done, three-quarters of the people on the year-old chart had been eradicated.
The FAA
is an independent agency within the Department of Transportation. The
agency’s chief executive is a presidentially appointed administrator. He
serves directly beneath the secretary of transportation. Washington being Washington, these relationships are in constant flux. So is the agency.
It
is run by political appointees who are constantly shuffled and who
themselves then shuffle the chairs beneath them. The average tenure of
transportation secretaries over the past 15 years has been 22 months.
The average tenure of FAA administrators has been 18 months. That’s when there is an administrator. For long stretches, the office has been unoccupied.
David Hinson, the current FAA
administrator, was nearly forced to sign a blood oath when he took the
job, guaranteeing he would stay in it through President Clinton’s term
of office. Hinson said the past turnover was “terrible. No, terrible is
the wrong word. The right adjective is that’s unfortunate.”
“You can’t keep jerking an organization around, based on the arrival of a new plan,” said one former senior FAA
executive. “I can’t think of an organizational plan we haven’t tried.
We never stay with one long enough to know if it works or not. It takes a
year after a reorganization just to settle in. By then, we’re off to
another plan.”
One
ironic result of this constant change is an agency that resists change.
The churning up above promotes resistance from career staff down below,
said one senior airline executive.
“If they don’t like what’s going on, the full-time people simply hunker down and wait for the guy to leave.”
FISH BONES AND BUCKETS
As a society, we have at least two fundamentally different conceptions of aviation safety.
To the FAA, as well as to the industry it oversees and represents, safety is an airplane that flies.
To nervous passengers, safety is an airplane that might not.
To
say that a modern airliner is safe or unsafe, given these contrary
definitions, is futile. Airplane safety very seldom is or is not. It is a
matter of degrees and margins; it is layered.
You
might peel away one layer, puncture others, and normally what will
happen is the margin will shrink, but the plane will not in most cases
crash. For one small example, most airliners have at least three
electrical generators. Even if all three fail, there is usually a
battery backing them up.
All
commercial transport airplanes fly almost all the time. They almost
never crash. This is indisputable. You are more apt to die eating dinner
than flying in a modern airliner.
This
year, more people worldwide will die being transported by horses than
airplanes. More people die every day in automobiles than die in
airplanes in a year. Many more people will choke to death on various
chicken bones, fish bones, and other foods, some as innocent as a piece
of whole-grain bread, than will die in airliner crashes.
According
to the National Safety Council, in some years your chances of drowning
in a five-gallon bucket are nearly as good as dying in an airliner
crash.
An
airline passenger dies in a crash on average once every 2 billion
miles. In other words, if you boarded an airliner today and started
flying nonstop at 500 mph, you could expect to crash sometime in the
year 2451.
But airplanes do crash.
That airplane you are about to board might not get where it’s going. It might crash and if it does, chances are, you will die.
The FAA is charged with preventing this.
The
question of how offers a clear distinction between the reigning
political ideologies of the day. One regards government regulation as
overbearing, and one sees it as insufficient.
The distance between these positions is not merely extreme. It is infinite. Somewhere in the vague middle is the FAA. It’s a tricky place to be.
WHO’S IN CHARGE?
Like most FAA
administrators before him, David Hinson came from the industry he
oversees. A former Navy pilot, he has spent his career in aviation,
eventually founding and running Midway Air, a small carrier that ended
in bankruptcy. He has also sold airplanes for McDonnell Douglas. He is
an unabashed advocate.
There
is, for example, a handsome scale model of the Boeing 777 prominently
displayed in his office. Hinson does not discriminate, he said, taking
care to point out a McDonnell Douglas MD-11 across the room.
In this, he is a willing soldier in the Clinton
administration’s effort to transform trade policy into foreign policy.
His boss, Transportation Secretary Federico Pena, last month told Boeing
employees, “We in the administration are going to do everything we can
to support your sales . . .. We’ll travel to every continent of the
world to help Boeing sell airplanes.”
Pena
and Hinson, in fact, led a delegation of American officials on a trip
to Saudi Arabia in 1993 to sell airplanes, including the 777, which the FAA at that point had not even begun flight testing.
Hinson sees no conflict in this.
“I think that’s the president’s view, that the United States
government has a role to play in promoting American commerce, not just
airplanes, but American commerce in general. He’s not bashful about that
and I don’t think the secretary is, either. I’m not. I’m certainly not
bashful about promoting American airplanes.
“This
is a very good airplane, by the way,” Hinson said of the 777. “I flew
it three hours, three months ago . . . . I used to be an engineering
pilot, so I know what I’m doing. So I flew it 2 hours and 51 minutes and
did 26 items on a work card.
“It’s going to be a fine airplane, like they all are. All of them. There are no bad airplanes now.”
The
process of proving that an airplane is “fine” takes five years. Flight
testing, which is often thought of as the entire test of a new aircraft,
is more akin to the final exams at the end of a school year.
The
overall system is not much different in principle than applying for a
building permit. The process begins with a manufacturer’s application to
the FAA for certification of the airplane’s design. Filing
this application, a single sheet of paper, starts the clock on a
five-year process that covers 1,800 subsets of things to be tested.
The basic process is simple:
Agree on the rules that apply to the airplane.
Assure that the design meets the rules.
Assure that the airplane and all its parts meet the design.
Test the airplane to make sure it does what the design predicted.
Assure that the manufacturer has a system to build each airplane according to the design.
The
end of all this – the graduation ceremony – is the awarding of three
certificates: a Type Certificate, given to the design; a Production
Certificate, given to the factory; and an Airworthiness Certificate,
given to each airplane that is built.
This
work is done within the FAA’s Aircraft Certification Service (ACS),
which is divided into different “directorates” for large, small and
foreign airplanes, for engines and helicopters. The ACS headquarters is
in Washington, D.C. The
different directorates are in the regions of the country where most of
their activities take place. The Transport Directorate, which certifies
airliners, is in Renton.
In
Hinson’s view, aircraft manufacturers have become so skilled, what they
need mainly from government is efficiency, not impediments. Throughout
the agency, people talk about providing a service to industry, routinely
referring to manufacturers and airlines as “customers.”
Tom
McSweeney, head of the Aircraft Certification Service, said the agency
“is always in the situation of, if we drag our feet and we don’t do our
job right, the applicant suffers an economic burden, significant.
“I’m
sure Boeing has delivery penalties with United (Airlines, the first
purchaser of the 777), and if we miss our (schedule) by a week because
we screwed up, you better believe the administrator would hear about it.
I would hear about it in a heartbeat.”
Indeed, throughout the 777 testing program, Boeing repeatedly warned the FAA
against delays. One 1991 letter, obtained under the Freedom of
Information Act and written by Boeing’s John Miller, chief engineer on
the program, argued against an FAA request to audit what it considered “critical” software.
This
“would have a major impact on the work schedule,” Miller wrote, adding
that such audits “could even delay the program.” The FAA
and Boeing refused to say how the conflict was resolved, but it is clear
from numerous such exchanges that the certification schedule and much
of its content was dictated by Boeing. Boeing routinely reminded the FAA that technical documents submitted for review “should be returned to Boeing immediately following use by the FAA . . .. Boeing does not authorize the FAA to retain any portion of the materials being supplied.”
Even most safety-test data is kept by Boeing. FAA inspectors must request it for review.
This
system of record-keeping began for the most mundane of bureaucratic
reasons. Several years ago, the General Services Administration, which
functions as a sort of landlord for federal agencies, decreed that
federal offices could only have an amount of file storage proportionate
to the number of employees.
The FAA no longer had enough room for its files. The agency solved the problem by making manufacturers store the test data themselves.
RULES OF THE GAME
FAA headquarters in Washington is across the street from the Smithsonian Institution’s National Air and Space Museum. While the museum, with its banners and videos and souvenir shops, is more obvious about it than the FAA with its standard-issue tile-floor and steel-desk office building, both places are in a way celebrations of flight.
The museum contains aviation’s past. The FAA, through its husbandry of aviation’s rules, determines its future.
One
shortcoming in the museum’s history is its limited acknowledgement of
the degree to which aviation’s successes have been shaped by its
failures. Keeping heavy vehicles in the air has been, for much of the
time people have attempted it, a difficult task. The panorama of human
flight is littered with corpses.
Dead bodies are everywhere at the FAA. Visions of them, stacked like cord wood, dominate the psyche of the place.
Failure
and fear of it, to a significant degree, shape what might be regarded
as the operative definition of safety, the Federal Aviation Regulations.
The regulations, called the FAR, do indeed define a lot of things. At
198 chapters, they would seem to be about as much definition of anything
as anyone could want.
“Each
system,” the FAR says, “must be designed and installed so that the
error in indicated pressure altitude, at sea level, with a standard
atmosphere, excluding instrument calibration error, does not result in
an error of more than 1/-30 feet per 100 knots speed for the appropriate
configuration in the speed range between 1.3 VS0 with flaps extended
and 1.8 VS1 with flaps retracted.”
“Flight
level means a level of constant atmospheric pressure related to a
reference datum of 29.92 inches of mercury,” the FAR says.
This goes on for more than 4,000 pages, but even at that length, the FAR is often not as specific as it needs to be.
McSweeney,
the certification director, said: “Most of our rules are very, very
general in their safety intent. You go into many of our rules, it
doesn’t say you need four of this and two of that. You need to be able
to show for a certain kind of maneuver the airplane is stable. Well,
what the hell does stable mean?”
The
process of approving a new airplane is largely a question of answering
such questions, of applying the agency’s rules to the technology.
We
forget sometimes that aviation is still a relatively recent phenomenon.
The Wright brothers’ famous first flight in 1903 occurred within memory
of people still alive. What this means is that the culture of aviation
is still attached to the uncertainty of its beginnings. There is still
as much “flying by the seat of the pants” as computerized “flying by
wire.”
In
practice, this is less evident all the time, but in the life of the
mind, a lot of pilots still have some of that attitude. And certain
parts of the FAA, notably its certification branch, have more of the pilot mindset than do others.
Hand
me that wrench. Pull on that stick. We’ll get this thing up. This is
cowboy country, a land of baling wire and barrel rolls. It’s Neil
Armstrong overriding the computer on the lunar lander.
“You’re not looking for something to fail, you’re looking, how can I fly this thing,” said a lead FAA test pilot on the 777 program.
“No
airline pilot is going to walk up to a brand-new airplane and say,
`This looks unsafe to me.’ Well, he gets in there and says, `I can fly
anything you can pull out of the hangar.’ ”
This subjectivity is at the core of the way the FAA
approaches airplanes. It reveals itself in the agency’s general
disregard for data analysis, in the wide latitude given individual
safety inspectors, in the variable interpretations of rules from region
to region across the country and in the agency’s approach to testing.
Even
in the many instances where the FAR are definitive on one subject or
another, the law provides an all-purpose escape clause that allows the
agency to abide by the FAR, or do whatever the administrator decides. It
is a reflection of the basic service culture of the agency, which sees
itself in association with, not antagonistic to, the industry it
regulates.
Overwater
flights by twin-engine airplanes like the 777 are a good example. The
FAR say large transport aircraft – known to the world outside the FAA
as commercial airliners – are not allowed to fly farther than two hours
from any airport, that is, over oceans, unless an airplane has at least
three engines. Even in cases like this, the FAA is willing to accommodate industry if industry is persuasive enough.
It often is.
THE WIZARDS
Boeing’s
theoreticians, if not quite the kings of this domain, are at least its
wizards, making not just the planes but the rules. They are able –
magically, it seems at times – to conjure up just about anything.
Need
a plane that will haul 500 people? Here’s a 747-400. Want a hypersonic
transport to haul 700? We’ve got one on the boards. Need a
fuel-efficient plane to fly over the ocean? Try one of these nifty
twin-engine 767s.
They’re not allowed to fly that far from land? We’re working on that.
Indeed,
they were. Boeing, beginning in the early 1980s, began building a case
for the possibility of two-engine planes flying as far as three hours
from the nearest airport, something that would make ocean crossings
possible. Called Extended-Range Twin-Engine Operations (ETOPS), this
concept ran head-on into the FAA prohibition against it and won in a walkover.
When asked how the government determined ETOPS was feasible, three different FAA
executives said the question really ought to be addressed to Dick
Taylor, the Boeing vice president who pushed the idea at innumerable
industry gatherings in the 1980s.
It
was Taylor who produced the logic-shaking argument that airplanes with
fewer engines are less likely to fail than airplanes with more engines
simply because there are fewer engines. “In fact,” he wrote, “the
probability of a jet engine-caused accident is minimized by reducing the
number of engines on an aircraft.”
(If
you carry this argument out, of course, you conclude that a one-engine
airplane would be safer than a two-engine plane; and a no-engine
airplane safer than either. This argument gained more supporters than
its nearly comic tautological quality might predict.)
McSweeney: “I know when he first came into the FAA, that was years back, I know he stood up in front of a bunch of FAA people who went, `You gotta be kidding me.’ ”
He
wasn’t, and beginning in 1985 twin-engine planes were allowed to fly
routes up to two hours from land, providing they demonstrated superior
reliability over a period of years. This proved to be a camel’s nose
under the tent. In 1988, the FAA began allowing three-hour
ETOPS flights. The economies of this so delighted airlines, Boeing
decided to go for the whole camel, not just the nose.
When it began developing the 777, Boeing again approached the FAA.
This time, the company wanted the rules changed to allow the new plane
to begin flying the long, over-ocean routes as soon as it entered
service, skipping the usual two- to three-year reliability
demonstration. Since the whole concept of ETOPS was based on what the FAA called “demonstrated as opposed to projected reliability,” this seemed impossible.
“We
were very skeptical in the beginning,” said McSweeney. “What we didn’t
say was no. What we said was, `If you can show us.’ Quite frankly, I
don’t think anybody ever thought they’d get there. But we didn’t say no.
We didn’t say it’s absolutely impossible. We said, `It’s entirely
possible if you do it right.’
“And that’s always our position. We do not hold back industry, but we also make them meet the rules.”
In
this case, the rules that had to be met, however, were not the old
ones, but brand new rules written specifically to accommodate the
request.
DELEGATED EXPERTISE
Last year, during the testing of the 777, an FAA
official noted in an internal memo the complaint of a European
counterpart. The Europeans were conducting tests simultaneously with the
FAA so the 777 could be flown to their countries. The
European said that in trying to get certain information about the
airplane, he had been told by FAA engineers “they don’t know what is going on with the certification program because (Boeing employees) are taking care of it.”
The FAA
official who authored the memo, Kanji Patel, castigated the engineers
not so much for not knowing, but for telling the Europeans they didn’t
know.
“So
next time, when your (European) counterpart asks you a question for
which you are responsible, please find the correct information and call
him/her back at a later time.”
Within
the industry, the FAA’s new-airplane-certification work is highly
praised, described as one of the best things the agency does. But it is
just as frequently said that the best thing the agency does in
certification is stay out of the way of the manufacturers, who control
the process from beginning to end.
In
fact, most of the hands-on testing of a new plane is done by regular
line employees, paid by the manufacturer, assigned for temporary duty to
the FAA.
The FAA has been using these “designees” since the 1940s.
The
agency argues that designees effectively multiply its work force and
allow it to do things it would otherwise be unable to accomplish. The
use of designees has been studied repeatedly and generally been found to
be effective, if a little bit troubling.
Most outside examinations of the system conclude the use of designees could cause the FAA to completely lose touch with the certification process, its technical skills to disappear. The FAA needs to develop more of its own expertise, they say.
The FAA
asserts it has spent more than 100,000 hours certifying the safety of
the 777, more time than on any certification effort in history. But the
fact remains that a tiny percentage of the hands-on testing of the 777’s
complex systems was conducted by FAA employees.
The
777 is many times more complex than the space shuttle or an atom
smasher. The only machines in frequent use by consumers that might rival
it in complexity are computer-controlled telephone switching and
routing networks. And they do not fly.
The
777 is unusual even among sophisticated airplanes in its complete
reliance on computers for sensing the plane’s environment, translating
it into instructions, and sending those instructions back along
electronic pathways to fly the aircraft. The pilot mostly watches. Most
of the 150 computers on board are integrated with one another and the
software code that governs what they do is dauntingly complex.
“You
can’t look at it, you can’t feel it; it’s collections of ones and zeros
and there’s no way you can test it, in and of itself, and be sure that
it’s good,” one engineer said.
When
software fails, nothing breaks, there’s no visible evidence, it’s just
zeros and ones diving invisibly off the ends of connector pins. Judging
the reliability of such systems requires specific expertise and intimate
knowledge of how they are designed and built.
Yet GAO investigators have determined the more complex the task, the more likely the FAA is to let the manufacturer judge it. A 1993 internal FAA
study said the agency’s engineers did not understand the complex
flight-management system on the 747-400 and had delegated oversight of
it and 10 other systems entirely to Boeing employees. The study said FAA
staff “were not sufficiently familiar with the system to provide
meaningful inputs to the testing requirements or to verify compliance
with the regulatory standards.”
This was at the absolute edge of responsible use of the designee system, the FAA study said. In reviewing this, the GAO concluded delegation has since increased; if there was an edge, the FAA had gone over it.
In reply, Anthony Broderick, associate FAA administrator for regulation and certification, said the need for scientific skill was overstated:
“In general, the GAO report places far too much emphasis on FAA
certification personnel having detailed scientific knowledge. It is far
more important for our engineers to understand the regulation and how
they can be acceptably complied with . . . FAA engineers do not have to design airplanes.”
The FAA
does not keep records that identify who did what in the testing and
certification process. Test reports, even the test schedule, are treated
as trade secrets by Boeing and so are unavailable for any public
review. But it is clear that an overwhelming majority of safety tests on
the 777 were conducted by Boeing employees under supervision of other
Boeing employees designated to act as FAA agents. Most of
the work of these designees is only loosely supervised. Most test
reports are not even glanced at. The FAA’s technical specialists make
spot checks. Internal FAA documents talk about “following threads” through complex systems.
Do you read all test results for all the systems in your area of responsibility, an FAA avionics specialist was asked?
“Impossible,”
he said. “The ones that pass we don’t even look at; they’re fine. That
means we trust that the work was done as expected.”
In
summary, Boeing designed the airplane, wrote the plan to test the
design, executed it, and largely affirmed that it had been executed.
Joseph Del Balzo, the former acting administrator, said that in some ways the FAA has made virtues of necessity.
“At the end of the day you end up with a set of regulations. What the FAA must do is rely on industry to meet the regulations. FAA will never have the capacity to do more than that. The agency does nothing hands-on and never will.
“Even
if you had the resources, I’m not sure you would want to. If you don’t
have the confidence in industry it will never work anyway. That doesn’t
mean mistakes weren’t ever made. They were. They are.”
CLOSE ENOUGH?
The FAA’s influence on the certification of a new airplane is greatest at the very beginning of the program.
“This
is not a business that lends itself to `you guys build it, we’ll be at
arm’s length and when it’s ready to test, you bring it to us and we’ll
test it,’ ” Hinson said. “That is a disaster in the making. That is a
bad idea that has outlived its time.”
The
agency’s relative lack of staff depth is less a hindrance when the
plane is still only a concept. At that level, when the work is more
narrowly contained, research papers are written, issues are seriously
debated, argued and worked out.
Then all hell breaks loose.
The
five-year development and testing of the 777 has been an often-frantic
affair in which seemingly stringent testing requirements were liberally
bent, if not broken, in order to meet deadlines.
When
schedules slipped, tests on what were to have been “mature” systems
became tests on systems still under development. For example, the final
version of the 777’s flight-control computer software was delivered in
April this year – 11 months over schedule and just four weeks before the
first airplane was delivered to United Airlines.
Changes like this, FAA officials say, were to be expected.
“You
are going to have problems and corrective actions,” a senior executive
of the Transport Directorate said. “If there’s a problem, what does that
mean? It means that they’ve identified something and corrected it.”
A
test pilot said: “We don’t say, boy, this is a piece of junk, it’s
never going to go. We just get in and say, OK, this is how it worked. We
come back in the post-flight and say, this is how it’s supposed to
work. And this is how it did work. You’ve got some work to do. There’s
no such system, there’s no such thing as they can’t fix it. They made it
in the first place. There’s nothing that can’t be fixed. It’s just a
matter of when.”
In part, this is why you have a test program: to identify problems.
The
difference in the case of the 777, however, is that a significant
portion of the test process was supposed to be conducted after
development ended.
In
order to qualify for the immediate permission to fly the plane on long
ocean routes (ETOPS), Boeing had proposed that a series of stringent
tests on a mature aircraft replace the normal two years of demonstrating
reliability in service.
It was a hurdle that many people thought Boeing couldn’t clear.
The FAA accepted the test plan. But a senior FAA test pilot said agency employees at the working level never took completely seriously all of Boeing’s proposed test plans.
“We
knew that was B.S. as soon as we saw it,” the pilot said of the ETOPS
testing regime. “Their original theory was that it’s going to be so good
that the airplane will never have a squawk on it. Just fly off into the
sunset and live happily ever after. We knew that wasn’t true and the
people in the trenches at Boeing knew that wasn’t true. And we said,
well, we’ll take a look at it.
“In
truth, they only got about halfway to where they thought, the perfect
airplane, maybe 60 percent. But it was so much better than any other
certification program that’s ever been done that it’s amazing . . . It’s
amazing to me they did as well as they did.”
FAA managers assert the 777 met the rules, albeit rules that were not as firm as had been assumed.
For example, getting ETOPS approval required a special series of 1,000 flight tests beyond the regular testing. The FAA
had said publicly that development work was supposed to be done by the
time the airplane entered those tests. It wasn’t completely finished.
The tests went ahead anyway.
“It
goes without saying that in any program you’re going to have minor
things come up that have to be adjusted. To say that that airplane would
have to go all the way through the 1,000-cycle program and never change
– that’s not the real world,” said one of the FAA managers on the program.
In other words, the final determination is subjective?
“Right,” he said. “The intent of the condition was as close to maturity as you can get.”
McSweeny
calls this subjectivity the “ability to look behind the rule” for the
rule’s intent. It is not, he says, a license to evade the rule.
The question of subjectivity goes beyond ETOPS. Subjectivity suffuses an FAA
process that many suppose ought to be objective. But interpreting rules
too strictly would stifle innovation, McSweeney and others argue.
“When
we talk to our people about our rules and regulations, we spend a lot
of time talking about the safety objective of the rule. My feeling is if
we just applied the letter of the rule as it was written . . . well,
our industry wouldn’t be where it is now because we would have been
holding them back.”
The rules could be changed, but, McSweeney asked: Do you know how long it takes to write a rule?
THE NEVER-ENDING STORY
Moses
spent 40 days on the mountain top getting the Ten Commandments.
Franklin Roosevelt needed only 100 days to change the course of American
history. The atomic bomb was invented in three years.
Not everything goes quite so fast.
The FAA
has been trying to figure out its rules on flame-proofing aircraft
interiors for 30 years. The agency has been proposing, amending,
recommending, and reproposing requirements for flight data recorders for
40 years, and some aircraft still in service have 1960s-technology
recorders.
The
agency’s rule-making process is so slow, a former administrator said,
it seemed designed to prevent rules, not make them. The histories of
some rule-making proposals fill entire shelves.
Certification
rules are based on decades of monitoring airplanes in service. This
surveillance of airline operations is the other half of the FAA’s effort
to make commercial airplanes safe.
FAA
staff monitor by telephone, computer and written report a huge, if
largely unorganized, agglomeration of data daily. Major noninjury
events, such as near collisions, are treated as if they were accidents.
Equipment failures are routinely reported and analyzed. Much of this is
done nonsystematically by the same technical staffs that certify new
airplanes. Maintenance logs are checked by inspectors assigned to each
airline.
All
of this monitoring of airplanes in service is regarded as one of the
agency’s highest priorities. Ninety to 95 percent of their work is
proactive, executives say.
There is probably no point on which the agency and its critics are farther apart.
The
critics say the agency has a “tombstone mentality,” doing nothing until
people die in an accident, then doing only what industry allows.
Nothing could be further from the truth, said the FAA’s McSweeney.
“Almost
exclusively, all safety problems get resolved before they cause
accidents,” he said. He points to all the nonaccident-related rules the
agency issues annually in the form of Airworthiness Directives. These
directives are orders issued to airlines and manufacturers for
corrective actions in some operation or airplane design. The FAA
issues on average 350 of them a year. Only a handful of fatal accidents
occur each year; obviously, most of the directives are the result of
something other than accidents.
Yet the agency gets little credit, the FAA
says, largely due to its defensive position vs. the National
Transportation Safety Board. The NTSB, with significant staff support
from the FAA, investigates every major civilian air accident. Few things are more riveting than an air-carrier crash.
“A great big smoking hole in the ground is a pretty spectacular sight from the public standpoint,” said one FAA executive.
The NTSB issues recommendations at the end of every investigation. The FAA is compelled by law to respond to the recommendations. Most often, the FAA
agrees with and attempts to implement the NTSB’s recommendations. About
one fifth of the time, according to NTSB data, it does not. The NTSB
makes similar recommendations on highway, marine and railroad safety.
The FAA’s rate of acceptance is annually among the highest.
But the public nature of air crashes and, frequently, the reasons the FAA cites for not accepting NTSB recommendations have fostered the impression that the NTSB wants safer airplanes and the FAA does not.
WHAT’S A LIFE WORTH?
The FAA
is required by a series of executive orders dating to the Carter
administration to determine the cost of any rule changes it makes. Some
rule proposals must go through 17 different types of cost-benefit
analyses.
These
calculations are done both by the FAA’s own economists and by outside
agencies, such as the Office of Management and Budget. Sometimes these
analyses predict a proposed rule would cost far more than it would save.
The “savings” are often human lives.
The NTSB does not consider the costs of its recommendations.
“We’ll develop recommendations on what’s realistic, not what’s economic,” said Mike Benson, a spokesman for the board.
John
Rodgers, head of the FAA’s cost-benefit office, defends the practice as
a decision-making tool. “All that you try to do is do something a
little more orderly, a little more comprehensively than `by guess and by
golly.’ ”
Calculating human lives as cost savings (the going rate is $2.7 million per) can seem cold-blooded.
For example, the NTSB recently recommended the FAA require infants in airplanes to ride in child-safety seats. The FAA
has thus far resisted this requirement, raising the convoluted but
interesting objection that doing so could have the effect of actually
killing more children.
This
argument holds that if infants were required to have their own seats,
rather than ride on someone’s lap, the airlines would make them pay.
Selling seats is the business airlines are in, after all. The net result
would be that fewer families could afford to fly and would be forced
from the relative safety of the sky to the bloody battlefield of the
open road, where accident rates are far higher.
Since
so few infants actually die in air crashes, the benefit of having the
safety seats – the number of infants who would not die – is simply too
small to offset the cost, the FAA says.
NTSB
Chairman James Hall, in arguing for the rule change, said it should be
enacted regardless of cost, “One death is too many,” he said.
Secretary of Transportation Pena similarly campaigns for “zero accidents.”
These notions strike many people in aviation as absurd.
“The
cost of attaining zero accidents, even if it were feasible, would be
infinite,” said Stuart Matthews, president of the Flight Safety
Institute, a group that lobbies for air safety. “Then you would have no
more flying.”
“We
probably push for safety improvements that don’t make much sense,” said
Clinton Oster, former director of the Aviation Safety Commission.
“We’re not applying the same standards to other transport modes.”
If
zero is an impossible goal, what is appropriate? The operative
definition, as expressed in the FAA’s design standards, is that an
airplane part or system must be designed in such a way that it will
fail, at most, one time in a billion opportunities.
That’s
the design standard. In practice, airplanes crash about once every 2
million flights. Most of those crashes are due to what the aviation
industry calls “human factors.” Human error, in other words, and human
incompatibility with the airplanes they’re trying to operate – the
so-called man-machine interface.
These
errors occur throughout the system, from pilots in the air to
manufacturing defects back in the factory. They are much more resistant
to cure than many aviation problems. The structural integrity of a
component is measured. The blood-sugar level of a lathe operator in
Boeing’s Everett plant on the morning after the celebration of his daughter’s college graduation is not.
In its place, the FAA
has inspectors assigned to every aviation manufacturer, nominally, at
least, in the world. In practice, many plants are seldom visited. Much
of the inspection is delegated to the large manufacturers. Boeing, for
example, is responsible for the integrity of the parts it buys from
outside suppliers.
The FAA then assigns permanent staff to inspect Boeing’s factories.
BEAT COPS
The Boeing plant in Everett
where the new 777 is being assembled is frequently described as the
world’s largest building, containing 472 million cubic feet. It is large
in another way, as well. It is, after the White House and the State
Department and possibly a few others, arguably the most important
building in the United States’ foreign-policy apparatus.
Boeing annually is the largest single exporter in the U.S.
and, as the market for new airplanes continues to expand abroad, looks
to get ever larger. As such, the company and its output are essential
ingredients in the increasingly contentious new world order of
economics.
The
responsibility for policing this building, as well as the rest of the
Boeing empire, has recently been placed in the very small hands of K.C.
Yanamura, newest and – as a speech-communications major in charge of
assuring the quality of the most complex machines in the world – one of
the least likely industrial beat cops in the country.
Boeing has 110,000 employees. Yanamura has nine FAA inspectors. Three of the nine are in Everett. The FAA thinks this is a fair match. The plant used to have just one.
Manufacturing
inspectors are charged with ensuring that airplanes get built the way
they are designed. Plant inspectors randomly audit different systems,
but largely respond after problems have been discovered on airplanes in
service.
As in much of the rest of the FAA, the gap between FAA staff and the job they are asked to do is immense and is filled by designated Boeing employees.
Being
a designee, one said, is “kind of a career dead-end” but it gives him
freedom to roam throughout the plant. He and other designees say they
seldom have to worry about what would seem to be the biggest potential
problem designees might face – being pressured to cast a kind eye in
their employer’s direction.
“It simply doesn’t happen,” he said.
Inspectors
say they spend a lot of time “jumping around,” troubleshooting
potential problems. The inspectors are authorized to demand corrections
in plant operations, and can seek civil penalties against manufacturers
who violate quality-control procedures.
Such
penalties are infrequent and most tend to be “like warning tickets,”
said a senior executive in the FAA’s Northwest Region Manufacturing
Inspection Office. For example, according to FAA records, in the past 10 years, Boeing’s 737 line in Renton,
which has produced more airliners than any other in the world, has been
cited for 31 infractions. Nineteen were warnings. Twelve resulted in
fines totaling $235,800.
In
the same period, Boeing had revenues of more than $200 billion. For
somebody who earned $50,000 a year, an equivalent fine would be a
nickel.
The
agency can theoretically seek to have a company’s production
certificate revoked. Except for companies that have gone out of
business, this has never happened in the history of the FAA. FAA executives say they would much rather work with, than against, a company.
LUCKY OR GOOD?
At
some point, it has to be considered that rather than being a horrifying
example of government irresponsibility, the FAA’s trust of private
industry to do the government’s job might be a model for the proper
functioning of a regulator in an age of minimalist government.
There are two obvious questions:
Does this really work?
And
if it does, then what use does the agency’s safety staff – 4,500 people
spending a third of a billion dollars a year – serve? Maybe the money
would be better spent for, oh, school lunches.
McSweeney, director of the FAA’s Aircraft Certification Service, regards these questions seriously.
“I
don’t know. I’ve thought about that,” he said, when asked what
contribution the agency makes to safety. “We ought to start with asking
ourselves if we should even exist.
“While
we don’t impact the biggies like Boeing and Douglas, Piper and Beech
and some of those, well, we don’t really impact what they do a lot
because they’re going to do a lot of that stuff whether we’re here or
not, with liability and everything else.
“The
analogy I see is the forest and the trees. You can be a Boeing or a
Douglas, so involved in the details of the forest down at the dirt level
that you miss the fact that right in the middle of that forest is
something else. And the FAA has that ability, because we’re not digging in the details, to kind of step back.”
How far back has the FAA stepped? A considerable distance.
The
GAO has concluded: “The current certification process generally results
in safe aircraft designs because of the efforts of the manufacturers
and expertise of their FAA-designated employees . . ..”
A
congressional staffer who has studied the agency for more than a decade
concludes: “The process seems to have worked. The airplanes are flying
and flying very safely. Either the process has worked or we’re really
incredibly lucky.”
It
is hard to imagine a system that has evolved in such a haphazard,
largely accidental way could actually work. But the results, FAA executives like to say, are indisputable. If, they say, U.S. air carriers had the same accident rate last year as in 1961, there would have been 242 major crashes.
There were three.
The
arithmetic works in the other direction, too. Air traffic is growing
rapidly. If we have the same accident rate in the year 2020 that we have
today, we’ll have a major air accident every week.
Whatever has happened in the last 30 years to improve airplane safety has to happen again in the next. If the FAA is falling out of touch now, can it possibly catch up in an even more complex future?
————————– STEPS TO FAA CERTIFICATION ————————–
1. APPLICATION TO BUILD AIRPLANE – Manufacturer submits to FAA
technical drawings, design description and preliminary schedule for
development and testing of the airplane. This is called the Type Design.
2. CERTIFICATION REQUIREMENTS ESTABLISHED – FAA
determines which sections of the Federal Aviation regulation (FAR), the
basic laws governing aircraft design and operation, apply to the new
plane and how the requirements will be met. Special conditions are
written to cover unique features of the design not addressed by the
regulations. Exemptions to portions of the regulations are sometimes
granted.
3. TESTS BEGIN
– Applicant proposes kinds of tests that should be conducted; FAA reviews.
– Applicant conducts tests of parts and systems; documents them; FAA gives pass/fail grade.
– FAA
does its own tests of critical areas. All parts for the airplane must
be shown to confirm to the Type Design, and the process for making them
must have an approved quality-control system.
– Plane built.
4. FLIGHT TEST – Testing includes performance, flight characteristics, systems, engines and noise. Some tests conducted by FAA pilots, but most done by manufacturer.
5. CERTIFICATES ISSUED
– Type Certificate issued. Certifies the basic Type Design meets FAA standards and lists operating limitations.
–
Production Certificate issued to the factory where the plane is made.
Ensures the factory’s quality-control system meets standards.
–
Airworthiness Certificate given to each airplane that comes off the
production line. Certifies the airplane was built according to the Type
Design in a factory with the Production Certificate.
– Airplane flight manual created.
6. CONTINUING OPERATIONAL SAFETY
– Data gathered from airlines and manufacturer. Service problems, accident and incident data analyzed.
– Advisories issued.
– Design changes mandated by airworthiness directives if safety problems occur.
