This year marks the 30th anniversary of the Hubble Space Telescope. When you see all the great pictures today, it is hard to remember that when it first launched, it was nearly a failure, taking fuzzy pictures. The story of how that problem was fixed while the telescope was whizzing through space is a good one. But there’s another story: how did a $1.5 billion satellite get launched with defective optics? After all, we know space hardware gets tested and retested and, typically, little expense is spared to make sure once a satellite is in orbit, it will work well for a long time.
The problem was with a mirror. You might think mirrors are pretty simple, but it turns out there’s a lot to know about mirrors. For astronomy, you need a first surface mirror which is different from your bathroom mirror which almost certainly reflects off the back of the glass. In addition, the mirrors need a very precise curve to focus light.
Perkin Elmer — a name you don’t hear much anymore outside of the medical field, used to make many kinds of things including computers after they acquired Interdata. However, they also had a history working with optical systems, including for the KH-9 spy satellite. They were in charge of building and testing the Hubble’s mirror. It was off by about 2 micrometers. That doesn’t sound like much, but when you are focusing light — especially light from billions of miles away — it is a lot.
How Bad Was it Really?
Here are two pictures of the M100 galaxy taken by the Hubble. You can probably guess which one was before the correction and which was after.
To give you an idea of how sensitive the mirror is, they use special glass made with titanium that has a very low thermal expansion — nearly zero. The Hubble holds the mirror temperature at 70 F and the mirror only deviates about 1/800,000 of an inch in normal operation. I don’t know how to do the math, but I’ve heard that if the mirror were the diameter of the Earth, the highest deviation on it would be six inches. Amazing. Turns out the mirror isn’t as big as the Earth, but it is over 94 inches across — almost as big as the mirror at the Palomar observatory.
A Tale of Two Mirrors
There were actually two mirrors, both made by Corning. One went to Perkin Elmer. The other went to Kodak who was making a backup mirror just in case. PE had bid $70 million while Kodak wanted $105 million to make the mirror. The cost of the Kodak mirror proposal was, in part, because they wanted to build two and use them to test each other. PE was only going to make one mirror and after NASA requested a backup mirror, they subbed that job back to Kodak who got the second blank for polishing.
I know $70 million sounds like a lot, but for this job, it really wasn’t. That means PE was scrambling to do too much without enough people and without enough time. We’ve all been on programs like that.
A technician shimmed an optical test device — a null corrector — using common washers. A piece of worn paint caused a laser that tracks the distance between the tester and the mirror to be off by just a slight amount. The official story is the tech “failed to report it,” but I would guess he was told not to report it in order to meet schedule. Wherever the blame lies, the error put the test equipment off by 1.3 mm. This led to the 2200 nanometer defect which caused the severe spherical aberration in the instrument.
According to the NASA investigation report:
The spacing of the field lens in the corrector was to have been done by laser measurements off the end of an invar bar. Instead of illuminating the end of the bar, however, the laser in fact was reflected from a worn spot on a black-anodized metal cap placed over the end of the bar to isolate its center (visible through a hole in the cap). The technician who performed the test noted an unexpected gap between the field lens and its supporting structure in the corrector and filled it in with an ordinary metal washer.
Conventional null detectors could not measure the required 10 nanometer flatness required for the mirror, so PE designed and built a special one — the one that was assembled incorrectly. Ironically, two standard null correctors reported the error, but since they were not as accurate as the custom one was supposed to be and — again, under budget and schedule pressure, they decided the certified tester was right when it said the mirror was perfect and didn’t investigate. In fact, the NASA investigation mentions that several engineers on the project had concerns about the mirror’s defects, but they were ignored.
And the Kodak mirror? It used a different test method and was actually reported to be slightly better than the PE mirror. But since PE was the prime contractor, it didn’t want to admit to Kodak having a better mirror, so those reports were not highlighted until the investigation, after it was too late. Had the Hubble flown with the Kodak mirrors, it would have been fine from day one. If you are ever at the Smithsonian Air and Space Museum, it is sitting there as a reminder.
Aftermath
Some experiments could use the telescope the way it was, but others were impossible. So for three years, the telescope did the best it could do until a Shuttle mission brought it a new camera and some corrective lenses. Over time, astronauts added other instruments to the telescope, but they all had to account for the bad mirror. Even the original “eyeglasses” were eventually replaced with better ones.
Perkin Elmer didn’t look good in the investigation. They already had a strained relationship with NASA due to schedule and budget overruns. In the end, they agreed to pay back $15 million to avoid a lawsuit. Hughes, who had since bought the PE optical division, also agreed to pay $10 million. Apparently, NASA felt like Hughes found out about the problem during the acquisition about a year before the telescope’s launch and decided not to bring it up. Hughes, of course, denied that but noted that the $10 million was a “goodwill gesture.”
Lessons Learned
There are so many lessons to draw from this sad story. Make sure your test instruments are giving you the correct answers. Investigate problems even if they seem unlikely to be real. Don’t rely on single tests. But maybe the most important one is to design your tests correctly, and heed their results even under pressure.
If you read the NASA report, one of the problems was that everyone assumed the mirrors would be close to correct. To totally test the mirror would require an even bigger mirror which would be cost-prohibitive — especially since you’d have to test that mirror somehow, too. No one wanted to pay for that test. But even simple tests would have shown the relatively gross error in the mirror that no one thought was possible. After all, the conventional refractive null detectors showed a problem, but either because no one wanted to hear that or the detectors were considered inferior, no one dug into the issue. So whoever designed the test planned failed to imagine that the mirror could be built that far off. But it was.
Perhaps my favorite lesson, though, is the one from what happened after all this. It would have been easy to give up once the flawed mirror became apparent. But, instead, engineers and scientists worked to find a way to fix it. How you handle success tell less about you than how you handle failures.
Of course, another lesson is to be careful with estimates and be transparent with customers. But those aren’t really technical issues.
History is full of cases where engineers knew something was wrong but either failed to act or were suppressed. The Brown’s Ferry nuclear plant comes to mind. In other cases, engineers just fail to pay attention to changes, like with the Hyatt Regency walkway collapse. My advice is don’t be one of those guys!
No comments:
Post a Comment