Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
United States

Alaska Airlines Plane Appears To Have Left Boeing Factory Without Critical Bolts (wsj.com) 89

Regulators put limits on Boeing 737 MAX production; grounded MAX 9 jets have resumed flying after required inspections. From a report: Bolts needed to secure part of an Alaska Airlines jet that blew off in midair appear to have been missing when the plane left Boeing's factory. Boeing and other industry officials increasingly believe the plane maker's employees failed to put back the bolts when they reinstalled a 737 MAX 9 plug door after opening or removing it during production, according to people familiar with the matter. The increasingly likely scenario, according to some of these people, is based partly on an apparent absence of markings on the Alaska door plug itself that would suggest bolts were in place when it blew off the jet around 16,000 feet over Oregon on Jan. 5.

They also pointed to paperwork and process lapses at Boeing's Renton, Wash., factory related to the company's work on the plug door. The National Transportation Safety Board has been conducting metallurgical analysis of the plug door but hasn't released the results of the testing. Laboratory tests might show whether the bolts were in place or not there at all. An update in the NTSB probe is expected as soon as this week. New evidence could later emerge before accident investigators reach final conclusions. It couldn't be determined how many people were involved with work on the plug door at Boeing's 737 factory. Supplier Spirit AeroSystems delivered the 737 fuselage to Boeing's factory with the door plug installed. The plug door itself was constructed in Spirit's Malaysian factory, while the fuselage was assembled in Wichita, Kan. Boeing opened or removed the door plug after the 737 MAX 9 jet's fuselage arrived at the plane maker's Renton, Wash., factory for final assembly, The Wall Street Journal and other news outlets have reported.

This discussion has been archived. No new comments can be posted.

Alaska Airlines Plane Appears To Have Left Boeing Factory Without Critical Bolts

Comments Filter:
  • Never buy a 737 MAX that was built on a Monday. [jalopnik.com]
  • Okay ... (Score:4, Interesting)

    by fahrbot-bot ( 874524 ) on Monday January 29, 2024 @03:19PM (#64198382)

    ... the plane maker's employees failed to put back the bolts when they reinstalled a 737 MAX 9 plug door after opening or removing it during production, ...

    Assuming this is accurate, how does a worker install this door plug, that requires several bolts to secure, and fail to also install the bolts? Smoke break? Shift change? If you're going to start a procedure, finish the procedure or leave a clear indication that it hasn't been finished, like a BIG red tag. On the other hand, kudos to the door plug design for staying on as long as it did w/o any bolts.

    • by ceoyoyo ( 59147 )

      You close the door when you're done using it because it doesn't seem like a good idea to leave it hanging open, but you don't bolt it because you expect the next shift to need to use it. The next shift manages with the regular door only, and nobody remembers the door wasn't properly reinstalled.

      It's highly unlikely it was a mistake as dumb as wandering off and forgetting to install the bolts. Much more likely it was a subtle failure in the process somewhere.

      • You close the door when you're done using it because it doesn't seem like a good idea to leave it hanging open, but you don't bolt it because you expect the next shift to need to use it. The next shift manages with the regular door only, and nobody remembers the door wasn't properly reinstalled.

        It's highly unlikely it was a mistake as dumb as wandering off and forgetting to install the bolts. Much more likely it was a subtle failure in the process somewhere.

        I can see that logic, though I don't think this "plug" is something that opens/closes like a regular door. It seems to be something that bolts on where a (emergency) door could be, but isn't for this particular configuration. If so, it's something that is either in place or not. In this case it seems it was in place but not bolted down.

        • If it opened/closed like a regular door, you wouldn't have to unbolt it. I would imagine that, in this type of situation, you unbolt the door and set it aside, do your work, then bolt it back in. Bur maybe there wasn't a good place for it so somebody just set it back in the space rather than putting it away from the plane. And then forgot. I'm purely speculating here.
        • Re:Okay ... (Score:5, Insightful)

          by MachineShedFred ( 621896 ) on Monday January 29, 2024 @06:11PM (#64199044) Journal

          The door plug can be removed in the factory in order to give another opening for loading and installing cabin equipment like seats, overhead bins, etc.

          I could easily see them taking them out in the factory in order to have as much access to the interior as possible for personnel and material. And I could just as easily see that someone put the plug back in place to get it out of the way, and then a shift change happens and someone else doesn't realize the bolts aren't there because there wasn't a super obvious visual indicator applied (like a big red tag on the bolt holes, for example) and re-assembles the interior over the bolt-less plug.

          It's a miracle that thing didn't blow out until it did if the bolts just weren't present and the inspections were just not performed.

        • by ceoyoyo ( 59147 )

          It's got hinges on the bottom.

          I don't know how those hinges work, and I don't know whether Boeing's assembly procedures involve removing the door entirely or just opening it, but it's a possibility. Even if it were removed entirely, it's quite possible that some procedure involves it being reinstalled temporarily.

          There was speculation originally that Spirit temporarily installed the doors for transport and then Boeing ended up not needing to remove them. That seems not to be the case, but the same sort of t

      • given a few moments thought and having worked in places like this, I feel that the most likely problems are a lack of QC personnel (my experience has been that once they create a QA department, they fire all the QC personnel) and at the same time are constantly on the backs of the workers to SPEED UP and not spend so much time on the processes that ensure safety. For example, the door plug could have had 8 bolts and they only applied 4 nuts, hence those nuts/bolts were overstressed for the entire lifetime o
        • by ceoyoyo ( 59147 )

          Boeing apparently reduced the number of their QC inspectors in favour of more automated systems. There are rumours that they have documentation showing the bolts being removed, but never added. That would suggest that their electronic QC system was lacking a check.

          Would a human inspector have caught it? Maybe. Human inspectors still have to be assigned to inspect things, so procedural oversights can compromise them too.

          • most businesses appear to hate funding any level of QC, so, no surprise that this was missed, however it was.
          • in ancient times, I was a QC inspector on a system being sold to the Army, and in many instances our technicians were just trying to hurry up and move on because lunch is coming or end of day is coming. They would happily try to BS us (if we didn't know enough about the systems in question) and claim systems passed when that actually hadn't even been tested as yet. I discovered that many of my fellow inspectors were (like myself) very much in favor of knowing the system better than the technicians did so th
    • Many assembly lines use computer vision to verify quality control as a product moves between stations.

      This can be facilitated with color coding. For instance, the bolts can have orange heads, so the CV cameras can identify them more reliably.

      • Many assembly lines use computer vision to verify quality control as a product moves between stations.

        This can be facilitated with color coding. For instance, the bolts can have orange heads, so the CV cameras can identify them more reliably.

        Sure, but some actual person must have known he/she hadn't installed the bolts.

        • Sure, but some actual person must have known he/she hadn't installed the bolts.

          Expecting humans to perform perfectly without any quality control is not an acceptable practice in aircraft assembly.

          There should never be a single point of failure.

          • Sure, but some actual person must have known he/she hadn't installed the bolts.

            Expecting humans to perform perfectly without any quality control is not an acceptable practice in aircraft assembly.

            There should never be a single point of failure.

            Agreed, just noting that regardless of any subsequent QA, an actual person was in the loop somewhere and didn't install the bolts.

            • But at the same time, there should have been a SECOND person to verify work was done properly.

              Part of the huge cost of one of these airliners is the knowledge that there is proper testing and inspection at all phases. This is not Bob's Discount Airplanes! If it is not acceptable for a bicycle manufacturer to have these flaws then it should go without saying that it is not acceptable for the manufacture of a product that costs 100 million USD.

              An example is on Toyota assembly lines, when any worker can pull t

      • Thanks for the academic lesson. That does nothing to address the obvious deficiencies in their process. Also, us humans have world class object detection sensors built in.
        • Re: (Score:2, Insightful)

          by Anonymous Coward
          This is a minimum of two failures.
          1.the bolts were not installed.
          2. The bolts weren't checked.

          I started being pedantic to razz paul_engr, but then realized maybe Boeing needs to be notified of this.
          • by BranMan ( 29917 )

            This is a minimum of THREE failures.
            1. The bolts were not installed.
            2. The bolts weren't checked.
            3. The process to sign off on delivering the plane did not discover the bolts weren't checked.

        • Also, us humans have world class object detection sensors built in.

          The wetware sensors only work when we're paying attention and obviously failed in this case.

          Cameras don't get distracted.

    • How it was forgotten will be investigated. One possible explanation is breakdown in tasks where the worker who fixed the door put it back on the frame but it was not their job to install the door. Then it was a separate worker put the interior panels on the plane with no one installing the door. Another troubling thing that came up is how this situation passed inspection. It appears that the door removal was put in the system as a door "opening" which does not trigger an inspection or the installation worke
    • Re:Okay ... (Score:5, Informative)

      by Richard_at_work ( 517087 ) on Monday January 29, 2024 @03:44PM (#64198496)

      I've worked tangentially to some of these sort of critical installation processes, and in all cases Ive been exposed to, the job uses "installation packages" for all consumables.

      In other words, your task is "install door plug", you go grab the package for that task - that package comes with all the nuts, bolts, wires and other things involved in that specific single job. Sometimes even tools. You start with the right amount of items, and by the end of it you should be left with a defined quantity of specific things - and its all laid out in a tray. You sign the tray out, and you sign it back in.

      If the task goes over a shift change, then theres a process for transferring the tray and its contents.

      I know that this process is used in some aspects of aircraft manufacture, so if its being used here then it should be obvious to whomever is taking over that the job is incomplete - and an audit of the trays at the end of the day or completion of the airframe should show if the tray is missing or has additional items.

      So the only way in which this can really screw up is through deliberate action.

      • Thanks, that was interesting and informative about the (presumed) process. The last bit about "deliberative" was what was getting at ...

      • Re: (Score:1, Flamebait)

        by Narcocide ( 102829 )

        You're right, and I know for a fact it was deliberate sabotage, and I know Satanism was involved, and I know this because cult members gloated about it to me directly, years in advance of this actual incident.

      • by mjwx ( 966435 )

        I've worked tangentially to some of these sort of critical installation processes, and in all cases Ive been exposed to, the job uses "installation packages" for all consumables.

        In other words, your task is "install door plug", you go grab the package for that task - that package comes with all the nuts, bolts, wires and other things involved in that specific single job. Sometimes even tools. You start with the right amount of items, and by the end of it you should be left with a defined quantity of specific things - and its all laid out in a tray. You sign the tray out, and you sign it back in.

        If the task goes over a shift change, then theres a process for transferring the tray and its contents.

        I know that this process is used in some aspects of aircraft manufacture, so if its being used here then it should be obvious to whomever is taking over that the job is incomplete - and an audit of the trays at the end of the day or completion of the airframe should show if the tray is missing or has additional items.

        So the only way in which this can really screw up is through deliberate action.

        This, as part of your job you're also meant to check if you've all the right parts in the correct quantities before starting the job.

        However whilst procedures will be clear on how to avoid simple mistakes, I think the issue at Boeing is that procedure isn't being followed. Pressure from management to get more product out the door often leads to corners being cut, especially if management aren't connected to the manufacturing and/or simply don't care how it's done as long as it'd done. This is really, rea

    • What I have gleaned from other accounts is that the process was tagged as "opening the door" rather than "removing the plug." With a door there are no bolts, so there was no QA process to inspect the bolts nor an automatically generated checklist to re-install the bolts. In no way is that a satisfactory excuse for the failure, but it is likely about what happened. The plug and interior trim were likely reinstalled without following any approved procedures.

    • They decided they should be one of the very proud employees working from home.
    • The "plug door" is just a normal door without a handle. It attaches and closes just fine with no bolts. The bolts are to secure it in the closed position, but that is where it normally wants to be anyway. Missing the bolts means it will stay closed, until the pressure differential finally works it open.

    • Probably separation of work items in different union roles. There are enough old Telco people around here to back me up on this theory.
    • by gweihir ( 88907 )

      What about "clueless idiots" hired by "greedy and even more clueless management"? Also note that there should have been an inspection by a separate expert directly after installation and one later for final certification by yet another expert. Seems both these inspections were missing as well, doubtless to "improve profits".

      Well, if you want to kill a large number of people, you really have to arrange a series of fuck-ups, so it makes sense.

    • by tlhIngan ( 30335 )

      Assuming this is accurate, how does a worker install this door plug, that requires several bolts to secure, and fail to also install the bolts? Smoke break? Shift change? If you're going to start a procedure, finish the procedure or leave a clear indication that it hasn't been finished, like a BIG red tag. On the other hand, kudos to the door plug design for staying on as long as it did w/o any bolts.

      How about "Get this task done in 1 hour"?

      There are several testimonials that workers at both Spirit and Boei

    • by Aczlan ( 636310 )

      ... the plane maker's employees failed to put back the bolts when they reinstalled a 737 MAX 9 plug door after opening or removing it during production, ...

      Assuming this is accurate, how does a worker install this door plug, that requires several bolts to secure, and fail to also install the bolts? Smoke break? Shift change? If you're going to start a procedure, finish the procedure or leave a clear indication that it hasn't been finished, like a BIG red tag. On the other hand, kudos to the door plug design for staying on as long as it did w/o any bolts.

      From what I read (from someone who claimed to be a employee) its because they found a way to avoid fully removing the door, so they didn't have to do that checklist item. Source (worth exactly what you paid for it): https://leehamnews.com/2024/01... [leehamnews.com]

      Current Boeing employee here – I will save you waiting two years for the NTSB report to come out and give it to you for free: the reason the door blew off is stated in black and white in Boeings own records. It is also very, very stupid and speaks volumes about the quality culture at certain portions of the business.

      A couple of things to cover before we begin:

      Q1) Why should we believe you?
      A) You shouldn’t, I’m some random throwaway account, do your own due diligence. Others who work at Boeing can verify what I say is true, but all I ask is you consider the following based on its own merits.

      Q2) Why are you doing this?
      A) Because there are many cultures at Boeing, and while the executive culture may be throughly compromised since we were bought by McD, there are many other people who still push for a quality product with cutting edge design. My hope is that this is the wake up call that finally forces the Board to take decisive action, and remove the executives that are resisting the necessary cultural changes to return to a company that values safety and quality above schedule.

      With that out of the way why did the left hand (LH) mid-exit door plug blow off of the 737-9 registered as N704AL? Simple- as has been covered in a number of articles and videos across aviation channels, there are 4 bolts that prevent the mid-exit door plug from sliding up off of the door stop fittings that take the actual pressurization loads in flight, and these 4 bolts were not installed when Boeing delivered the airplane, our own records reflect this.

      The mid-exit doors on a 737-9 of both the regular and plug variety come from Spirit already installed in what is supposed to be the final configuration and in the Renton factory, there is a job for the doors team to verify this “final” install and rigging meets drawing requirements. In a healthy production system, this would be a “belt and suspenders” sort of check, but the 737 production system is quite far from healthy, its a rambling, shambling, disaster waiting to happen. As a result, this check job that should find minimal defects has in the past 365 calendar days recorded 392 nonconforming findings on 737 mid fuselage door installations (so both actual doors for the high density configs, and plugs like the one that blew out). That is a hideously high and very alarming number, and if our quality system on 737 was healthy, it would have stopped the line and driven the issue back to supplier after the first few instances. Obviously, this did not happen. Now, on the incident aircraft this check job was completed on 31 August 2023, and did turn up discrepancies, but on the RH side door, not the LH that actually failed. I could blame the team for missing certain details, but given the enormous volume of defects they were already finding and fixing, it was inevitable something would slip through- and on the incident aircraft something did. I know what you are thinking at this point, but grab some popcorn because there is a plot twist coming up.

      The next day on 1 September 2023 a different team (remember 737s flow through the factory quite quickly, 24 hours completely changes who is working on the plane) wrote up a finding for damaged and improperly installed rivets on the LH mid-exit door of the incident aircraft.

      A brief aside to explain two of the record systems Boeing uses in production. The first is a program called CMES which stands for something boring and unimportant but what is important is that CMES is the sole authoritative repository for airplane build records (except on 787 which uses a different program). If a build record in CMES says something was built, inspected, and stamped in accordance with the drawing, then the airplane damn well better be per drawing. The second is a program called SAT, which also stands for something boring and unimportant but what is important is that SAT is *not* an authoritative records system, its a bullentin board where various things affecting the airplane build get posted about and updated with resolutions. You can think of it sort of like a idiots version of Slack or something. Wise readers will already be shuddering and wondering how many consultants were involved, because, yes SAT is a *management visibilty tool*. Like any good management visibilty tool, SAT can generate metrics, lots of metrics, and oh God do Boeing managers love their metrics. As a result, SAT postings are the primary topic of discussion at most daily status meetings, and the whole system is perceived as being extremely important despite, I reiterate, it holding no actual authority at all.

      We now return to our incident aircraft, which was written up for having defective rivets on the LH mid-exit door. Now as is standard practice kn Renton (but not to my knowledge in Everett on wide bodies) this write-up happened in two forms, one in CMES, which is the correct venue, and once in SAT to “coordinate the response” but really as a behind-covering measure so the manager of the team that wrote it can show his boss he’s shoved the problem onto someone else. Because there are so many problems with the Spirit build in the 737, Spirit has teams on site in Renton performing warranty work for all of their shoddy quality, and this SAT promptly gets shunted into their queue as a warranty item. Lots of bickering ensues in the SAT messages, and it takes a bit for Spirit to get to the work package. Once they have finished, they send it back to a Boeing QA for final acceptance, but then Malicious Stupid Happens! The Boeing QA writes another record in CMES (again, the correct venue) stating (with pictures) that Spirit has not actually reworked the discrepant rivets, they *just painted over the defects*. In Boeing production speak, this is a “process failure”. For an A&P mechanic at an airline, this would be called “federal crime”.

      Presented with evidence of their malfeasance, Spirit reopens the package and admits that not only did they not rework the rivets properly, there is a damaged pressure seal they need to replace (who damaged it, and when it was damaged is not clear to me). The big deal with this seal, at least according to frantic SAT postings, is the part is not on hand, and will need to be ordered, which is going to impact schedule, and (reading between the lines here) Management is Not Happy. 1/2
      2/2
      However, more critical for purposes of the accident investigation, the pressure seal is unsurprisingly sandwiched between the plug and the fuselage, and you cannot replace it without opening the door plug to gain access. All of this conversation is documented in increasingly aggressive posts in the SAT, but finally we get to the damning entry which reads something along the lines of “coordinating with the doors team to determine if the door will have to be removed entirely, or just opened. If it is removed then a Removal will have to be written.” Note: a Removal is a type of record in CMES that requires formal sign off from QA that the airplane been restored to drawing requirements.

      If you have been paying attention to this situation closely, you may be able to spot the critical error: regardless of whether the door is simply opened or removed entirely, the 4 retaining bolts that keep it from sliding off of the door stops have to be pulled out. A removal should be written in either case for QA to verify install, but as it turns out, someone (exactly who will be a fun question for investigators) decides that the door only needs to be opened, and no formal Removal is generated in CMES (the reason for which is unclear, and a major process failure). Therefore, in the official build records of the airplane, a pressure seal that cannot be accessed without opening the door (and thereby removing retaining bolts) is documented as being replaced, but the door is never officially opened and thus no QA inspection is required.
      This entire sequence is documented in the SAT, and the nonconformance records in CMES address the damaged rivets and pressure seal, but at no point is the verification job reopened, or is any record of removed retention bolts created, despite it this being a physical impossibility. Finally with Spirit completing their work to Boeing QAs satisfaction, the two rivet-related records in CMES are stamped complete, and the SAT closed on 19 September 2023. No record or comment regarding the retention bolts is made.

      I told you it was stupid.

      So, where are the bolts? Probably sitting forgotten and unlabeled (because there is no formal record number to label them with) on a work-in-progress bench, unless someone already tossed them in the scrap bin to tidy up.

      There’s lots more to be said about the culture that enabled this to happened, but thats the basic details of what happened, the NTSB report will say it in more elegant terms in a few years.

      Aaron Z

    • > how does a worker install this door plug, that requires several bolts to secure, and fail to also install the bolts?

      From a technical standpoint (as I understand it), these bolts didn't hold the plug in place as such. They're more like a way to prevent the plug from sliding out after it's been "clipped" into place. You could run at that plug (without bolts) and it wouldn't budge - so unless you know about the bolts, you could conceivably not know that *something* more needed to be done.

      From a process po

  • My understanding is there was a discussion as to whether the plug should be opened or removed. If it was just opened it didn't need additional QA, if it was removed it did. To save time/money someone who should be held responsible decided the plug was opened, not removed.

    That in a nutshell is why Boeing sucks today, and why it's going to take decades to get their reputation back.
  • look at the case of Day Davis safety at the factory is pushed a away to keep the line moving and when you have an rush mandate things get skipped

  • by takochan ( 470955 ) on Monday January 29, 2024 @03:48PM (#64198506)

    Here is the original leak by the original Boeing employee whistleblower. It explains exactly what happened at the factory in detail for this particular plane / how the mishap exactly occurred point by point.

    https://leehamnews.com/2024/01... [leehamnews.com]

    • I read the  whistleblower- link. WOWser Boeing / Spirit  sure is in some trouble. Manufacture did everything possible for a known aircraft fault ... except fix it  !
    • Thank you, this is good stuff.

      One point puzzles me where he says "The mid-exit doors on a 737-9 of both the regular and plug variety come from Spirit already installed in what is supposed to be the final configuration"

      My understanding is those doors are in place but bolts are not tightened. They are not "flight-ready". The assumption is the team want to remove some or all the doors temporarily to ease the task of bringing in all the internals to the fuselage such as seats.
      So it is the job of the final team

      • by fgouget ( 925644 )

        My understanding is those doors are in place but bolts are not tightened. They are not "flight-ready".

        What is your source for this?

  • I want to support Boeing. I really do. But they’re makin’ it real difficult.

    I’d still fly any boeing jet that’s cleared to fly, cause I’m more likely to die of a stroke walking from the departure area to the gate than die in a crash. But still
  • by DrMrLordX ( 559371 ) on Monday January 29, 2024 @04:12PM (#64198648)

    Boeing should have a process in place to document every torque electronically with networked hand tools. The unit shouldn't be cleared for subsequent steps of production without a successful torque in their logs. Failure to complete these torques should only be overridden by a superior who documents their entire reason for clearing the failed torque (typically, electronic torques only allow a few attempts with the designated hand tool before failure to avoid stripping out holes due to crossthreads or other problems). That's the standard with a lot of automotive manufacturers, so that if there is a bad bolt somewhere, you can figure out who installed the bolt on which day and who may have overridden the torque requirement (where applicable). Instant accountability.

    Plane manufacturing is even more difficult and potentially deadly for customers if it's done wrong. The need for proper documentation of torque logs is there. Does Boeing not do this?

  • Boeing and their supplier Spirit Aerospace were using different quality management systems. That particular door failed quality control at Spirit and they need to fix it on the Boeing side. However, there's direct translation of the issue across the systems and they use the wrong procedure to document the fact that they have to open the door to fix a seal issue, which introduced other problem.

    https://www.youtube.com/watch?... [youtube.com]

  • by cathector ( 972646 ) on Monday January 29, 2024 @04:49PM (#64198800)

    diagram:
    https://images.seattletimes.co... [seattletimes.com]

    article:
    https://www.seattletimes.com/b... [seattletimes.com]

    the summary of the article is that someone claiming to be a current boeing employee posted a whistleblowey account on a forum which says that boeing's own internal documentation system points strongly to the bolts (which are not the primary load-bearing item holding the door plug in place) having never been installed.

  • The reason that this door plug was worked on at all in Renton was because it came from Spirit with the gasket"seal. improperly installed.

  • "It couldn't be determined how many people were involved with work on the plug door at Boeing's 737 factory." sounds like they should not be trusted with working on a coffee-maker, much less an airplane. There is really no hope f fixing that utterly defective company now.

  • Can't have loose bolts, if there are no bolts...

  • Dare we assume there was some incompetence at Boeing?
    Every action has a cause.
    In some cases multiple causes.
    So too here.
    Under these circumstances
    Can we feel safe?
    Knowing what we know.
    Safety first please.

  • The root cause of all this mess is the way that "do it fast" is more important than "do it right" at Boeing and its contractors.

    The FAA needs to stop giving Boeing a pass on this and force them to stop prioritizing speed and cost over doing things properly (and having proper QA in place to verify that things are done properly)

"Confound these ancestors.... They've stolen our best ideas!" - Ben Jonson

Working...