Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
United States Technology

Flights Grounded Across US After FAA Failure (nytimes.com) 147

Posted by msmash from the breaking-news dept.
A Federal Aviation Administration system failure caused some flights across the United States to be grounded, the agency said early Wednesday. From a report: The full extent of the delays was not immediately known, but the delays were spread across several airlines. More than 700 flights within, into and out of the United States had been delayed on Wednesday, and more than 90 were canceled, according to FlightAware, a flight tracking company. On social media, would-be passengers across the United States said their flights had been delayed, some reporting that their pilots or airline representatives had blamed the F.A.A. technical problem. "The F.A.A. is working to restore its Notice to Air Missions System," the agency said. "We are performing final validation checks and reloading the system now. Operations across the National Airspace System are affected." Several airports, including in Philadelphia, Tampa, Fla., and Austin, Texas, advised passengers to check with their airlines for the latest information.
This discussion has been archived. No new comments can be posted.

Flights Grounded Across US After FAA Failure More

Flights Grounded Across US After FAA Failure

Comments Filter:

  • Over 700 flights grounded due to a Federal system. Has this ever happened before? Was such a critical system not built with proper redundancy? Why?

    Sadly, when mass events happen in the US, we now tend to look for the actual reason for outage, not the one being reported by the media.

    • Re: (Score:1)

      by gtall ( 79522 )

      "Sadly, when mass events happen in the US, we now tend to look for the actual reason for outage, not the one being reported by the media."

      Only by dystopian people like yourself.

      • He's got a point. If the issue was found to be a specific company's software (like IBM or Microsoft), is it in CNN/Fox/MSNBC's best interest to report that? It might impact their ad revenue from those software providers in the future.

        If they keep the details of the outage more generic, their content becomes more advertiser-friendly and they don't have to worry about confusing their viewers.

        I doubt that your typical CNN/Fox/MSNBC viewer is an IT expert anyway. If you ARE an IT person who wants to know what R

    • Well, the actual reason for this is most often also the one that gets reported. But that's usually not enough for the sensation hounds or the conspiracy loons, so why bother with boring reality when we can up much more exciting and inciting reasons?

    • Re:History of this? (Score:4, Interesting)

      by serafean ( 4896143 ) on Wednesday January 11, 2023 @08:34AM (#63198758)

      I'm amazed at how not resilient we are as a society. A luxury item fails (in this case flights), and in all our richness, we don't have the ability to "suck it up" and wait for a day. Getting back up from this in a few hours is the actual miracle.

      Yes, there is less than 1% of affected who would actually be threatened by this. But that is what actually critical fallbacks are for (such as diesel generators in hospitals for electricity failures)

      > actual reason for outage
      The front fell off... That's not very typical
      https://www.youtube.com/watch?... [youtube.com]

      • Re: (Score:2, Interesting)

        by 140Mandak262Jamuna ( 970587 )
        Backups, redundancies, additional capacity etc cost money. Republican party has been openly hostile to all government agencies, and FAA in particular since the days of Reagan. Regan busted the unionization attempt by FAA and fired them all. The rift caused by that action has not healed yet. Competent people dont want to work for FAA. Govt does not fund it. The archaic obsolete rules help the existing players and creates a barrier to entry.

        FAA's problems developed over time, slow degradation of quality and

      • we don't have the ability to "suck it up" and wait for a day

        Uh, isn't that exactly what the passengers of the cancelled flights did? Society didn't crumble. People grumbled about the inconvenience, waited until the problem was fixed, and then went on about their business when the airline rebooked them.

        • My comment was more of a knee-jerk reaction to the first post

          > Over 700 flights grounded due to a Federal system. Has this ever happened before? Was such a critical system not built with proper redundancy? Why?

          to basically say "shit happens" .

          So yes, "suck it up" is what more or less what happened, but the general reaction is one of "this should never have happened" instead of "it's awesome that this doesn't happen too often, let's see what we got wrong" . And I will concede that within the FAA, the sec

          • the general reaction is one of "this should never have happened" instead of "it's awesome that this doesn't happen too often, let's see what we got wrong" . And I will concede that within the FAA, the second approach is probably what will happen.

            Both of those things could be true. We'll have to wait to see what the actual problem was, assuming we ever find out, before we can decide how much of column A, and how much of B.

    • So far, the media isn't reporting a reason of any kind, they are saying they don't yet know what caused the outage.

      YOU are jumping to conclusions How do you know the critical system wasn't built with proper redundancy? That statement has yet to be substantiated. The fact that it makes news, that it's the first time this has ever happened in the history of US aviation, says to me that it is practically rock solid. How much of YOUR software has continued to function without incident, for decades?

    • Multiple software system failures by the private airlines every year causing hundreds of not thousands of delays.

      "Efficiency working as intended!"

      By their own admission can't think of another failure coming from the FAA besides this one.

      "Welcome to your government dystopia"

    • Somebody was using the computers to mine Bitcoins.

    • The dumbest developer I ever worked with had a side gig writing software for airlines. Apparently the bar isn't set very high. I don't know who he contracted to, but whenever these outages happen, I'm sure he's responsible.

    • Re: (Score:2)

      by shanen ( 462549 )

      Over 700 flights grounded due to a Federal system. Has this ever happened before? Was such a critical system not built with proper redundancy? Why?

      Sadly, when mass events happen in the US, we now tend to look for the actual reason for outage, not the one being reported by the media.

      On what basis are the trolls and sock puppets with mod points trying to censor your comment? Not the strongest FP, but it's relevant and even asks a reasonable question. What button did you push? Or do you have a reputation? (The handle sounds vaguely familiar, but...)

      Related current reading is The Glass Cage by Nicholas Carr. (Same guy who did The Shallows about our growing Internet-driven mental afflictions.) The titular cage is a reference to fly-by-wire jets where the pilots are surrounded by glass

    • There was redundancy. The backup system started up as it was supposed to. The only problem was that the backup system had corrupted data, so was unusable. More than likely no one had tested the backup system for years.

      • I'll try to keep from punching my screen, by pretending that there are otherwise competent people who've never heard of database replication before.

  • Cyberattack (Score:2, Insightful)

    by dknj ( 441802 )

    When "no one can explain the problem" itâ(TM)s probably a cyberattack

    • Re:Cyberattack (Score:5, Insightful)

      by SpzToid ( 869795 ) on Wednesday January 11, 2023 @07:23AM (#63198612)
      Don't assume malice when incompetence is an adequate explanation. --Hanlon's Razor

      Time will tell.

    • This reminds me of a programmer I once worked with, who, when a problem was found in the software he wrote, he always suggested it was likely a "Microsoft bug." Upon further inspection, it always turned out to be just a mundane bug that he himself had introduced.

      If it was a cyberattack, that claim can only be made if there is actual evidence of a cyberattack.

    • Except we're talking about the FAA here, so the problem is probably caused by a system running on an old IBM mainframe coded in COBOL, and "no one can explain the problem" is likely code for "anyone who still knows how it works is retired or dead".

      • Flight control software is written in ADA and is relatively new-ish in govt terms (within last 15 years if I remember correctly).
  • "Notice to Airmen", or NOTAM. Not Notice to Air Missions. I am a private pilot BTW.

    • Re: (Score:2, Insightful)

      by Anonymous Coward

      Probably deemed sexist or something, and got renamed. That will also turn out to be the root cause of this failure.

      Though why flights get canceled because pilots can't get the latest NOTAMs is a little... weird. This didn't use to be a problem back before computerisation. But it does mean the system is now mission critical and perhaps wasn't marked as such. Then again, looks like the FAA let themselves be captured by Boeing (see 737MAX et al) so what else is now broken and unsafe within that agency?

      • My understanding (note I'm not a pilot, and don't play one on TV, etc., etc.):

        You're preparing for takeoff heading to an airport 1000km away. As far as you know the weather is fine. Except that just before you take off, a sudden thunderstorm develops at your destination, out of nowhere, and a lightning strike knocks out the ILS (instrument landing system). There are rapidly shifting crosswinds, and visibility is crap.

        Taken together, these problems mean it won't be safe to land there. You'll have to dive

      • Weather information is indeed part of your preflight and you can get updated information during the flight, but NOTAM's also have tons of crucial information having nothing to do with weather like: Flashing red beacon light on this particular radio tower are not working. (don't crash into it because you can't see it) Runway 25 is out of service Taxiway XX is out of service ILS-25L approach is not available ATIS (automated terminal information system) rain detector is not working There is a 30nm wide TFR (te

    • Re: I thought it was called... (Score:5, Informative)

      by JoeRobe ( 207552 ) on Wednesday January 11, 2023 @07:50AM (#63198666) Homepage

      FAA calls it notice to air missions:

      https://www.faa.gov/about/init... [faa.gov]

      "Changed the acronym NOTAM from Notice to Airmen to the more applicable term Notice to Air Missions, which is inclusive of all aviators and missions"

      • Re: (Score:2, Insightful)

        by poptopdrop ( 6713596 )

        There is a certain pathetic sort of person who gets job satisfaction from doing stupid shit like this.

        • Re: (Score:3, Insightful)

          by mjwx ( 966435 )

          There is a certain pathetic sort of person who gets job satisfaction from doing stupid shit like this.

          No, just the certain pathetic sort of person who gets upset about it.

        • Re: (Score:2)

          by tlhIngan ( 30335 )

          There is a certain pathetic sort of person who gets job satisfaction from doing stupid shit like this.

          Well, given there is a current push to attract more women to aviation (they only make up around 18% of the pilot population), getting rid of loaded terms is one way.

          And the problem is, NOTAM is a word used internationally. so you're not going to change the acronym used - everyone knows what it is when you request the NOTAMs for the route (and yes, that plural does not make sense if you expand it out).

          Howeve

          • Re: (Score:2, Insightful)

            by Anonymous Coward

            There is a certain pathetic sort of person who gets job satisfaction from doing stupid shit like this.

            Those people are called "bureaucrats".

            Well, given there is a current push to attract more women to aviation (they only make up around 18% of the pilot population), getting rid of loaded terms is one way.

            No, it's not, for the kind of woman that goes "oh noez it means Notices To Aliens from Mars, that's not me so I'ma powder my nose instead of read these NOTAMs", isn't the kind of woman that is safe in the cockpit. Same goes for men, or actual aliens from mars, for that matter.

            Really, this sexism is at least as bad as any and all lingering sexism it wants to combat. Worse really, for it is ideologically motivated, and not grounded in reality. Where, for example, is the

          • "Well, given there is a current push to attract more women to aviation"

            Only among pathetic woke people.

            Men and women are different and like doing different things.
            Stop trying to homogenize the world.

            And if you want to do some actual good, start a push to prevent male suicides or prevent men dying violently.
            Stuff that matters.

          • "Hell, I used to think NATO stood for North American Treaty Organization,"

            And you thought wrong. NATO has never stood for "North American Treaty Organization".

            " and somewhere it turned into North Atlantic Treaty Organization"

            It didn't "turn into" North Atlantic Treaty Organization", that's what has always been, from when it was first formed in 1949.

          • Female Pilot in Training: "I was thinking about becoming a pilot, because I like flying. However, they still use Notice to Airmen, and I cannot abide by that indecency". Said no one ever. Also, wasn't it always North Atlantic Treaty Organization? Perhaps you're getting it mixed up with NORAD?

        • There is a certain pathetic sort of person who gets job satisfaction from doing stupid shit like this.

          Yeah, the kind of person that doesn't forget to reboot the mission critical Windows 98 system ever 50 days.

    • Re: (Score:2, Funny)

      by bill_mcgonigle ( 4333 ) *

      A husband was told, in a sultry voice, "tonight I want you to whisper, in my ear, something dirty."

      "OK," he complied, "the dishes in the sink need washing."

      The other husband wasn't expecting that at all.

    • Re: (Score:2)

      by cstacy ( 534252 )

      "Notice to Airmen", or NOTAM. Not Notice to Air Missions. I am a private pilot BTW.

      Apparently you missed the NOTAM a few years ago when the FAA decided to rename it. Because "Airman" was sexist. I kid you not.

  • Maybe the software is Chinese! /s

  • Possible Ransomware Attack (Score:4, Interesting)

    by jerryjnormandin ( 1942378 ) on Wednesday January 11, 2023 @08:21AM (#63198740)
    Many Colleges, Universities, and Police Stations have been getting hit by Ransomware Attacks. It appears to be "Clop". I know of a University that stopped it thanks to a Honeypot deployment, all infected computers have been removed from the network before any servers were infected. They are scanning the hardware now. Clop can re-encrypt itself, resulting in a unique signature, the only way to find it is to look for data that is not supposed to be there. So scanning takes longer. To prevent paying Ransomware make sure all your servers with data that you care about are behind multiple firewalls. All Workstations that work with data you care about are behind multiple firewalls, and deploy honeypots throughout your entire infrastructure. For connectivity to the outside world I would recommend multiple firewalls and a proxy. Businesses should have a proxy that denies all outside access and allows access to approved sites through a whitelist. Anything networks that carry company sensitive data should not be connected to the internet. Just like a military installation the networks should be air-gapped. Our Flight Mission Systems, Electric Grid, LNG Grid, All should be air-gapped. The very nature of the IP stack is insecure. Once you are in a network and scan looking for a database backend there are many SQL injection attacks that are so easy to do that a 12 year old script kiddie would be successful in obtaining administrator privileges. It's time to smarten up and secure our networks.

    • Unlikely, as this stuff is all mainframe based.

      • In many Mainframe applications there is Middleware implemented on Unix and Windows Systems. Perhaps that's the reason why they were back up so fast! Itrs quick and easy to restore a Middle Ware Node

    • I saw several people get a fine for jaywalking and thus every fine that I hear about now without a clear explanation is jaywalking? Is that your logic here?

      These are called "independent" events. The fact some people are getting hit by ransomware doesn't mean every problem everywhere is ransomware. In fact the age and ancient infrastructure of the FAA likely makes them somewhat immune to modern ransomware style attacks. Incidentally the age of infrastructure also goes a long way into explaining the fact they

  • I can just picture an old, dark computer room with people running around yelling. In the middle of a room, a Windows 95 computer is repeating a BSOD and hard reboot. Oops! Maybe they should have upgraded to Windows 98 already!

  • Stop grandstanding (https://www.cnn.com/videos/travel/2022/12/27/southwest-airlines-flight-cancellations-buttigieg-intv-tsr-vpx.cnn) and do your job, please and thank you.

  • Mayor Pete??? (Score:3, Funny)

    by I75BJC ( 4590021 ) on Wednesday January 11, 2023 @09:38AM (#63198918)
    Mayor Pete dissed Southwest for failing to upgrade their computer system and now the FAA fails to upgrade their/his computer system?
    Oh, the irony!

    • Re: (Score:3)

      by smap77 ( 1022907 )

      Agreed: Southwest Airlines is run just like the US Federal Government.

      Excellent comparison.

  • Nothing much was lost (Score:5, Informative)

    by Vlad_the_Inhaler ( 32958 ) on Wednesday January 11, 2023 @09:39AM (#63198924)

    I worked on an Air Traffic Control system around 20 years ago.
    They had the "active machine" and another one one in standby. If the active machine went down they would switch to the reserve and an empty flight-movement database - all the data is pretty much transient anyway. The flight information would come flooding in and after a couple of minutes it was business as usual.
    We had an outage during my time there where records were not being deleted from one file, this was down to a new but defective software feature. That was not noticed in the 60 minute stress test (on the backup machine) because the file was large enough for 50 hours of data, and who cared about a file where only 2% of the records were in use. Switching to the spare machine worked perfectly and the programmer found the bug as soon as he heard what had happened. The bug still caused a lot of flights to be grounded because 5 (?) regional ATCs all had to agree to the switch and one of the 5 did not accept that there was even a problem for well over an hour, then someone described the exact symptoms and it was "oh, we're seeing that as well".

    • Can you give any details the hardware used? Was it x86 based?

      • Re:Nothing much was lost (Score:4, Informative)

        by Vlad_the_Inhaler ( 32958 ) on Wednesday January 11, 2023 @10:44AM (#63199130)

        It was a mainframe, not x86.
        They migrated to Sun-based servers after that, that was a real mess. There were around 5 servers, one for each ATC. They started with the same software but they all forked at that point - corrections for one regional ATC were not applied to the others because no-one knew if they needed them. Not my problem, I was out of there by then.

  • ... and our important infrastructure runs on antiquated code and hardware. Yay, capitalism.

    • The FAA is part of the government, therefore it's socialism that failed.

      I won't blame you if you want to blame Congress though. They are supposed to ride herd on the bureaucracy.

    • The world’s banks run on “antiquated” code. That means the code is well vetted and understood. Let’s dump all that legacy shit and switch to python. Not like anything major happened between versions 2 and 3. You can buy a new million dollar mainframe and it will natively run binaries from 50 years ago.

  • Did they try and reboot using a re-cycled AOL floppy disk?

    I know it's worked before, but . . .

Slashdot Top Deals

Truly simple systems... require infinite testing. -- Norman Augustine

Close