Air Traffic Control "Telephone Glitch" Delays Hundreds of UK Flights 40
First time accepted submitter biodata writes "The BBC is reporting that hundreds of UK commercial air flights have been delayed for most of Saturday due to an internal telephone systems problem in the National Air Traffic Control Service, and delays are likely to continue into the evening. A spokesperson said that it was a different software bug from the one which grounded flights in the summer."
Re:RTFA says it's not telephones (Score:5, Informative)
They do operate it as a 24/7 operation. However, at night time there are less planes in the sky, so each traffic controller is given a bigger area to work on and there are fewer of them on duty. During day time, these areas are subdivided into smaller areas and more controllers are brought on-line to work on the larger number of areas. It was this switch-over that failed.
This was definitely not intentional. (Score:5, Informative)
It's just an unfortunate incident.
British Telecom has had an issue (which has happened a number of times) which led to a minor timing glitch in one of their systems. When this happens, the data reliability on the FARICE line to Iceland drops and you start getting corrupted flight messages. Shanwick was alerted to the problem and both sides consulted and decided that the best solution in the interrim would be something that had been done previously, disconnecting FARICE and thus forcing all connections through the backup line, DANICE, which appeared to be operating normally.
Unfortunately, the problem was even worse on DANICE. What appeared to be normal operation was only normal up to the data logger. Once it actually got to the flight tracking software, the messages were being refused, and corrupted messages being sent in the other direction. So while BT was working on getting their system fixed, flight control managers were being forced to basically manually dig up ATC messages and copy-paste them off to the air traffic controllers (as much was handled through voice as possible as well).
But it got even worse. A totally unrelated communications network, Datalink, decided to misbehave during all of this, which may or may not have been due to the Shanwick problems. On the Iceland side, the general solution is to force a switchover to the backup system. Which was done... except a critical component on the backup system immediately crashed. Repeated attempts to switch and ultimately switch back caused even more problems for the air traffic controllers.
Eventually the fixed FARICE line was brought back up, Datalink back online (with the switchover-crash problem postponed to be investigated during a low-traffic timeperiod)
It's terrible that there were so many delays, but these are extremely complicated systems with a challenging task, built up over decades with tons of computer components, protocols, lines, routers, radar systems, transmitters, and on and on, scattered all over the world. On a weekend. Everyone was scrambling and doing their damndest to fix it as soon as possible. It should also be noted that it was never a safety issue - even in the absolute worst case, air traffic control could go all the way back to the old paper-and-pencil method. What the systems give is, primarily, speed, and thus when there's big problems, there's delays.
And that was my weekend, how was yours? ;)