The Inside Story On the San Francisco Network Hijacking

snydeq writes "A source with direct knowledge of San Francisco's IT infrastructure has tipped off Paul Venezia to the real story behind Terry Childs' lockout of San Francisco's network, providing a detailed account of the city's FiberWAN, interdepartmental politics, and Terry Childs himself. Childs pleaded not guilty to charges of tampering yesterday and is being held on $5 million bail. According to the source, Childs' purview was limited to the city's FiberWAN — a network he himself built and, believing no one competent enough to touch the network but himself, guarded religiously, sharing details with no one, including routing configuration and log-in information. Childs was so concerned about the network's security that he refused even to write router and switch configurations to flash. But what may prove difficult for the prosecution in its case against Childs is that his restricted access to the network was widely known and accepted among managers and the city's other network engineers. Venezia, who has been suspicious of the official story from the start, suspects that the Childs case may be that 'of an overprotective admin who believed he was protecting the network — and by extension, the city — from other administrators whom he considered inferior, and perhaps even dangerous.' Further evidence is that fact that the network, from what Venezia understands, has been running smoothly since Childs' arrest."

He's still not justified...

You can try and defend him and glorify him all you want... but as a professional system administrator he should have known that his singular access and pathological behavior was more dangerous than helpful.

What if, instead of being fired he was the victim of an accident or crime? What if he had a health problem? What if a serious, life threatening issue came up (say, you know, an earthquake) that caused the system to be unstable and, at the same time, prevented him from getting there to fix things?

He's still a criminal. But, he's not alone in his behaviour. Whoever his managers are sound to be guilty of criminal negligence. This never should have been possible in a city government the size of San Francisco. Especially when it comes to critical infrastructure. If I were a citizen of San Fran I'd be asking why heads aren't rolling at the highest levels. Why was this allowed to happen? In San Francisco, where you think they'd have no problem finding competent replacements.

Absolutely mind boggling.

Re:He's still not justified...

If this was a case of "He was the only one with the passwords and knowledge, we stupidly fired him without getting that info, and now we realized we're screwed" then he isn't a criminal. His boss maybe, but not him.

Hell, even if the situation was "tell us the info so we can replace you - no - you're fired", he still isn't a criminal. Other than maybe stretching a denial of service crime to fit, other than he hasn't really denied them a service if it is still running.

Re:He's still not justified...

We still don't know all the details. Perhaps all the accusations are trumped. But, if when his performance became a question he started hiding backups, monitoring his managers' email exchanges and is now not cooperating, he's definitely a criminal.

How can you possibly argue otherwise? Sure, he's the network admin, but does that authorize him to read people's email without authorization?

Sure, he's the admin, but does that give him the right to create a situation that basically takes the city's IT infrastructure hostage?

I'm not questioning that his superiors should share the larger part of the blame here. But I can't see how he's not at all at fault.

Re:He's still not justified...

>

How can you possibly argue otherwise? Sure, he's the network admin, but does that authorize him to read people's email without authorization?

Not at all. But then charge him with that, not some pseudo-terrorist computer tampering charge.

Are you sure he's a criminal?

He's certainly guilty of being a bad employee, as well as affirming all of those user-unfriendly IT sterotypes (those are often true, BTW). But criminal?

In America, they have to prove that first. Looking at the statute, it seems it all comes down to the issue of "without permission." The main point the article makes is that he might have had at least understood or standing permission to do most or all of what he did. Just like when you take your parents' car somewhere as a teenager, it isn't theft if it's understood that you are allowed to use it.

The article is one-sided, and his alleged refusal to give up the passwords looks bad (perhaps he is remaining silent until he speaks with counsel), but proving he didn't have permission might be hard. Ergo, no criminal.

Re:Are you sure he's a criminal?

He was in their employ. Once they asked for access and/or recinded his 'permission' and he refused to cooperate he became a criminal. Let's not rationalize or glorify him just because he's a geek...shades of the apologists for Reiser come to mind now, though this crime isn't as bad as murder.

Re:Are you sure he's a criminal?

We're getting the same sort of wagon-circling that we saw when Hans Reiser was charged. No one seems willing to admit that some of us "geeks" are self-important prima donas who border on pathologically criminal behavior. This guy is clearly a criminal. Of course, proper management would have recognized this behavior much earlier, and wouldn't have given him the keys to the kingdom, so it's a combination of a very bad guy and some very incompetent guys. There's no worse a combination.

It's guys like this that bring our IT occupations into ill-repute, by furthering their stereotype of Coke-swilling social retards on power trips. I hope they throw the book at him, and I hope that while he's sitting in prison he has time to ponder the fact that he isn't a god, but merely an employee.

Re:He's still not justified...

>In San Francisco, where you think they'd have no
>problem finding competent replacements.

I guess then that you've never been to San Francisco? San Francisco can't balance their budget and had a hiring freeze since 2007 [sanfranciscosentinel.com] and laid off a lot of people, and only had a skeleton crew running things like IT departments. So things like a network freeze were just bound to happen sooner or later.

George W. Bush isn't the only political leader in the USA who can't balance a budget and is also incompetent and has an incompetent staff. Just look at many state and local governments in places like New York and California. They all want Federal hand-outs to help balance their budgets.

configs are not written to flash, eh?

so the network is NOT locked up, it's just unrestoreble after "password recovery."

sounds like what they need to do is get some qualified engineers to redesign it, and when it's on paper, pull the plug on everything, and reconfigure from scratch.

because if it isn't saved in flash, it's going away as soon as the power light goes out.

which makes our jailed genius a little less than blazing fast. in fact, about half fast. parts of the system ARE going to go down. it's the nature of the beast. no records, no writes... the first time the janitor plugs in a 18-amp vacuum in a rack, it's gone.

they'll come along and take his Cisco cert away for not saving the configs, if for nothing else.

Bail

IANAL, but isn't $5 million US for bail a bit excessive for this?

FiberWAN should not have been deployed then

That's my first reaction to the news. Critical infrastructure should have redundancy everywhere, including the support staff.

To give a stupid but obvious example what if Childs was run over by a car? OK, he wouldn't care but all the rest of SF would.

So they should never have put the network online until the information was in several places (the brains of several people if formal electronic/paper records were too inflexible).

Stll, this sounds like political infighting more than ever. Given the situation why were they trying to fire a critical person like Childs? Sounds like some bureaucrat with an ego as big as Childs would be involved to cause this, rather than Childs "going rogue". And he (the bureaucrat) was more skilled in the political game. Of course this person would be covering his tracks, and not be obvious in any way. So Childs and the whole of SF lost. His firing does not make sense otherwise, given his critical position.

Ah, the fun of weaving conspiracy theories :-)

Like This is Shocking

Every software company I have worked for... if one or two people were hit by a bus... the company would be out-of-business. Management knew this... fellow developers knew it. Its a commonplace thing. Engineers take the work so *personally*. "No one can touch that code but me... " blah... blah. Ånd the stupid management goes along w/ these primadonna's. Of course... if they demanded more money... they'd be gone in a NY minute.

Accidents happen, too.

Every time I see a situation like this, I have to wonder what would happen if an "indispensable" person got hit by a bus. It strikes me that Childs was using his absolute control of the network as a way to put the fear of god in others within the department while attaining more prestige and autonomy than he deserved. The fact that Childs locked everyone out of the system after apparently receiving a poor job assessment backs that up. Sooner or later, the IT department had to take action to strip his stranglehold of the network, especially if he was on the verge of burnout or increasingly difficult to deal with.

I suspect that no one had the interpersonal wherewithal to figure out how to approach him in a non-confrontational manner. The best approach would have been to find someone who Childs respected who could share the load and provide backup and support while the organization attempted to deal with an overly possessive employee who is behaving irrationally.

Complete bunk...

I know someone who worked on the cisco side with this guy. This had been going on for a while. The dude was threatening co-workers doing all kinds of odd stuff. The idea that he was somehow just a little protective is an off the charts miss-representation.

Re:and in stargate news.....

So instead of letting the air out of the car's tires, a car he loved, he simply wouldn't give the keys to dangerous drivers.

short version

short version: if you bad to computers, we bad to you!

Re:short version

Dude, you're never going to stay in office by communicating the simple truth.

Re:Is this really the case?

It's hard to believe that management didn't care that a single employee was the only one who knew anything about critical infrastructure, no matter whether the employee arranged things this way because he thought no-one else was good enough or because this was his was of becoming entrenched.

I find that easy to believe. Even easier to believe that they didn't know this was the case, or knew but did not understand.

Re:Is this really the case?

Never worked for the government, have you? ;)

Management is where people who are too incompetent for technical work go. No one gets fired, they get moved to different departments. As a last resort, they get assigned to 'special projects' for about a year in the hopes that everyone will forget what an imbecile they are, and will be safe to move back into the management structure.

Re:Is this really the case?

Or you write them a glowing recommendation and help them get promoted out.

Re:Is this really the case?

If management behaved like this they would have been fired before this guy was

It's nice to believe that but, to abuse an oft-quoted phrase, quis sacko ipsos pointyhaires?

Before you can fire someone for being a complete idiot, you have to not be totally out to lunch yourself. More importantly you have to possess evidence to back up your decision which is at least strong enough to outweigh the political costs of making it.

If you think this all sounds like a load of crap, then consider yourself lucky that you have never been in the middle of it.

Re:Is this really the case?

funny I find it VERY easy to believe. Right now only 3 people in my own district now the running of the network, and only 1 by extension of that the complete configuration of the OS X server running the mac portion of the district. I have a emergency recovery manual I wrote myself, but it is under lock and key by me to keep all but 2 people from knowing it because I KNOW the other techs and administrators are incompetent political appointees who will royally screw things up and cause much more damage than they solve if they try to implement it without know what is going on.

Re:Is this really the case?

It seems pretty idiotic to me. I still think they should throw this guy in the clink, but at the same time, I think some of his superiors should be told to collect their belongings and then have security escort them through the front door, because there was a colossal breakdown of management here if a single guy was permitted to basically hold the entire network's architecture in his head.

Re:Is this really the case?

If the others were so stupid as to not do anything about this waaaaayyyyy before, then maybe, just maaayyyybe he was right. They are too stupid to be let loose on the network. :-D

My Point was.

People who fiddle with government machines get let of and win people elections! Those that STOP people fiddling with Machines get locked up on $5 mill bail....:D:D