Forgot your password?
typodupeerror
Security News IT

The Inside Story On the San Francisco Network Hijacking 471

Posted by Soulskill
from the connection-reset-by-lack-of-peers dept.
snydeq writes "A source with direct knowledge of San Francisco's IT infrastructure has tipped off Paul Venezia to the real story behind Terry Childs' lockout of San Francisco's network, providing a detailed account of the city's FiberWAN, interdepartmental politics, and Terry Childs himself. Childs pleaded not guilty to charges of tampering yesterday and is being held on $5 million bail. According to the source, Childs' purview was limited to the city's FiberWAN — a network he himself built and, believing no one competent enough to touch the network but himself, guarded religiously, sharing details with no one, including routing configuration and log-in information. Childs was so concerned about the network's security that he refused even to write router and switch configurations to flash. But what may prove difficult for the prosecution in its case against Childs is that his restricted access to the network was widely known and accepted among managers and the city's other network engineers. Venezia, who has been suspicious of the official story from the start, suspects that the Childs case may be that 'of an overprotective admin who believed he was protecting the network — and by extension, the city — from other administrators whom he considered inferior, and perhaps even dangerous.' Further evidence is that fact that the network, from what Venezia understands, has been running smoothly since Childs' arrest."
This discussion has been archived. No new comments can be posted.

The Inside Story On the San Francisco Network Hijacking

Comments Filter:
  • by ufpdom (556704)
    The giant flash was just some solar burst.. it wasnt anubis' ship
    • by GovCheese (1062648) on Friday July 18, 2008 @10:07PM (#24250439)
      So instead of letting the air out of the car's tires, a car he loved, he simply wouldn't give the keys to dangerous drivers.
      • Mods on crack (Score:5, Insightful)

        by A nonymous Coward (7548) * on Friday July 18, 2008 @11:47PM (#24251007)

        This analogy is spot on, and whoever modded it off-topic obviously is incapable of understanding the topic and shouldn't have had the keys to the mod-car in the first place.

  • by l2718 (514756) on Friday July 18, 2008 @10:11PM (#24250467)
    It's hard to believe that management didn't care that a single employee was the only one who knew anything about critical infrastructure, no matter whether the employee arranged things this way because he thought no-one else was good enough or because this was his was of becoming entrenched.
    • by russotto (537200) on Friday July 18, 2008 @10:13PM (#24250483) Journal

      It's hard to believe that management didn't care that a single employee was the only one who knew anything about critical infrastructure, no matter whether the employee arranged things this way because he thought no-one else was good enough or because this was his was of becoming entrenched.

      I find that easy to believe. Even easier to believe that they didn't know this was the case, or knew but did not understand.

      • by l2718 (514756) on Friday July 18, 2008 @10:23PM (#24250551)

        Even easier to believe that they didn't know this was the case, or knew but did not understand.

        This doesn't sound reasonable. If management behaved like this they would have been fired before this guy was -- the management problems would be greater than the technical ones.

        • by Xzzy (111297) <sether@tr u 7 h . o rg> on Friday July 18, 2008 @10:40PM (#24250649) Homepage

          Never worked for the government, have you? ;)

          Management is where people who are too incompetent for technical work go. No one gets fired, they get moved to different departments. As a last resort, they get assigned to 'special projects' for about a year in the hopes that everyone will forget what an imbecile they are, and will be safe to move back into the management structure.

          • by Televiper2000 (1145415) on Friday July 18, 2008 @10:58PM (#24250755)
            Or you write them a glowing recommendation and help them get promoted out.
          • Re: (Score:3, Funny)

            by JudgeFurious (455868)

            You just described Harris County, Texas to a degree that's downright frightening. Recently the payroll information for the entire county was put online by the Houston Chronicle newspaper and a quick look around has confirmed, at least to my mind that the straightest path to a high salary and zero responsibility here is to excel at being a nincompoop.

            I'm getting right on that by the way. Look for me in upper-middle management in about three years time.

          • Exit stage left (Score:5, Insightful)

            by westlake (615356) on Saturday July 19, 2008 @12:17AM (#24251181)
            Management is where people who are too incompetent for technical work go.
            .

            modded +3, Informative.

            but this attitude sets off alarms.

            exposing a geek who despises his supervisors and is used to thinking of the server rooms as his personal playground.

          • by mooneypilot (1157115) on Saturday July 19, 2008 @12:28AM (#24251235)
            haha..probably right on..I came from the outside, now I work in county govt going on 7 years. I have more knowledge in my little finger than 99 net-sys admins / network engineers picked at random who are working inside the county govt. CLUELESS! No excuse for getting yourself arrested thou... maybe not too late for "I forgot the password" as a defense. Any ideas how to clean up these laggards? Its our freekin tax money down the drain!!
        • by Minwee (522556) <dcr@neverwhen.org> on Friday July 18, 2008 @10:47PM (#24250699) Homepage

          If management behaved like this they would have been fired before this guy was

          It's nice to believe that but, to abuse an oft-quoted phrase, quis sacko ipsos pointyhaires?

          Before you can fire someone for being a complete idiot, you have to not be totally out to lunch yourself. More importantly you have to possess evidence to back up your decision which is at least strong enough to outweigh the political costs of making it.

          If you think this all sounds like a load of crap, then consider yourself lucky that you have never been in the middle of it.

        • Re: (Score:3, Insightful)

          by Dun Malg (230075)

          Even easier to believe that they didn't know this was the case, or knew but did not understand.

          This doesn't sound reasonable. If management behaved like this they would have been fired ...

          Hah! You clearly have never worked for the government. It may not sound reasonable, but bureaucrats are almost always some combination of ignorant and oblivious. I mean, part of the reason they put this guy in charge is that he's probably the only person who knew how to do anything. And you have to ask yourself, who's going to fire these marginally competent managers? Their marginally competent bosses? People who know what they're doing are unfortunately the exception in government. Most competent folks fin

      • Re: (Score:3, Interesting)

        by sasha328 (203458)

        It's hard to believe that management didn't care that a single employee was the only one who knew anything about critical infrastructure, no matter whether the employee arranged things this way because he thought no-one else was good enough or because this was his was of becoming entrenched.

        I find that easy to believe. Even easier to believe that they didn't know this was the case, or knew but did not understand.

        It's not actually the case of people not knowing the passwords or such, from what I've read

      • by Anonymous Coward on Saturday July 19, 2008 @01:35AM (#24251535)

        I post AC because of my position, which is basically a guy who was hired as the second network tech to help manage the network for a sizeable city (large enough that we host several professional sports teams). I had no real qualifications other than knowing how to google my way out of most basic computer situations. My supervisor managed all City-owned Cisco equipment and it has only been 2 of us for 2 years. We manage over 300 Cisco devices at over 100 sites and I can honestly say that after reading a few more details on this story, I can easily understand how this can happen in a local government. I believe that the problem is in management. We have similar problems in our City regarding the lack of passing of knowledge and lack of staffing, but we have a good security team that knows more about Cisco networks than the 2 of us that regularly work on the Cisco equipment in our City. They are not normally watching our backs (that we know of) but they would certainly do so if they got a bad vibe about us. We have to share passwords with them and they have as much access to our equipment as we do. It is simply a requirement in a publicly owned system that knowledge is shared. Taxpayers have payed for the equipment and expect that there are not single points of failure. There are many reasons that more people than work on one thing on a regular basis have knowledge of and access to the most basic systems. If there was no redundancy, then it is a fundamental failure of management.....I'm not saying the guy should have set one password and not passed it on.....but I understand.

    • by falcon5768 (629591) <Falcon5768@comca ... t minus caffeine> on Friday July 18, 2008 @10:47PM (#24250695) Journal
      funny I find it VERY easy to believe. Right now only 3 people in my own district now the running of the network, and only 1 by extension of that the complete configuration of the OS X server running the mac portion of the district. I have a emergency recovery manual I wrote myself, but it is under lock and key by me to keep all but 2 people from knowing it because I KNOW the other techs and administrators are incompetent political appointees who will royally screw things up and cause much more damage than they solve if they try to implement it without know what is going on.
    • by MightyMartian (840721) on Friday July 18, 2008 @10:54PM (#24250731) Journal

      It seems pretty idiotic to me. I still think they should throw this guy in the clink, but at the same time, I think some of his superiors should be told to collect their belongings and then have security escort them through the front door, because there was a colossal breakdown of management here if a single guy was permitted to basically hold the entire network's architecture in his head.

      • Re: (Score:3, Insightful)

        by Detritus (11846)
        You can't do that to the Mayor. The higher the position, the less likely that the person occupying it was hired based on their qualifications for the job.
    • by theshowmecanuck (703852) on Friday July 18, 2008 @11:01PM (#24250773) Journal
      If the others were so stupid as to not do anything about this waaaaayyyyy before, then maybe, just maaayyyybe he was right. They are too stupid to be let loose on the network. :-D
    • by mkcmkc (197982) on Saturday July 19, 2008 @12:03AM (#24251097)

      In my experience, it's a rare company indeed whose managers can fathom the implications of a situation like this. In general, I'm unable to get management to even understand Rule Zero of system administration. Which is: Do everything you need to do to be drop dead certain that you always have a reasonable backup of your important systems. This doesn't sound too difficult, but in practice it's difficult to convince managers that an event that could happen with probability == 0.01 could ever happen...

    • Re: (Score:3, Informative)

      by pluther (647209)
      It's hard to believe that management didn't care that a single employee was the only one who knew anything about critical infrastructure,

      You've obviously never worked for local government.

    • by v1 (525388) on Saturday July 19, 2008 @09:25AM (#24253089) Homepage Journal

      This all depends on who accepts it when. If when Childs started his lockdown, he was under the watch of a manager that either didn't care, or agreed with him, and so he did as he pleased.

      A lot of IT organizations have a single person that is the core, the one person that has comprehensive knowledge of all systems and fully understands how they interact. These are the people that are brought in on any major problem or decision, and whose input counts more than double. (and often simply hold "veto power") Now I'm not saying this is a good thing, I'm just saying it happens from time to time and you have to accept that. Some systems just evolve in this direction, and once they get past a certain point, it becomes very hard to change them.

      There IS one easy way to solve these problems, but it involves the managers taking a walk out on a shaky limb and take some heat. One example is a week of paid leave. On Monday Joe's manager announces "Joe is on paid leave effective immediately. (no warning to Joe OR the staff in advance of this) Go home Joe, see you in a week and enjoy your paid time off, courtesy of the company." Then, "OK for the next week you are on your own. NO ONE is to call, page, IM, email, or otherwise contact Joe for ANY REASON. Joe got hit by a bus this morning on the way to work, that's how you will behave. You are to keep written track of every problem you run into this week that you would normally rely on Joe to help with. Do not simply shelve problems for next week - treat them like Joe is never coming back." Any critical questions you bring to ME, and I will call Joe if it's really necessary, but I will not be happy about it, and be prepared to justify to me that you've already tried everything else possible. If I find someone is hiding problems for next week there will be serious disciplinary action taken.

      Needless to say, when Joe gets back on Monday, the next 2-4 weeks will probably be planned out, documenting things and teaching people how to do stuff. You could also make this a two week leave depending on your situation. If you're a big organization, the longer the better, but at the worst three weeks will shake out most of the bugs. This also gives the managers a very clear picture of how well distributed knowledge is within the department. You've probably heard someone say "but what if you got hit by a bus tomorrow?" when discussing something you are the only one that knows how to do. Now you get answers. We call this the "hit by a bus test". Any decently sized IT department with one central person should conduct this test periodically, say every two years. The first one should be a gimme. If on the second test, things have not improved over the first, time to take disciplinary action. Letting one of your staff continue to hold the keys to the kingdom is unacceptable and is everybody's fault to some degree.

      • by adamruck (638131) on Saturday July 19, 2008 @11:26AM (#24253819)

        As a sysadmin who is part of a small team that "holds the keys", I find your comment interesting. In most organizations, you don't have to specifically plan for a "hit by a bus test", because it happens all on its own. Don't your employee's take vacations? Don't they ever call in sick? If your employee's have to call the guy on vacation, that is a HUGE HUGE HUGE indicator that there is a problem.

  • by numbsafari (139135) <swilson.bsd4us@org> on Friday July 18, 2008 @10:13PM (#24250479)

    You can try and defend him and glorify him all you want... but as a professional system administrator he should have known that his singular access and pathological behavior was more dangerous than helpful.

    What if, instead of being fired he was the victim of an accident or crime? What if he had a health problem? What if a serious, life threatening issue came up (say, you know, an earthquake) that caused the system to be unstable and, at the same time, prevented him from getting there to fix things?

    He's still a criminal. But, he's not alone in his behaviour. Whoever his managers are sound to be guilty of criminal negligence. This never should have been possible in a city government the size of San Francisco. Especially when it comes to critical infrastructure. If I were a citizen of San Fran I'd be asking why heads aren't rolling at the highest levels. Why was this allowed to happen? In San Francisco, where you think they'd have no problem finding competent replacements.

    Absolutely mind boggling.

    • by Zerth (26112) on Friday July 18, 2008 @10:20PM (#24250529)

      If this was a case of "He was the only one with the passwords and knowledge, we stupidly fired him without getting that info, and now we realized we're screwed" then he isn't a criminal. His boss maybe, but not him.

      Hell, even if the situation was "tell us the info so we can replace you - no - you're fired", he still isn't a criminal. Other than maybe stretching a denial of service crime to fit, other than he hasn't really denied them a service if it is still running.

      • by numbsafari (139135) <swilson.bsd4us@org> on Friday July 18, 2008 @10:27PM (#24250579)

        We still don't know all the details. Perhaps all the accusations are trumped. But, if when his performance became a question he started hiding backups, monitoring his managers' email exchanges and is now not cooperating, he's definitely a criminal.

        How can you possibly argue otherwise? Sure, he's the network admin, but does that authorize him to read people's email without authorization?

        Sure, he's the admin, but does that give him the right to create a situation that basically takes the city's IT infrastructure hostage?

        I'm not questioning that his superiors should share the larger part of the blame here. But I can't see how he's not at all at fault.

        • by rwillard (1323303) on Friday July 18, 2008 @10:38PM (#24250635)

          >

          How can you possibly argue otherwise? Sure, he's the network admin, but does that authorize him to read people's email without authorization?

          Not at all. But then charge him with that, not some pseudo-terrorist computer tampering charge.

          • by bmo (77928) on Friday July 18, 2008 @11:56PM (#24251059)

            >>How can you possibly argue otherwise? Sure, he's the network admin, but does that authorize him to read people's email without authorization?

            >Not at all. But then charge him with that, not some pseudo-terrorist computer tampering charge.

            The Electronic Communication Privacy Act of 1986 protects administrators if "in the performance of their duty" they read email. Please note the date. If you are unfamiliar with it, you should be even if you're "just a user", no excuses.

            He's an administrator. He's shielded.

            Y'all should know that by now.

            You should also know that if you store your email on company servers/isp servers, they get /less/ protected as time goes on, with most protection going to those "in flight" and least to those being stored for over a year.

            If you have anything confidential, encrypt it and remove it from your provider's machines and store elsewhere. If you don't ever want the admin to see the email in flight, then end-to-end encryption. These days it's easier than the mid 1980's.

            OB On Topic: I can see where he's coming from. A network administrator, if he's doing his job, gains a bit of paranoia. Sometimes that can become unhealthy, and it appears that he's crossed the line into "unhealthy". Criminal? I don't think so. It appears that he's been severely mismanaged by those who never understood "Mack Truck Syndrome". One guy for an entire city? I'm not sure who's crazier, the management or him.

            --
            BMO

            • by IntlHarvester (11985) * on Saturday July 19, 2008 @02:58AM (#24251837) Journal

              I think you're full of brown smelly stuff.

              The ECPA only seems to apply to common carriers and public information services. I don't see any evidence it provides any liability for the sysadmins of internal networks.

              If you're not IANAL, here it is:
              http://www.usdoj.gov/criminal/cybercrime/wiretap2510_2522.htm [usdoj.gov]

              And even if so, you're being really retarded if you think that reading his bosses' email falls under the "system monitoring" provision of the law.

            • by YesIAmAScript (886271) on Saturday July 19, 2008 @03:40AM (#24251973)

              That law is there to make it possible for administrators to do their work. If you are working with emails, and you happen to see a few, you don't go to jail for it.

              But monitoring his bosses' email so you can tell what they are saying specifically about him is highly unlikely to be in his job description, and thus he is not protected when he does that. Nor should he be.

        • by Zerth (26112) on Friday July 18, 2008 @10:50PM (#24250713)

          If he really did explicitly "hold the network hostage", actually said "I'll trash it if I don't get what I want", then he commited a crime. But what it sounds like so far is "Do your job the way we want, not your way" and he said no and was fired for it, which is generally not a criminal act.

          I've known half a dozen people who "knew things" that would ruin their company if they were hit by a bus. None of them would get charged with a crime if they refused to give up that information *after* being fired(although their company might get sued by the shareholders). But none of them are in IT.

          As for the email, from the correspondance provided, it doesn't say if he had access to the city's mail servers, but then he isn't being charged with breaking in to them either. Seeing as he ran the network, it'd probably be easy to sniff and read the email "on the wire" without breaking into a computer, since I doubt anyone in the city government used encryption.

          Ok, now I'm being a bit nitpicky, sorry:), but how often do we compare email to sending postcards? Other than cellular communications, where else is it illegal to detect something broadcast in the clear?

          • by SL Baur (19540) <steve@xemacs.org> on Saturday July 19, 2008 @12:21AM (#24251201) Homepage Journal

            how often do we compare email to sending postcards?

            On the Cypherpunks mailing list, all the time. On Slashdot, I don't think I've ever seen anyone bring it up. Email is just that - a postcard. If you care about the privacy of your mail, encrypt it.

          • Re: (Score:3, Interesting)

            by Anonymous Coward

            I am not gonna use my normal login on this and you will understand in a second.

            I have been working in, on, and around computers (hardware and software) since before most of you were born. Some of you, your parents. And at almost all of the companies that I have worked for, there was something that "everybody knew" that would have gotten the company heavily penalized if not shut down if the word got to the appropriate regulatory agency.

            Example: did you know that the sales tax you paid on your CompUSA purch

        • Re: (Score:3, Funny)

          by nospam007 (722110)

          > Sure, he's the admin, but does that give him the right to create a situation that basically takes the city's IT infrastructure hostage?

          He's just following the most basic security rule that has been told here thousands of times:

          Never give anybody your password! Ever!

      • Re: (Score:3, Informative)

        by KenSeymour (81018)

        I have been following this story in the San Francisco Chronicle.

        According to their reporting, he was asked by management for the passwords. He said no.
        Then he was asked by police for the passwords. He still said no.
        Then they had him arrested.

        The reporting by various news organizations has been marred by confusion
        on the part of reporters and average people between controlling network hardware and controlling
        data on various servers. They often seem to describe it as data being
        stored on the network he contr

    • by unassimilatible (225662) on Friday July 18, 2008 @10:21PM (#24250533) Journal
      He's certainly guilty of being a bad employee, as well as affirming all of those user-unfriendly IT sterotypes (those are often true, BTW). But criminal?

      In America, they have to prove that first. Looking at the statute, it seems it all comes down to the issue of "without permission." The main point the article makes is that he might have had at least understood or standing permission to do most or all of what he did. Just like when you take your parents' car somewhere as a teenager, it isn't theft if it's understood that you are allowed to use it.

      The article is one-sided, and his alleged refusal to give up the passwords looks bad (perhaps he is remaining silent until he speaks with counsel), but proving he didn't have permission might be hard. Ergo, no criminal.
      • by dreamchaser (49529) on Friday July 18, 2008 @10:29PM (#24250595) Homepage Journal

        He was in their employ. Once they asked for access and/or recinded his 'permission' and he refused to cooperate he became a criminal. Let's not rationalize or glorify him just because he's a geek...shades of the apologists for Reiser come to mind now, though this crime isn't as bad as murder.

        • by Zerth (26112) on Friday July 18, 2008 @10:55PM (#24250737)

          Well, now that you've invoked Reiser, it'll probably be true. It'll be a new rule: "If somebody mentions Reiser, the accused geek is probably guilty."

        • by MightyMartian (840721) on Friday July 18, 2008 @11:00PM (#24250769) Journal

          We're getting the same sort of wagon-circling that we saw when Hans Reiser was charged. No one seems willing to admit that some of us "geeks" are self-important prima donas who border on pathologically criminal behavior. This guy is clearly a criminal. Of course, proper management would have recognized this behavior much earlier, and wouldn't have given him the keys to the kingdom, so it's a combination of a very bad guy and some very incompetent guys. There's no worse a combination.

          It's guys like this that bring our IT occupations into ill-repute, by furthering their stereotype of Coke-swilling social retards on power trips. I hope they throw the book at him, and I hope that while he's sitting in prison he has time to ponder the fact that he isn't a god, but merely an employee.

          • Re: (Score:3, Insightful)

            by Anonymous Coward

            I hope that while he's sitting in prison he has time to ponder the fact that he isn't a god, but merely an employee.

            If the article is right, the guy was on perpetual on call duty. Quite frankly, some of the things that are expected of certain IT people (and basically nobody else except the occasional doctor or military personnel) go beyond the realm of "merely" being an employee (and those other vocations are pitched as lifestyles rather than careers, as well). For folks in those positions, if you don't go

          • by Peter La Casse (3992) on Saturday July 19, 2008 @12:15AM (#24251167) Homepage

            We're getting the same sort of wagon-circling that we saw when Hans Reiser was charged. No one seems willing to admit that some of us "geeks" are self-important prima donas who border on pathologically criminal behavior.

            You seem willing to.

            This guy is clearly a criminal.

            I'm waiting to hear the whole story.

          • by grasshoppa (657393) <skennedyNO@SPAMtpno-co.org> on Saturday July 19, 2008 @02:25AM (#24251701) Homepage

            It's guys like this that bring our IT occupations into ill-repute, by furthering their stereotype of Coke-swilling social retards on power trips.

            On the other hand, the more people like this there are, the more employment I get. I may not be as technically capable as folks like Child seems to be, but I am able to work with large groups of people AND the work gets done and documented. I can turn a pretty penny because of how "Customer Service Oriented" I am and how well I document my work.

          • by jd (1658) <imipak@noSPam.yahoo.com> on Saturday July 19, 2008 @03:37AM (#24251959) Homepage Journal
            Be fair. Psychologists have been pointing out for some time that the same traits that make for good managers and administrators are listed in psychology texts as the traits of full-blown schizophrenics. I'm serious.

            Yes, he may very well have bordered on "pathological criminal behaviour", but that is what is expected of employees. It is necessary to exhibit exactly that behaviour if you wish to successfully rise through the ranks.

            I dislike that, and believe it is one incredibly unhealthy attitude - tied utterly to America's Puritan "Work Ethic". However, that is neither here nor there. The guy is a product of such attitudes. If he is a monster, then Herr Frankenstein bears the greater responsibility.

            That does not make him blameless. It means that he is culpable only to some degree below 100%. Punish him for that percentage he is culpable. Fine. But it means there is also a non-zero component of responsibility elsewhere, which should not go unpunished, and that there is a non-zero element of illness the guy has developed as a result, for which he should be treated.

            In any such system, blaming one person is extremely easy but utterly futile. It doesn't fix the underlying problems which made the failure possible, and the ultimate problem is invariably the mental illness prevalent in modern management methods.

          • by AK Marc (707885) on Saturday July 19, 2008 @11:23AM (#24253789)
            There is only one job I was ever fired from. I was laid off as part of a merger. I knew more about networking than anyone else at the 10,000 employee company. I was the only one there to my knowledge that had ever set up a VPN. I was the only one there that knew what spanning tree was and how it was used. When I left, I took no information with me, they had every log in for the many devices I was the only person to ever log into. Everything was written to flash so if a password recovery was necessary, they could perform it and not lose the config. As part of the merger, they tried to set up a VPN between the two headquarters. My understanding is that they had to pay $20k+ for consultants to come in and set up a single VPN that would have taken me an afternoon with spare gear. My manager would call and share stories of the networking difficulties. I didn't hide anything from them, but no one there was hired for networking capabilities except me. Prior to me, all networking was done by consultants that set up something then went away, much like an electrical infrastructure.

            Now, if the CIO had called me up and asked me to assist with something, by your statements, I'd be a criminal to tell him to fuck himself. I somehow have some duty to a company that was firing me. I disagree, and I had no requirement to assist them in making anything work better, and if there was a password I had neglected to pass along, I have no legal requirement to share that with them. I've worked with the protective guys, and I hate it, but I've never seen any of them as criminal and think that's an unfair characterization. If he's a criminal, then it's a conspiracy and his boss should be in jail beside him. His boss knew what he was doing, allowed it, and even paid him to do it. If you pay someone to commit a criminal act, knowing it is a criminal act, you are complicit.

            So yes, I can see how people can say it is "wrong" to do what he did. I agree. But the issue is the law. Murdering someone is a thing I think we can all agree is illegal. But not telling someone a work password when they demand it after you have already been fired? There is no law I know against that. We aren't circling tthe wagons because we think the guy is a saint. We are circling the wagons because we don't want a court ruling that could result in 10 years of jailtime for forgetting a password (and believe me, a cop demanding an answer from you takes "I don't remember" to be the same as "I know the answer and I won't tell you, fuck you pig").
        • Don't make ad hominem attacks please. I called the article one-sided, and merely presented a legal analysis of his case. I did not "rationalize" or "glorify" him. Truth be told, I actually tend to dislike IT geeks. They tend to be rude and have no personality and think they are smarter than everyone (which is usually not the case) and believe they are God's gift to an organization. Such attitudes should not be tolerated, regardless of how skilled an IT guy is.

          With that said, government organizations tend
        • by Motherfucking Shit (636021) on Friday July 18, 2008 @11:57PM (#24251067) Journal

          He was in their employ. Once they asked for access and/or recinded his 'permission' and he refused to cooperate he became a criminal.

          I'll be the first to admit that I don't know the entire story here, but since when is disagreeing with your boss a criminal offense?

          What he did is inappropriate, but once they asked for access and/or rescinded his 'permission' and he refused to cooperate, he became a candidate for termination and perhaps civil liability. Whether or not he committed any criminal acts is up for debate. I think it's very dangerous to suppose that resisting your employer - even, no, especially if your employer is the government - is illegal.

        • by Anml4ixoye (264762) on Saturday July 19, 2008 @01:03AM (#24251405) Homepage

          What? How is refusing to cooperate a criminal act?

          Look, I've worked in government. In fact, I've met with the SF staff before (many moons ago). It sounds to me like he got caught up in a political battle. His saying no has probably been what has kept the network up, and this time was likely no different, except that the guy he said no to decided to make a martyr out of him.

          Is what he did silly? Perhaps - but perhaps not. We can say that "They can just bring in Cisco", but I also used to work for MSFT - and I know that not everyone who comes in from the field is going to be able to transition vital systems without a hiccup.

          There's a lot more to the story, and given what history I know of the SF departments from back in the day, I'm sure that this guy is guilty of nothing other than wanting to protect a critical network, and being a little misguided.

        • Re: (Score:3, Interesting)

          by Fex303 (557896)
          Sorry, what is the crime?

          I think everyone agrees that the guy is being a jerk. And from what I originally read, I thought he was probably a criminal too. However, this makes it sounds like he might not be.

          Refusing to do what your boss says to do isn't a crime. Getting fired for it and then refusing to do what your former boss said to do isn't a crime. Since the situation before that was considered satisfactory by his former employers (because they were so naÃve to trust one guy with the keys to

        • by Minwee (522556) <dcr@neverwhen.org> on Saturday July 19, 2008 @09:54AM (#24253247) Homepage

          You're only hearing one side of the story, and that's not his. Consider this scenario. After Childs was fired by someone higher up in the poop-rolling-downhill order, said manager (Let's just refer to him 'Bonehead' for now) realized that the network policy he had written himself four years ago ("Leave all network devices at their default passwords, put a hard copy of all config files on the bulletin board in the lobby") was no longer valid. Bonehead then tried to break into the systems with a very clumsy brute force attack and got himself locked out. Rather than admit that he really had no clue of what he should be doing and should never have been allowed into the same room as anything more complicated than two cans and some string, Bonehead bravely announced that he was the victim of some kind of sabotage or terrorism and that the inconvenient former administrator should be arrested immediately.

          When contacted about the problem, Childs replied "What? I already told you all the passwords you need. Here they are again. Read your email, you twit.", but of course those accounts had already been locked out thanks to Bonehead's bumbling attempt at cracking his own network. Things have now gone from bad to worse and poor, misunderstood Terry is now torn being asked to cough up passwords to accounts that he has already provided and is torn between just keeping his mouth shut to avoid being drawn further into this whole stupidity and quoting Ray Patterson by saying "You know, I'm not much on speeches, but it's so gratifying to leave you wallowing in the mess you've made. You're screwed, thank you, bye."

          But what about those million-dollar-an-hour Cisco engineers who are desperately trying to fix the network but can't make any headway against Terry Childs' evil hackerish plots? It's hard to fix anything when Bonehead keeps getting in the way. "No no no... For security reasons nobody else can touch the network. Why don't you just tell me what commands you need me to run and then I will do them? No I already tried that! The admin password is 'password'. I already told you that. Stop wasting your time with this."

          I'm not saying that this has to be what happened, but remember that you're only hearing one side of the story here. It's just as possible that Childs is just taking the blame for someone else's screwup and is just pissed off enough about being arrested over it that he's not cheerfully volunteering to help clean it all up again as it is that he really is the James Bond style evil mastermind who is trying to hold an entire city for ransom.

    • >In San Francisco, where you think they'd have no
      >problem finding competent replacements.

      I guess then that you've never been to San Francisco? San Francisco can't balance their budget and had a hiring freeze since 2007 [sanfranciscosentinel.com] and laid off a lot of people, and only had a skeleton crew running things like IT departments. So things like a network freeze were just bound to happen sooner or later.

      George W. Bush isn't the only political leader in the USA who can't balance a budget and is also incompetent and has an incompetent staff. Just look at many state and local governments in places like New York and California. They all want Federal hand-outs to help balance their budgets.

    • by SL Baur (19540) <steve@xemacs.org> on Saturday July 19, 2008 @12:07AM (#24251117) Homepage Journal

      Why was this allowed to happen? In San Francisco, where you think they'd have no problem finding competent replacements.

      This man was living in Pittsburg. They could not find *anyone* in SF to do the job.

      I knew there was more to the story when we got the first article. The fact that he built the network, management allowed him to be the sole caretaker of the configuration *and* that the system is still running smoothly unattended makes it hard to accuse him of sabotage or "hijacking". The time to beg a system administrator to document his work is certainly not after you have him arrested.

      Heads should be rolling in the city government.

  • by pushing-robot (1037830) on Friday July 18, 2008 @10:15PM (#24250491)

    Simon Travaglia? [wikipedia.org] Is that you?

  • by swschrad (312009) on Friday July 18, 2008 @10:15PM (#24250495) Homepage Journal

    so the network is NOT locked up, it's just unrestoreble after "password recovery."

    sounds like what they need to do is get some qualified engineers to redesign it, and when it's on paper, pull the plug on everything, and reconfigure from scratch.

    because if it isn't saved in flash, it's going away as soon as the power light goes out.

    which makes our jailed genius a little less than blazing fast. in fact, about half fast. parts of the system ARE going to go down. it's the nature of the beast. no records, no writes... the first time the janitor plugs in a 18-amp vacuum in a rack, it's gone.

    they'll come along and take his Cisco cert away for not saving the configs, if for nothing else.

    • Re: (Score:3, Interesting)

      by bagboy (630125)
      Any cisco router/switch can be set to netboot their configuration. You can keep the full config on a secure linux/etc. box and netboot (encrypted) it. More secure that way? Possibly. Limited access to the box it's stored on could keep it more secure and tightly controlled.
      • Re: (Score:3, Interesting)

        by inKubus (199753)

        Yeah, but there's no evidence! What are they going to do, bring the router to the courtroom?

  • Bail (Score:5, Insightful)

    by Ceiynt (993620) on Friday July 18, 2008 @10:17PM (#24250499)
    IANAL, but isn't $5 million US for bail a bit excessive for this?
    • Re:Bail (Score:4, Insightful)

      by catmistake (814204) on Friday July 18, 2008 @11:24PM (#24250899) Journal

      I agree, however... high profile case, prosecutor (arguably much more powerful than a judge) wants to win with glory, so keep the suspect incarcerated to make him look guilty, makes an exaggerated case for flight risk, and pulls from his tool bag his only tool, his personal fly-swatter (which is actually an over-sized sledgehammer), and with absolutely zero finesse, smashed that fly with an absurd display of force. This is normal operating proceedure.

    • Re: (Score:3, Informative)

      by lpangelrob (714473)

      If you post the standard 10% for release, he could possibly come up with the $500,000. By mortgaging any property he owns, he just might be able to get that.

      The bigger deal is that I guess they think he's a flight risk.

  • by paratiritis (1282164) on Friday July 18, 2008 @10:18PM (#24250515)

    That's my first reaction to the news. Critical infrastructure should have redundancy everywhere, including the support staff.

    To give a stupid but obvious example what if Childs was run over by a car? OK, he wouldn't care but all the rest of SF would.

    So they should never have put the network online until the information was in several places (the brains of several people if formal electronic/paper records were too inflexible).

    Stll, this sounds like political infighting more than ever. Given the situation why were they trying to fire a critical person like Childs? Sounds like some bureaucrat with an ego as big as Childs would be involved to cause this, rather than Childs "going rogue". And he (the bureaucrat) was more skilled in the political game. Of course this person would be covering his tracks, and not be obvious in any way. So Childs and the whole of SF lost. His firing does not make sense otherwise, given his critical position.

    Ah, the fun of weaving conspiracy theories :-)

    • Re: (Score:3, Interesting)

      by Anonymous Coward

      More details here [informationweek.com]

      Childs, who works in the city Department of Technology, allegedly created a password that gave him exclusive access to the city's new FiberWAN (wide area network), authorities told the newspaper. He has refused to divulge the password, leaving other system administrators locked out.

      Undoing Childs' alleged tampering could cost millions of dollars, city officials said. In the meantime, the system is operating, even though administrators have limited or no access.

      Childs, who has worked for th

  • by Black-Man (198831) on Friday July 18, 2008 @10:20PM (#24250527)

    Every software company I have worked for... if one or two people were hit by a bus... the company would be out-of-business. Management knew this... fellow developers knew it. Its a commonplace thing. Engineers take the work so *personally*. "No one can touch that code but me... " blah... blah. Ånd the stupid management goes along w/ these primadonna's. Of course... if they demanded more money... they'd be gone in a NY minute.

    • by IntlHarvester (11985) * on Saturday July 19, 2008 @03:12AM (#24251865) Journal

      Engineers take the work so *personally*. "No one can touch that code but me... " blah... blah.

      I dunno. There's a fundamental difference between someone being naturally protective over their work and someone who volunteers to be on call 24/7 because he doesn't trust his coworkers with the passwords.

      I've been in both positions. Shitty political situations where I hand over documentation and walk out the door with my head held up. And as the guy who comes in to inherit the mess when the "indispensable guru" quit.

      Neither situation is really all that life-threatening. Nobody is really indispensable.

      I don't believe for a second that the guy was irreplaceable except for the passwords that he intentionally withheld. The city could easily make a call and have an even bigger Cisco genius on site within a week. (After all the Bay Area is where Cisco is HQed.) A legend in his own mind.

  • by Dzimas (547818) on Friday July 18, 2008 @10:22PM (#24250545)

    Every time I see a situation like this, I have to wonder what would happen if an "indispensable" person got hit by a bus. It strikes me that Childs was using his absolute control of the network as a way to put the fear of god in others within the department while attaining more prestige and autonomy than he deserved. The fact that Childs locked everyone out of the system after apparently receiving a poor job assessment backs that up. Sooner or later, the IT department had to take action to strip his stranglehold of the network, especially if he was on the verge of burnout or increasingly difficult to deal with.

    I suspect that no one had the interpersonal wherewithal to figure out how to approach him in a non-confrontational manner. The best approach would have been to find someone who Childs respected who could share the load and provide backup and support while the organization attempted to deal with an overly possessive employee who is behaving irrationally.

    • by PIPBoy3000 (619296) on Friday July 18, 2008 @11:21PM (#24250873)
      I get a little tired with the "hit by a bus" example. My coworkers use it all the time as an excuse to make me document everything to the Nth degree.

      Maybe they could suggest "crushed in an orgy" or "broke lightspeed and turned to photons". Getting hit by a bus is such a boring way to go.
    • Re: (Score:3, Funny)

      by Lemmy Caution (8378)

      Every time someone describes "hit by a bus" scenario, I think that we really should get rid of buses altogether.

  • Complete bunk... (Score:5, Interesting)

    by Anonymous Coward on Friday July 18, 2008 @10:29PM (#24250597)

    I know someone who worked on the cisco side with this guy. This had been going on for a while. The dude was threatening co-workers doing all kinds of odd stuff. The idea that he was somehow just a little protective is an off the charts miss-representation.

  • by Mr. Lwanga (872401) on Friday July 18, 2008 @10:44PM (#24250679) Journal
    1. Terry: you selfish bastard, if your network cannot be maintained without you, you have failed as an admin 2. The city of SF: common sense - try it out some time 3. The tax payers: what did you do to deserve this?
  • by Captain Sarcastic (109765) * on Friday July 18, 2008 @11:28PM (#24250925)

    Let's leave out the legal ramifications here, and let's not go to the hysteria of "he's being thrown to the wolves to protect management" or "he's an evil hacker who shut down the city government networks."

    When it comes down to it, one has to ask what Childs' job was. He was supposed to manage the network for the San Francisco city government.

    As a result, he was supposed to implement policy as communicated to him by his bosses... but he also had the latitude to take actions to support the spirit of those policies where the right action was unclear. And yes, this is a Pollyanna-esque (is that a word?) view of the situation, but it leaves out the concept of malice as the driving force for either side - because it didn't start out as a plan to shut down the city.

    Somehow it morphed into him becoming the sole support for the network routers, be it through arrogance ("I can't believe anyone else would do this right!") or being the only one available ("There's nobody else who works here who even understands the need!"), and at that point this became an incident waiting to happen.

    So, either he refused to do his job (at which point he would have deserved to be fired), or his job was such that he was prevented from doing it (at which point professional ethics would have suggested his resignation - or at least, that's what engineering associations would have recommended in similar scenarios).

    Instead, he stayed on and we have the current state of affairs.

  • Simple test (Score:3, Informative)

    by sthomas (132075) on Friday July 18, 2008 @11:58PM (#24251069)

    Power cycle the network equipment. If it comes back up, pay him for the rest of the year as severance and let him go his own way. If it doesn't come back up, put him away for 10-15 years for public endangerment, and fine him whatever the cost is to the city to recreate the network and for any loss of productivity in the meantime. Either way he is a terrible admin - no one single person should be a single point of failure. What if he got hit by Muni at lunch one day?

  • all networks once configured properly, run smoothly until they don't.

    when they don't, there's one man who can fix it.

    I can fully understand setting up a complex system and getting it working perfectly and then some other admin or consultant coming in and fucking it up.

    when they fuck it up, you have to fix it. And you don't get bonus pay for that.

    not only that, but network/system administrators have to worry a lot about whether management wants to can them simply because things are running so smoothly that they have nothing to do. Which is bullshit because half of the job is keeping up with current tech trends, learning new technologies, and protecting your network on a daily basis. I don't blame the man for guarding his creation jealously. When you start handing over the keys, you are no longer necessary. You get paid too much and this kid who just quit his job of six months from bimblebomble.com seems to know how to do what you do. And we can pay him a lot less and potentially cut out benefits.

  • by Evets (629327) on Saturday July 19, 2008 @02:57AM (#24251835) Homepage Journal

    The city has a huge issue here.

    This guy will have a hard time getting a job in the future, and a guy with his credentials commands a lot more than he is making right now.

    If it turns out that the facts of this case are far from the original story, and nobody from the city is stepping in to correct it, then SF is in the same situation as the US when Ashcroft pointed the finger at the Anthrax guy (who recently won a big chunk of change for the false accusation).

    Something tells me that the wheels of government turn slowly enough that even if they wanted to correct themselves at this point, they won't until well after the publicity is over.

  • by Kaashar (738775) <kaashar@hRASPotmail.com minus berry> on Saturday July 19, 2008 @05:08AM (#24252225)
    I find the situation startling familiar. It's downright creepy to read this scenario. Back in the late 90s I was the sysadmin of a moderately sized ISP. When we started out I was one of three network engineers hired to build the ISP; eventually I ended up in 'charge' of the system. Like the article I also was very protective of my network, and as paranoid as this individual is made out to be. Granted I was in my 20s and suitably arrogant to boot, more on this in a moment. As time went on first one, then the other guy quit after working 80 hours a week without the possibility of time off...things only got worse as people quit. When it was down to me I made sure the owners knew the passwords to everything, but they lacked any knowledge of how to do anything. This came back to haunt me later as you'll see. Eventually I too got fed up and went to work for another company that wasn't a direct competitor. Before I left I advised management on changing all passwords for both of our sakes. I tried to explain everything but nobody understood the technical aspects. Two months later I got a visit from the FBI. 8 grueling hours of interrogation later from armed men I found out that the entire network had crashed, and I was under suspicion as having remotely logged in and crashing their system. It wasn't until later I found out they never hired a replacement, and my system simply collapsed due to lack of maintenance. It's easy to be painted out as the bad guy when you intimately know the network while being managed by a bunch of clueless twits. I don't know if that's the case in this guy's case, but I can see it working either way.
  • by walterbyrd (182728) on Saturday July 19, 2008 @01:57PM (#24254947)

    Childs, who now sits in a jail cell on US$5 million bond, also happens to be a former felon convicted of aggravated robbery and burglary stemming from charges over two decades ago, which the city knew when it hired him as a city computer engineer.

    http://www.csoonline.com.au/index.php/id;1895501252;fp;2;fpid;1 [csoonline.com.au]

    Yet the city gave hime full admin access to a critical, and sensitive system. The city also didn't bother to insure that the system was safe from being locked out in that manner.

    IMO: if Childs goes to prison, the city's IT managers should go with him.

The meat is rotten, but the booze is holding out. Computer translation of "The spirit is willing, but the flesh is weak."

Working...