Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
×
Security United States IT

Disgruntled Engineer Hijacks San Francisco's Computer System 1082

ceswiedler writes "A disgruntled software engineer has hijacked San Francisco's new multimillion-dollar municipal computer system. When the Department of Technology tried to fire him, he disabled all administrative passwords other than his own. He was taken into custody but has so far refused to provide the password, and the department has yet to regain admin access on their own. They're worried that he or an associate might be able to destroy hundreds of thousands of sensitive documents, including emails, payroll information, and law enforcement documents."
This discussion has been archived. No new comments can be posted.

Disgruntled Engineer Hijacks San Francisco's Computer System

Comments Filter:
  • Backups? (Score:5, Funny)

    by anonieuweling ( 536832 ) on Tuesday July 15, 2008 @07:53AM (#24194381)
    With backups no data will be lost. Oh, those are encrypted?
    • Re:Backups? (Score:5, Insightful)

      by shbazjinkens ( 776313 ) on Tuesday July 15, 2008 @07:55AM (#24194409)
      Or they could just unplug it? Lost productivity is better than lost data here, I'll bet.
      • Re:Backups? (Score:5, Insightful)

        by Harmonious Botch ( 921977 ) * on Tuesday July 15, 2008 @09:34AM (#24196035) Homepage Journal

        Productivity? By a government agency?

        This is not about productivity, it is about control.

    • Re:Backups? (Score:5, Insightful)

      by Brian Gordon ( 987471 ) on Tuesday July 15, 2008 @08:11AM (#24194635)
      I don't understand how it's possible to be locked out of a system that you have direct local access to. You should at least be able to pop in a livecd and edit /etc/password from a livecd. If you need to decrypt stuff might as well start cracking the hash.. they certainly have the computing power to do it o_O
      • Re:Backups? (Score:4, Insightful)

        by cboscari ( 220346 ) on Tuesday July 15, 2008 @08:18AM (#24194755)

        Are you sure it's a UNIX variant? I assumed it was big iron, and I am not sure those have cd-rom drive. What's more, if he choose a REALLY good password, brute force decrypt might take a *long* time...

      • Re:Backups? (Score:5, Insightful)

        by azrider ( 918631 ) on Tuesday July 15, 2008 @08:57AM (#24195353)

        I don't understand how it's possible to be locked out of a system that you have direct local access to. You should at least be able to pop in a livecd and edit /etc/password from a livecd.

        That gets you into the operating system. Once you are there, what do you do? SQL databases can/should use passwords.
        Web servers can/should use passwords.
        Payroll systems MUST use passwords, with all data encrypted.
        The above (and others) are where the problem lies, and no single user reboot will fix this.

        • Re:Backups? (Score:5, Interesting)

          by TheLink ( 130905 ) on Tuesday July 15, 2008 @09:20AM (#24195765) Journal
          The only problem is if encryption was used AND he hasn't left an open session somewhere which you can somehow get access to.

          If the data is not encrypted it doesn't matter if the SQL DB uses passwords or not. Same for the webserver and other stuff.

          I've patched programs stored in a DB without knowing the DB admin password, just by hexediting the DB files. Didn't have to wait for the vendor's developers in the USA to get back to us ;).

          As long as you have read access to the unencrypted data you have enough access - even if it means changing the drives and reloading the data.
      • Re:Backups? (Score:4, Insightful)

        by spydum ( 828400 ) on Tuesday July 15, 2008 @09:33AM (#24196009)
        For what it's worth, the guy is a network engineer, I'm assuming these are switches and routers. You don't boot them off a CD. Resetting the password on some of these devices is made possible only by resetting the config. If nobody kept proper config backups, you would have a hard time reconfiguring the device from scratch.
  • This is why... (Score:5, Insightful)

    by Gallenod ( 84385 ) on Tuesday July 15, 2008 @07:53AM (#24194387)

    ...you disable his account *before* you tell him he's fired.

    • Re:This is why... (Score:5, Insightful)

      by Televiper2000 ( 1145415 ) on Tuesday July 15, 2008 @07:57AM (#24194433)
      I was just about the say the same thing. You also escort them directly out of the building and let them pick up their personal things a week later.
      • Re:This is why... (Score:5, Insightful)

        by damburger ( 981828 ) on Tuesday July 15, 2008 @08:01AM (#24194489)
        Is holding his possessions captive in such a way legal? Its certainly arseholey.
        • Re:This is why... (Score:4, Interesting)

          by zr ( 19885 ) on Tuesday July 15, 2008 @08:09AM (#24194613)

          fedex it. nothing at workplace is private from employer.

      • Re:This is why... (Score:5, Interesting)

        by Anonymous Coward on Tuesday July 15, 2008 @08:43AM (#24195131)
        My employer doesn't fire anyone... they just lay them off, with some amount of severance. That way the person has money and can get EI (Employment Insurance - we're in Canada and like to make unemployment seem nicer than it is), and is less likely to try to sue the company for wrongful dismissal or tell everyone about the shady things the company does.

        The employee is usually taken to one of the front meeting rooms under the pretense of an "important staff meeting". As soon as they leave their desk, someone swoops in and piles everything not owned by the company into a box, and takes it to reception. The employee gets their dismissal meeting from their direct boss with someone from HR present, and then they're taken to reception, given their box of stuff, and told to GTFO.

        Network Operations gets the call to reset the ex-employee's password so they can't get in through the VPN (have to keep their account so someone can answer their email, etc), and work goes on.

        The last thing the ex-employee gets to see on the way out is the hot receptionist. Could be worse.

        Sorry for posting anonymously, but I don't feel like getting laid off if someone from work happens to recognize my username.
        • by BigDaddyOttawa ( 948206 ) on Tuesday July 15, 2008 @09:32AM (#24195991) Homepage
          Paul, is that you? Could you come to Meeting Room 1 for an important staff meeting. Ignore John standing behind you with that box, he's just collecting them to build a fort.
        • by phorm ( 591458 ) on Tuesday July 15, 2008 @09:52AM (#24196349) Journal
          It's just not that easy for a sysadmin, especially a major one. For myself, I've got passwords, SSH-keys, and many other access points everywhere in my company. It's not because I want to screw with them, but because they tend to call me at all sorts of different times and I never know if I'll need secure access to the server.

          So, routing rules from home. Public SSH keys on various border-servers with my USB-drive having the private keys, etc. They're all used for doing my job, and if I'm fired (not sure why I would be though) I'll just move on to the next one without tainting my career and doing something stupid to burn bridges. However, I could see a *bad* sysadmin using these same tools and more to entrench himself so deeply that you'd almost have to rebuild the entire infrastructure from scratch to find all the back-doors.

          If this guy was a real dick (but a clever+smart one), knew it, knew he was going to be canned, and prepared for it... then how are you going to know that your authentication methods, your binaries, or even your kernels haven't been messed with in some way? MD5 sums only go so far when you have hundreds of systems tied together.
    • RTFA (Score:4, Informative)

      by tomhudson ( 43916 ) <barbara DOT huds ... a-hudson DOT com> on Tuesday July 15, 2008 @08:01AM (#24194495) Journal

      He was arrested AFTER he disabled everyone else's account.

      What do you recommend they do next time, use a crystal ball or ouija board to predict who's going to pull such a stunt?

    • by chipmeister ( 802507 ) on Tuesday July 15, 2008 @08:02AM (#24194505)
      There was an unsuccessful attempt to fire him. The article also mentions that he was essentially spying on people to learn things being said about him.
      • by Anonymous Coward on Tuesday July 15, 2008 @08:53AM (#24195291)

        I've seen this sort of problem...it's really deadly. If you have somebody who has the keys to the entire computer system, is fully willing to snoop into people's personal data, and also is willing to really do some nasty things, you're in a bad situation. If you're going to fire him, do it fast and without warning...he absolutely can't know it's coming. With someone like that, you can't even discuss the issue via email with any other colleagues (i.e., he's probably reading your emails quite regularly).

        If he has any time to stew about things, then odds are he'll setup a variety of back-doors or other ways he can royally mess things up. In the situation I've seen, the boss knew the sysadmin was screwing around...though there was no hard proof, the sysadmin also knew that he was essentially caught. But in his position, he basically had the office by the balls. It's a stalemate...unless you're willing to dump the guy and completely sanitize/overhaul anything he's touched on the network. And of course, who knows how much personal data he's copied off-site in the meantime.

        Gotta post as A/C for this one...

      • by thelexx ( 237096 ) on Tuesday July 15, 2008 @08:53AM (#24195293)

        Well, if they had nothing to hide then they have nothing to worry about right?

    • by Swizec ( 978239 ) on Tuesday July 15, 2008 @08:03AM (#24194517) Homepage
      Is what I say ...
    • by martin-boundary ( 547041 ) on Tuesday July 15, 2008 @08:15AM (#24194697)
      Nah, they should just reboot the system. That always works, I've seen it countless times in movies.
      • by Rocketship Underpant ( 804162 ) on Tuesday July 15, 2008 @09:36AM (#24196065)

        Yes, but that involves a perilous trip through the cavernous sub-basement to some rarely touched master reboot switch, and while the system is restarting all the perimeter fences will be de-electrified and the motion sensors inactive. In movies, this situation inevitably leads to lots of screaming and mayhem.

    • Re:This is why... (Score:4, Insightful)

      by Anonymous Coward on Tuesday July 15, 2008 @08:18AM (#24194753)

      Except a lot of times someone is fired they know that's it's coming. It's possible this guy had set this all up in the case he got fired, and then we he saw it was going to happen he put it into motion. Article even says they tried to fire him before and he created his super password as a security device to keep his job. Now I'm sure the real irony here is that if this guy probably actually did his job instead of all this mess he probably wouldn't have been fired. I mean, this is a guy that's going to be looking at pretty serious jail time, and probably a severe restriction on his rights when he gets out. I like my job, but not enough to do something that's going to land me in the pokey.

    • by scuba_steve_1 ( 849912 ) on Tuesday July 15, 2008 @08:44AM (#24195153)

      Firing someone for poor performance (as opposed to firing someone for a single unacceptable action) takes time....and MUCH coordination...at least everywhere that I have worked.

      In a decently managed environment, the employee knows in advance that his management views his/her performance as unacceptable since the manager has discussed it with the employee and laid out a plan for improvement. Even an average employee could see the writing on the wall weeks/months in advance...but this individual was also using his administrative access to monitor related email messages.

      If his group comprised even a moderately-sized MIS group, you could pull his admin responsibilities and transfer him to a role with lesser rights during the period of performance review and monitoring...but this individual was most likely hired to do this very specific job...and there may not have been another position in to which he could transition naturally...even temporarily.

      My question - where are the backup tapes? Pull the tapes from a date prior to his manipulation of the system. Presumably, it should not be that long ago if they were ensuring that at least one other admin had routine access to the system. In such a case, they should have known within 24 hours that he had done something. If, on the other hand, he was a one man show, then I think that they are screwed until he gives up his password...which he will. Mark my word.

  • by dunelin ( 111356 ) on Tuesday July 15, 2008 @07:53AM (#24194391)

    Next thing you know, we'll have some dinosaurs on the Presidio.

  • Countdown... (Score:5, Insightful)

    by geminidomino ( 614729 ) * on Tuesday July 15, 2008 @07:58AM (#24194441) Journal

    Idiotic new law in 5...4...3...

  • Tried to fire him? (Score:4, Insightful)

    by OzPeter ( 195038 ) on Tuesday July 15, 2008 @07:58AM (#24194455)
    From TFA:

    "Childs has worked for the city for about five years. One official with knowledge of the case said he had been disciplined on the job in recent months for poor performance and that his supervisors had tried to fire him."

    How the hell do you "Try to fire" someone .. either you do it or you don't.

    (And please .. no Yoda BS. If you go back and look at when Yoda was first introduced as a character he didn't do that cutesy backwards sentence construction. That came later. So I put it in the realm of Jar Jar - obnoxious character development)

  • by Shivetya ( 243324 ) on Tuesday July 15, 2008 @08:03AM (#24194515) Homepage Journal

    Especially when it makes a crime a Felony. That is one of the four felonies charged to him. The other three are all related to tampering with a computer network.

    While this guy is obviously an idiot for thinking he could blackmail a government entity I am quite pleased the security on the system is sufficient to make it hard to get into when strong security is put into place. In other words, nothing annoys me more than so called secured systems having some means of password decryption, let alone the ones that allow admins to see them plain text.

    what is going to interest me is how many years they will attempt to land on him. Just how offensive to society is this type of crime versus murder or rape. It seems that every new crime invented by the government gets stronger penalties than existing ones; if only to make it appear more valid. After all the penalty wouldn't be so severe if it were not really a crime now would it?

  • Job Posting (Score:5, Funny)

    by Anonymous Coward on Tuesday July 15, 2008 @08:04AM (#24194531)

    Large municipal department of technology seeking software engineer for a multimillion-dollar computer system. At least 5 years of previous experience required. Must be able to gain administrative access to a system where the password is not known. Hiring immediately!

    • by Chibi ( 232518 ) on Tuesday July 15, 2008 @09:42AM (#24196179) Journal
      If they (the technology department) were smart, they would make it a practical interview. Ask the interviewee if they can gain administrative access to the system. If they say yes, let them try. If they can't do it, you thank them, but let them know that they aren't qualified for the position. If they *can* gain access, you thank them, and let them know that the position is no longer required.~
  • ok, you're mad at your employer, perhaps there reasons for firing you are invalid

    but taking it out on third parties, such as with locking up law enforcement documents that might decide the guilt of hardcore criminals: you're a selfish asshole for setting up that scenario

    maybe you didn't deserve to be fired

    but now you deserve to rot in jail for how you responded to your firing

  • I smell a rat (Score:5, Insightful)

    by stinky wizzleteats ( 552063 ) on Tuesday July 15, 2008 @08:10AM (#24194619) Homepage Journal
    FTFA:
    "At a news conference announcing Childs' arrest, District Attorney Kamala Harris was tightlipped about what his motive may have been."

    I think there's more going on here than we're being told.
    • Re:I smell a rat (Score:4, Informative)

      by Temkin ( 112574 ) on Tuesday July 15, 2008 @09:02AM (#24195463)

      FTFA:

      "At a news conference announcing Childs' arrest, District Attorney Kamala Harris was tightlipped about what his motive may have been."

      I think there's more going on here than we're being told.

      You have to understand the nepotism and corruption that runs SF. The DA is purportedly Willie Brown's ex-girlfriend. She probably hasn't been told what to say yet because her handlers have been locked out of their computers. They have to cover up the corruption that contributed to this (or was merely exposed) first, then they'll decide what he did and throw the book at him.

  • by Numen ( 244707 ) on Tuesday July 15, 2008 @08:10AM (#24194623)

    That director over there, he gets a golden handshake as he goes out the door... You want to keep him sweet because he knows where all your dirty secrets are and could cause all sorts of trouble for your operation.

    The sysadmin, youre going to kick out the door becuase hes blue colar... Oh, wait a minute... He really does know where all your dirty secrets are and really can bring your operation to its knees. In fact hes far more dangerous going out the door than the exec... pity you didnt think of that.

    Execs are heaved out the door all the time for being incompetent, but its done with kid gloves because theyre deemed to be potentially damaging... And they wear a suit.

    Word of advice: if youre sacking somebody who can bring your operation to a grinding halt, make sure you you keep them sweet, regardless of the job they do for your organisation. Its simple business.

  • by Anonymous Coward on Tuesday July 15, 2008 @08:11AM (#24194641)

    Thats why you run unpatched windows, it will take only 4 minutes to get access.

  • by FudRucker ( 866063 ) on Tuesday July 15, 2008 @08:11AM (#24194647)
    log in in init 1 (runlevel 1) and change the root password or;

    in /etc/shadow change this:
    root:$2$3bJ7DS4R$rV45lDlqNsfDRntfO1NCk0:14069:0:::::

    look exactly like this:
    root::14069:0:::::
    this and you can log in to root without any password

    maybe other *nixes are close enough to do the same (BSD or solaris)

    on ubuntu the root shadow is a little differrent since it is disabled with an asterisk:
    root:*:14069:0:::::
    just remove the asterisk
  • by 192939495969798999 ( 58312 ) <info&devinmoore,com> on Tuesday July 15, 2008 @08:14AM (#24194679) Homepage Journal

    From TFA: "Prosecutors say Childs, who works in the Department of Technology at a base salary of just over $126,000"

    No wonder he was disgruntled, that's not even a living wage in San Francisco.

  • Gruntled (Score:4, Insightful)

    by senor mouse ( 1227452 ) on Tuesday July 15, 2008 @08:16AM (#24194721)
    Poor soul. All pissy over a job that pays 150K/yr? This guy lacks perspective, huge. If incarceration and bankruptcy don't help him figure things out - perhaps a stint delivering pizza or a cardboard sign at the offramp.
  • by PinkyDead ( 862370 ) on Tuesday July 15, 2008 @08:19AM (#24194769) Journal

    because

    They're worried that he or an associate might be able to destroy hundreds of thousands of sensitive documents, including emails, payroll information, and law enforcement documents.

    Yes - that's the reason.

    Not because he showed up their complete incompetence and made them look like fools and now they want retribution. Protecting the public's right to privacy - yes, that's the reason.

  • by madcarrots ( 308916 ) on Tuesday July 15, 2008 @08:25AM (#24194841)

    None of us know all the facts of the situation, but I think it's pretty obvious that this guy was just trying to maintain his livelyhood through a misguided attempt at job security. If we had an IT Union looking out for our careers that gave us some sort of protection against the arbitrary whims of upper-management, then maybe this wouldn't have happened.

    As for the idea that the guy might have shared his password with some unscrupulous feind... how many of you, had you actually been given admin access to SAN FRANSISCO would really share that password with anyone? Drastic, misguided, sure... but stupid? Come on, there had to be a reason he got the job in the first place.

  • by bickerdyke ( 670000 ) on Tuesday July 15, 2008 @08:33AM (#24194941)
    "going municipal"?
  • Motive and Salary (Score:5, Interesting)

    by Jah-Wren Ryel ( 80510 ) on Tuesday July 15, 2008 @08:43AM (#24195127)

    Seems kind of funny that the article reports the DA is "tightlipped" about his motive. Makes me wonder if he is 'disgruntled' for a reason that would embarrass the agency if it got out.

    Also pretty funny that they go into great detail about his salary, which seems kind of low to me for the area or at least average. Sounds like they are trying to make him seem unsympathetic in the public eye.

  • Technical background (Score:5, Informative)

    by DF5JT ( 589002 ) <slashdot@bloatware.de> on Tuesday July 15, 2008 @09:00AM (#24195431) Homepage

    For those who wonder what kind of working environment DTIS has:

    PeopleSofts HRMS 8.x application software.
    PeopleTools 8.4x, PeopleCode, SQL, SQR, COBOL, Application Engine, Oracle and HP/UNIX.
    IBM hosts and DB2
    Microsoft SQL Server 2000

    Just look for open positions and you know what they are running.

  • Just stupid.... (Score:4, Interesting)

    by mlwmohawk ( 801821 ) on Tuesday July 15, 2008 @09:20AM (#24195777)

    I used to work at a bank. I was the "cash control teller" which means that I counted every single cash shipment into and out of the bank branch. Sometimes 1/2 million dollars.

    You know what? It isn't worth it. It isn't enough to live a good life on. If you get caught, the benefits do not out weight the risks.

    The same thing with this sort of hack. The guy screwed himself. He's ruined and will serve time in prison. "Everyone" (with any skills) knows you can get into any system you can physically touch.

    What is he going to get for his trouble? Will they pay him off and set him free? HA! no way. The worst that will happen is that they'll employ someone's 12 year old nephew to crack the system. Pay him off with a couple XBox games or a new PS3.

  • Unstable (Score:5, Insightful)

    by Sanat ( 702 ) on Tuesday July 15, 2008 @09:29AM (#24195933)

    Back in the 80's I had an analyst working for me that seemed to become more unstable as each day passed.

    We had a big project that he was working on and making great progress but then he started feeling like the software he created was his and not the company's.

    I talked it over with the regional VP as we did not have any reason to fire this guy but yet feeling more flaky with him all of the time.

    Plus replacing him would set the project back months.

    So I went in each evening (only lived a mile from the office) and made a backup of the files just in case.

    The project was successful and in retrospect making the backups kept me sane and kept the pressure off of him that he would feel if I was nervous or watching him too closely.

    It seems we attract those things we fear.

    Dealing with brilliant but somewhat unstable (supposedly) individuals is a tricky balance and occasionally the situation can tip in the wrong direction.

    Sounds like this case in SF tipped all the way.

  • by thc4k ( 951561 ) on Tuesday July 15, 2008 @10:25AM (#24196899) Homepage

    1. declare him a terrorist
    2. torture him
    3. ???? [redacted for national security reasons]
    4. password!

If money can't buy happiness, I guess you'll just have to rent it.

Working...