Forgot your password?
typodupeerror
United States Google Government Medicine Oracle Red Hat Software IT

Tech Titans Oracle, Red Hat and Google To Help Fix Healthcare.gov 404

Posted by samzenpus
from the with-a-little-help-from-my-friends dept.
wjcofkc writes "The United States Government has officially called in the calvary over the problems with Healthcare.gov. Tech titans Oracle, Red Hat and Google have been tapped to join the effort to fix the website that went live a month ago, only to quickly roll over and die. While a tech surge of engineers to fix such a complex problem is arguably not the greatest idea, if you're going to do so, you might as well bring in the big guns. The question is: can they make the end of November deadline?"
This discussion has been archived. No new comments can be posted.

Tech Titans Oracle, Red Hat and Google To Help Fix Healthcare.gov

Comments Filter:
  • Answer: No. (Score:5, Insightful)

    by Anonymous Coward on Thursday October 31, 2013 @07:30PM (#45295715)

    Nine women cannot make a baby in one month.

    • Re:Answer: No. (Score:5, Insightful)

      by BasilBrush (643681) on Thursday October 31, 2013 @07:34PM (#45295765)

      Nine women cannot make a baby in one month.

      True, but the website already exists. If it's a case of fixing defects rather than re-architecting from scratch, there's no reason why multiple teams can't work on different parts of the system. And multiple people within a team can't work on different defects.

      Defect fixing is indeed somewhat scalable.

      • Re:Answer: No. (Score:5, Insightful)

        by dmbasso (1052166) on Thursday October 31, 2013 @07:38PM (#45295825)

        It all depends on the quality of the existing code base. More often than not, it's better to start from scratch.

        • Instead of fixing a bunch of hopeless code, why can't they start over the damn thing - with a properly designed paradigm ?

          • by Runaway1956 (1322357) on Thursday October 31, 2013 @10:33PM (#45296921) Homepage Journal

            That's my idea. Government hired an incompetent contractor to build something. They built a freaking MESS. Just clear it all out. Sure, examine the code, see what the ideas were when they built the site. Take the best ideas, and rebuild the ideas, from the ground up.

            Years ago, I was called in to a construction job, where the previous foreman had really screwed up. He built a foundation and wall in the wrong place. We didn't try to make the wall fit into the plan - we wrecked the frigging wall, poured a new footer, and built the wall on top of our new footer.

            The site designers need to do the equivalent. Consider the "blueprint", see where everything went wrong, tear out the screwups, and build from the ground up. If that should happen to mean that not one single line of code remains, then so be it. If it means that 1/4 or 1/2 or even 3/4 of the code can be reused - fine. Just get it working. And, do it for less than another half billion freaking dollars!!

        • Re:Answer: No. (Score:5, Interesting)

          by recharged95 (782975) on Thursday October 31, 2013 @11:28PM (#45297177) Journal

          But all 3 companies listed will have those rock stars that will:
          a. look at the code and call is rubbish.
          b. ask to rewrite the whole thing
          c. charge an arm and a leg to do it within time.
          d. run it under agile (so THEY control the requirements, not the domain experts).

          Really they should have hired the guys that do turbotax and such.... it works for the type of users on this healthcare system. The above 3 will struggle through it as well... but will milk it for all it's worth.

          All I say to the Obamacare management team & Obama: TAKE A STEP BACK, WAIT.... ASSESS THE PROBLEMS one by one, THEN HIRE THE RIGHT FOLKS. This is a knee jerk reaction and will go down in flames. Of course, the valley and wall street is loving it....

          Young MBA folks: this is your Y2K computer problem moment. Remember those times: the panic, the flooding of cash, and nothing happened afterall? Yeah, get ready for another internet boom/bust.

          • by mysidia (191772)

            b. ask to rewrite the whole thing

            Seeing as it's Oracle, Redhat, and Google.... the application will probably be:

            Rewritten to run on Oracle Java, throwing away that old Visual-Basic code.

            Leverage Google AppEngine and BigTable for data storage, instead of the Microsoft Access-based backend

            Run on 64-bit Redhat Enterprise Linux servers, instead of 32-bit Windows 2003 and XP servers running IIS

            In other words..... it ought to be a smashing success

          • by ctishman (545856)
            I was under the impression that nothing happened because they poured a lot of money into fixing old systems for 4-digit years before they screwed up, not that nothing would have happened if they'd let it be.
      • Re:Answer: No. (Score:5, Insightful)

        by MightyMartian (840721) on Thursday October 31, 2013 @07:44PM (#45295873) Journal

        The last time I had to "re-architect" an existing website, I ended up putting in roughly twice the amount of time as the original "architects" (and I use that word very very loosely). Believe me, there's a lot of shit out there that will require a lot more effort to fix than originally went into building it.

      • Re:Answer: No. (Score:5, Insightful)

        by sexconker (1179573) on Thursday October 31, 2013 @07:49PM (#45295921)

        there's no reason why multiple teams can't work on different parts of the system

        You've never worked on anything with multiple teams working on different parts, have you?
        It never fucking works. You need knowledgeable oversight.

        • Re:Answer: No. (Score:5, Insightful)

          by MightyMartian (840721) on Thursday October 31, 2013 @07:58PM (#45296019) Journal

          Exactly. I think the Linux kernel is a damned good example of how a large number of developers working in very different kinds of development environments, some working in side-projects like Netfilter, are coordinated by one guy intimately acquainted with the kernel.

          You can say what you like about Linus's attitude at times, but the fact that the Linux kernel is running on everything from supercomputers to be Nexus 7 tablet tells you that there is a way to successfully and productively organize multiple teams to produce a successful software product.

      • Re:Answer: No. (Score:5, Informative)

        by kimvette (919543) on Thursday October 31, 2013 @09:24PM (#45296587) Homepage Journal

        > True, but the website already exists. If it's a case of fixing defects rather than re-architecting from scratch, there's no reason why multiple teams can't work on different parts of the system. And multiple people within a team can't work on different defects.

        You are assuming that there is a detailed (and accurate) functional spec, design spec, and that the code is organized and well-documented - and that it is architected in such a manner that throwing more engineers at it will actually fix the problem. More often than not, that is not the case.

      • More likely than not, there are likely components which will have to be entirely rewritten. While many of the bugs are trivial defects, it looks like in many areas, the design is just inherently flawed at the root. Particularly seeing the performance issues, I can't help but believe that it's just fundamentally architected in a very poor way, and while there might be quick hacks to at least get it standing, it won't really be functioning properly without seriously ripping out the internals.

        But let's take a

    • by Virtucon (127420) on Thursday October 31, 2013 @07:35PM (#45295771)

      No but I heard 18 Women can do it in two weeks. The guy from Infosys told me so.

    • Re:Answer: No. (Score:5, Insightful)

      by turkeydance (1266624) on Thursday October 31, 2013 @07:38PM (#45295823)
      the Mythical Man Month returns
      • by Jstlook (1193309) on Thursday October 31, 2013 @08:03PM (#45296077)
        My wife saw that book on my shelf last night and asked if it was related to a man's period. I had to chuckle.
      • .... with a vengeance. And this time, its personal .... health insurance that's at stake.

        At least the stakes are low. No worries.

        Obama Officials In 2010: 93 Million Americans Will Be Unable To Keep Their Health Plans Under Obamacare [forbes.com]

        • Re:Answer: No. (Score:5, Insightful)

          by Anonymous Coward on Thursday October 31, 2013 @08:42PM (#45296357)

          That article is so full of contradictory statements it's ridiculous. Which isn't to say I'm defending the excessively sugar-coated defenses the administration made in 2010. But 94 million is an upper limit, and it's mostly composed of private insurers and private companies purposefully choosing to change coverage, not because the law mandates it.

          And let's not forget about the 20-40 million people who will be unable to keep their lack of insurance coverage. What's the difference between being uninsured and underinsured? Maybe I should be allowed to get a car insurance policy with a $100 limit. I mean, freedom, right?

          If you want to diss the ACA, then diss it on its merits.

          I hate taxes as much as the next guy. More, in fact. My combined income is over $240k/year, almost all earned income, so its taxed heavily. It's a gigantic bitch. But you know what? I grew up in poverty, in foster homes. I benefited from a safety net. And the elder members of my family all depend on some sort of government assistance. So I just suck it up, because as the extremely conservative Justice Holmes once said, taxes are the price of civilization. And this civilization let's me make almost a quarter of a million per year. You think I could make that in Mexico, Brazil, or China?

          The penalties for having no insurance are is like $150/year. If you can't afford that, then you have bigger problems--and in any event, if you couldn't afford it the government would pay for it.

          • Re: (Score:3, Informative)

            by cold fjord (826450)

            Taxes may very well be the price of civilization, but what those taxes are spent on may be efficient and valuable, or destructive and wasteful. They can build bridges that are needed, and in a useful place, or expensive bridges to nowhere. The ACA is proving to be badly thought out, badly implemented, justified by lies, and seems to be headed towards being a train wreck for the American people, the economy, the healthcare industry, and even the Democratic party. It is already driving many jobs out of the

      • the Mythical Man Month returns

        It never went away.

    • Re:Answer: No. (Score:5, Insightful)

      by icebike (68054) on Thursday October 31, 2013 @07:54PM (#45295977)

      Nine women cannot make a baby in one month.

      But I bet even one woman could spell cavalry, and know the difference.

      Slashdot editors wanted. No Experience needed. We wouldn't know what to do with experience if we tripped over it.

    • Re:Answer: No. (Score:4, Insightful)

      by erroneus (253617) on Thursday October 31, 2013 @08:43PM (#45296365) Homepage

      Interestingly said.

      Even if by some miracle, they bring something up, it doesn't fix the actual problems. Ridiculously increased rates, it's a new tax on everyone, lies about keeping one's old policy and a general over-all burdon on the remaining who are employed above the poverty line.

    • by DragonTHC (208439)

      That's a valid analogy for babies.

      One person could have designed, programmed, and coded the whole site, including back end in about six months, If that person were skilled.

      Instead, HHS and CMS paid multi-million dollar contracts to 3 foreign corporations, who had a year and still couldn't do it.

      The site was doomed by salespeople and politicians.

      QSSI, who got the contract for the EIDM in 2012, evidently got it working for medicare and medicaid, but this site wasn't even coded or tested right.
      CGI Federal exec

  • by Cornwallis (1188489) on Thursday October 31, 2013 @07:31PM (#45295729)

    Our Gov is finally "out of patience" with Vermont's site (built by the same CGI that did such a bang up job on the Fed system: http://www.burlingtonfreepress.com/article/20131031/NEWS03/310310034/Governor-Peter-Shumlin-Web-woes-prompt-changes-to-Vermont-health-reform [burlingtonfreepress.com]

  • Calvary? Really? (Score:2, Insightful)

    by Anonymous Coward

    I think it's cavalry.

  • Amazon (Score:5, Interesting)

    by qzzpjs (1224510) on Thursday October 31, 2013 @07:32PM (#45295737)

    I think they should have just listed the plans on Amazon. Almost everyone already knows how to buy stuff from them and their servers would have handled it.

  • by Anonymous Coward

    bombing the hell out of it!

  • Let's see.. (Score:5, Funny)

    by Virtucon (127420) on Thursday October 31, 2013 @07:37PM (#45295799)

    In two months the site will be using Oracle and Ellison will charge the Feds a fortune for the license fees.
    Google will start mining every piece of data it can get off the website, of course the NSA will be stealing that and stashing it in Utah.

    Red Hat will push it all to RHEL which isn't necessarily a bad thing.

    • In two months the site will be using Oracle and Ellison will charge the Feds a fortune for the license fees.....

      Yea you gotta believe Larry is foaming at the mouth at this. Selling them a named-user license and getting paid for every man, woman, and child in the US? Cha-Ching!!!

  • Red Hat? (Score:4, Funny)

    by Anonymous Coward on Thursday October 31, 2013 @07:37PM (#45295807)

    No Microsoft? lol :)

  • Just kidding.
    • Re:Oracle! YES!! (Score:4, Interesting)

      by jbengt (874751) on Thursday October 31, 2013 @08:01PM (#45296049)
      I've had the misfortune of needing to use an Oracle system with a web interface to deal with a large client for construction management & billing. If that experience is any indication of how Oracle will fix the problem, the Feds would be better off keeping the very crappy existing system. (seriously)
      • Re:Oracle! YES!! (Score:4, Interesting)

        by FlyingGuy (989135) <flyingguy AT gmail DOT com> on Thursday October 31, 2013 @09:57PM (#45296779)

        One of the many problems is that most people do not know how to tune Oracle. Properly tuned Oracle, even when running on inadequate hardware, oracle can support TPS levels that many DB's only dream about with full ACID as a matter of course on the same hardware. I have watched Postgres, MS-SQL Server and DB2 just hit the floor while Oracle kept chugging right along, not always mind you, but more often then not.

        I am currently running 11gR2 on hardware that is at best adequate and can assimilate the entire output of 80% of the state of California's highway loop detectors ( approximately 50,000 raw data rows inserted every 30 seconds 24/7/365 ) and that into a rather poky 15TB drive array with 7500rpm 2TB drives, in raid 5 no less, then query all of that data filter,clean and analyze it and shove that data into another table all in the same 30 second period.

        The DMV project was a nightmare of never ending changes of requirements. When you think about the basic project, it aint that hard, but when there is no point at which you could say it was stable because the target just kept moving, I don't care who takes it on or who's DB engine you throw at it, it will fail.

        When it comes to scaling something out, you take you best guess at what you load will be. When your prospective load might be a large percentage of 300 million people it is a hard target to pin down and that is what ( along with a few bugs that escaped unit testing ) was their ultimate undoing. No one knows who's DB engine was behind it but I doubt it was any of the "web scale" DB's since they don't support ACID very well and this was one of those when it was absolutely essential.

        • 1,700 inserts per second is pretty easy for any modern RDBMS on decent hardware. Here's PostgreSQL doing 14,000 per second [blogspot.com] on a laptop, for example. My company routinely handles 40,000 inserts per second into Cassandra on midrange AWS virtual servers.

          Properly tuned Oracle is probably pretty quick, but these days so is properly tuned everything else.

  • Oracle? Seriously? (Score:5, Interesting)

    by jcr (53032) <jcr@NoSPaM.mac.com> on Thursday October 31, 2013 @07:45PM (#45295885) Journal

    I guess nobody in the decision making loop heard about Oracle's big California DMV fuck-up.

    -jcr

    • Or that Oracle already built a failed exchange website in Oregon [oregonlive.com].

      At the same time, it's kind of entertaining to watch the general public start to grapple and become aware of the same project management issues I've had to deal with for the last decade.
  • Brooks Law (Score:5, Insightful)

    by mccrew (62494) on Thursday October 31, 2013 @07:49PM (#45295919)
    Brooks Law [wikipedia.org] states "adding manpower to a late software project makes it later".
    • Brooks Law states "adding manpower to a late software project makes it later".

      +6 man, +6. That is exactly what first came to mind when they went for that "surge" mentaphor.

      Second thing that comes to mind is that the surge didn't work, it just happened to coincide with a change of local Iraqi politics (locals got sick of extremists killing locals instead of just americans so they started outing the extremists so the americans finally knew who to kill).

    • It's rather amazing how many quotes from Mythical Man Month apply to Healthcare.gov.

      "Failure to allow enough time for system test is peculiarly disastrous. Since the delay comes at the end of the schedule, no one is aware of schedule trouble until almost the delivery date."
      "False scheduling to match the patron's desired date is much more common in our discipline than elsewhere."
      "Take no small slips. That is, allow enough time in the new schedule to ensure that the work can be carefully and thoroughly do
  • I, for one, am glad to see government doing something right. They have fallen short of privatizing the site, but....

    Will the three tech giants also teach Economics?

    • by Skapare (16644) on Thursday October 31, 2013 @08:17PM (#45296205) Homepage

      The government should have done it in-house, using directly hired citizens as developers and project managers. Use top developers that fully understand the selected technology. This site is something that will be changing a lot over many years, so continued staff where most developers already know how it's built would keep it upgraded.

      • That's the kind of idea that sounds great until you get to the details. Who actually employs them, how do they get hired, who watches over the project managers as a stakeholder?

        The reason contractors get used is they offload all of these problems.

        "I know someone who is employed by the government therefore they can hire people directly." "My brother works in IT, they can just shore up the team and have them do the website."

        No, these do not work. Adding infrastructure to handle these employed people is an

  • The choice of these companies makes it obvious that it is not an asp.net fix. Being from Canada I have no idea what front end the site is using in the first place. But if it is not a based upon Microsoft style asp.net in the first place then you can bet that the choice of who gets government contracts will be effected in the future.

    Here in Canada the government has completely sold out to Microsoft and in some cases if you need to access government services on the net it is all coded in asp.net especially t

  • huh? (Score:5, Insightful)

    by Charliemopps (1157495) on Thursday October 31, 2013 @08:10PM (#45296137)

    I can understand Google and Redhat... but Oracle? Talk about having a fox in the hen-house.

    • Someone must not realize that Oracle DB is the go to database for big government projects. As this was a "big government project" to begin with, Oracle DB was probably used as the backend in the existing design.
  • by liwee (3407373) on Thursday October 31, 2013 @08:15PM (#45296185)
    Enlisting JUST ONE of the tech giants would be more productive.
  • Why does everyone in the country need to use the website at once? Couldn't the problem be fixed with a little javascript function:

    1. Enter your Social Security Number
    2. Based on your Social Security number, your enrollment date is 1-Nov-2013 - 7-Nov-2013 or anytime after 31-Dec-2013. If you do not know or do not have an SSN, your enrollment date is after 15-Jan-2014. Click here to have an email reminder sent on your enrollment date.

    They could instantly cut the website demand by 90% by dividing enrollments

    • by tftp (111690)

      They could instantly cut the website demand by 90% by dividing enrollments up by the last digit of the SSN of the primary enrollee.

      There aren't enough people as it is to pay double and triple for health plans that they don't need. I, personally, have no desire to even visit "that website," whatever URL it may have. I can pay for my own healthcare without involvement of moneychangers.

      • by hawguy (1600213)

        They could instantly cut the website demand by 90% by dividing enrollments up by the last digit of the SSN of the primary enrollee.

        There aren't enough people as it is to pay double and triple for health plans that they don't need. I, personally, have no desire to even visit "that website," whatever URL it may have. I can pay for my own healthcare without involvement of moneychangers.

        Unless, of course you suffer a catastrophic illness or injury. I know someone whose husband slipped while getting out of the shower, he hit his head on the floor, and ended up with a brain injury and needing brain surgery and months of rehabilitation. So far it's cost over half a million dollars. He was in his 30's, a triathlete in perfect health. Fortunately, he had insurance and his wife was able to take 3 months leave to care for him and can support the household on her income.

        Few people can afford a $50

  • Google?.... (Score:5, Funny)

    by tooyoung (853621) on Thursday October 31, 2013 @08:32PM (#45296289)
    Crap, now the NSA will have a backdoor into the government!
  • And we thought it was expensive and past deadline NOW.

  • Google will spend all of their time working on tracking users, Oracle will insist on integrating dozens of Oracle products costing hundreds of millions of dollars, and RedHat will rewrite the system while removing the capability of running it over a network.

    But in all seriousness, the reason this web site is in shambles is because the developers weren't given nearly enough time to implement a product this complex. And if years of development wasn't enough time, the government thinks that a few big tech
  • by goosebane (740956) on Thursday October 31, 2013 @08:49PM (#45296407)
    Alright, who is getting crucified over this one?
  • Webserver? Middleware? Database? Hardware??
  • I have. It's not that bad. Really.

    Now I don't need insurance as I already have it from my employer, but I was curious how bad the site was. But it didn't turn out being difficult or error prone at all to sign up. It took about 15 minutes total and I had the eligibility report for me and my daughter. Some nit picks:

    1) The confirmation email was one of three emails i got from healthcare.gov when signing up. That could confuse some people.
    2) One required field on one page was scrolled off the bottom, and no scroll bar appeared to indicate that. Mouse wheel scrolling down solved that, but if there are many pages with that problem it could be confusing.

    That's about it. Maybe I just lucked out, bit it was an easy site to use.

"Just think of a computer as hardware you can program." -- Nigel de la Tierre

Working...