Forgot your password?
typodupeerror
Supercomputing The Almighty Buck

JPMorgan Rolls Out FPGA Supercomputer 194

Posted by timothy
from the rejected-rejected-rejected dept.
An anonymous reader writes "As heterogeneous computing starts to take off, JP Morgan have revealed they are using an FPGA based supercomputer to process risk on their credit portfolio. 'Prior to the implementation, JP Morgan would take eight hours to do a complete risk run, and an hour to run a present value, on its entire book. If anything went wrong with the analysis, there was no time to re-run it. It has now reduced that to about 238 seconds, with an FPGA time of 12 seconds.' Also mentioned is a Stanford talk given in May."
This discussion has been archived. No new comments can be posted.

JPMorgan Rolls Out FPGA Supercomputer

Comments Filter:
  • by fuzzyfuzzyfungus (1223518) on Monday July 11, 2011 @07:00PM (#36727726) Journal
    It fills the heart with inspiration to watch the best and brightest constructing advanced computers to solve the problems of mankind...
    • by khallow (566160)
      This is a special case of a trade problem. And trade, namely, deciding how to exchange valuable things with each other, is a big problem of mankind. So yes, my take is that it should fill your heart with inspiration.
    • by Bengie (1121981)

      you'll be glad to hear I have a 2TF super-computing chip crunching out massive frames-per-second as we read.

    • by Weaselmancer (533834) on Monday July 11, 2011 @09:26PM (#36728918)

      I would positively love to do something like this. The purpose of an engineer is to solve problems. That's what makes me happy at work. Solving problems. So here you have a very specific problem that required the construction of a custom computer made out of banks of FPGAs. Tell me that's not sexy! Who cares if it's for bankers. That is a damn nifty gadget to work on building.

      Imagine building it yourself. Switching networks, Linux on ARM cores peppered here and there coordinating and dumping program code to the FPGA banks, writing the drivers to grab the data once the run is completed...

      And at the end of the day a problem solved: What once took all day now takes a couple of minutes.

      This would have been a thoroughly nifty machine to work on.

      • by decora (1710862) on Monday July 11, 2011 @10:18PM (#36729194) Journal

        there have been articles about the computer guys who were working inside the CDO machines of wall street in 2000-2008 before the whole thing came crashing down.

        the only people who could hold their nose and not-care what their work was being used for are complete pscyopaths, who went through some kind of personality-cracking process so that they can act like normal people while they help destroy the planets economy.

        • It takes a special kind of pig-headedness to come up with a moral conclusion that doesn't allow for any doubt. The CDO guys could easily have concluded that what they were doing was useful to society. Or they could have decided that they weren't pious enough to come up with a completely solid moral case, just like we all do, all the time, with just about every decision we ever make, and said "well, I can't see any obvious harm" and gotten on with the work.

      • Re: (Score:2, Interesting)

        by Anonymous Coward

        Who cares if it's for bankers. That is a damn nifty gadget to work on building.

        Replace "bankers" with "the Third Reich" and you would no doubt find some IBM engineers in the 1930s thinking exactly the same thing about the computers they were building.

        Just because the technology is cool doesn't mean you're absolved of any moral or ethical dilemmas that may arise from working on it.

        • Tom Lehrer's "Wernher von Braun" song, as one satirical example of science being divorced from ethics.

      • by hjf (703092) on Monday July 11, 2011 @10:55PM (#36729434) Homepage

        Well, why don't you? The beauty if VHDL is that it's really, really basic. The rest is up to you to implement. Come join us in ##vhdl at irc.freenode.net or try fpga4fun.com. $100 should get you an FPGA training board which you can use to learn.

        • by RKBA (622932)
          Ba Humbug! Verilog is far more intuitive and easy to use than VHDL. Verilog is more akin to the C language; whereas, VHDL is more like (Ugh!) COBOL, or to be slightly more charitable -- ADA, the DOD's failed computer software language.
          • by chthon (580889)

            If you can't write a remote login shell in COBOL, or an application to organise your libraries, then you cannot program . Yes, I did both, on a platform (WANG VS) for which there was only a COBOL compiler installed. But I had access to all the OS calls through the compiler, so I could program what I wanted.

            And I haven't seen anything in the last 10 years after I stopped with COBOL, that surpasses the ease to make proper financial calculations with it.

          • by TeknoHog (164938)

            I can attest that Verilog is easy and intuitive, since it got me from a total newbie to a project that other people find useful, in a couple of weeks. Of course, some electronics and computing background helped a lot.

            My choice of language was partly influenced by the apparent verbosity of VHDL. It's the same reason why I preferred to learn Python instead of Java, for example. However, I also have the impression that VHDL is in some sense lower-level and more strictly defined, so you can have finer contro

          • ADA, the DOD's failed computer software language

            Ada failed? If you can write SPARK Ada, I can point you at several companies that will hire you at an insane rate for entry level positions. The language has no serious competitors for situations where it's actually important that the software works.

            • by s122604 (1018036)
              If you've ever flown in a 777 you've trusted your life to ada.

              The pilots control stick is nothing more than a computer entry device, every motion on the stick is interpreted by the software, which then actually tells the engines/ailerons/rudders/flaps what to do.

              You wanna do that in C, be my guest...
        • by ballpoint (192660)

          I never thought I'd hear beauty and VHDL in the same sentence, unless there was a negative operator in there too.

        • I've been advocating FPGA supercomputers where I work for a while, but the problem I hit is that the people I try to sell the idea to are systems guys who absolutely refuse to learn VHDL or Verilog coding. They want tools to turn their C and Matlab code directly into something they can download to the supercomputer. There's lots of work being done on this, but I've yet to see a turnkey solution because people are still building their own supercomputers.

      • The purpose of an engineer is to solve problems.

        Engineers solve real-world problems: when they make simplifying assumptions, they try to avoid transparent falsehoods like "a market functions with perfect transparency," and "all players in a market behave rationally." Hell, there's no way to even check the latter statement for truth: destroying a major company to build your relatively minor personal fortune may or may not be rational. If any solid engineering had been invested into the mortgage market of the past decade, the ultra-rich would not have cl

      • So here you have a very specific problem that required the construction of a custom computer made out of banks of FPGAs. Tell me that's not sexy!

        I do the same thing, but for things IN SPACE!

        Still not sexy, sorry.

    • The brightest heartless are inspired by the best computers to advance the problems of mankind.

      I eagerly await the next big bombing in NYC.

  • by seanadams.com (463190) on Monday July 11, 2011 @07:01PM (#36727736) Homepage
    That will fix our banking system for sure!
    • by blair1q (305137)

      It just accidentally moved the decimal point on your account two places to the left.

      But it did it hella fast.

      • Ok! Ok! I must have, I must have put a decimal point in the wrong place
        or something. Shit. I always do that. I always mess up some mundane
        detail.

    • That will fix our banking system for sure!

      I can't agree more. An increase in the number of golf matches and accelerated round-robin tournament configuration would go a long way to keeping those pesky bank officials occupied on the links and safely out of their offices. The Florida Professional Golfers' Association will benefit while also helping protect our nation's assets from executive malfeasance.

      ...

      Wait a minute, what are we talking about here?

      Cheers,

    • by Idbar (1034346)
      Well we all hope that at least their bubble crashes faster, so they get a bailout and we all move on sooner.
    • by syousef (465911)

      That will fix our banking system for sure!

      Banks for sure?

    • by c0lo (1497653)

      That will fix our banking system for sure!

      GIGO at tremendous speed.

    • High school stuff.

      Debt:
      http://en.wikipedia.org/wiki/Exponential_function [wikipedia.org]

      52 trillion dollars and counting.

  • A friend in the industry once remarked that some of the best and brightest in software engineering have been going into the financial industry as of late. It's hard not to wonder what they might have achieved in more productive areas of work...
    • Weep, USA, weep! (Score:2, Insightful)

      by Anonymous Coward

      This technological breakthrough is an important milestone on the downfall of the mighty USA. When the brightest of a country are engaged into a completely nonproductive activity and wastes important resources (in terms of education, knowleddge and to some extent money) to achieve nothing which benefits the society, it's a high-mark - it's all downhill from here.

      Let's come back here after 20 years and see how this comment stands up to time.

      Weep, USA.

      • It's not just the USA. I periodically get job offers from financial organisations in the UK, offering 5-10 times what I currently make. It's easy to see why it's difficult to recruit people for more productive occupations.

        If they didn't require me to relocate to the cesspit known as London, I'd almost be tempted, in spite of the soul-destroying nature of the work.

        • by tehcyder (746570)
          Well, OK, but what field do you work in? Because if it's anything commercial, it's pure snobbery to differentiate between (legally) making money at A or (legally) making money at B. If you're working on a cure for cancer at minimum wage, fair play to you.

          And there's nothing wrong wih London if you're earning a lot of money...
          • Well, OK, but what field do you work in?

            A variety of things. I do some work on HPC compilers that are used, for example, by people mapping the human genome and doing protein folding simulations to search for cures for various diseases. I write books. I'm currently involved with a project to develop the IT infrastructure in Tanzania.

            Because if it's anything commercial, it's pure snobbery to differentiate between (legally) making money at A or (legally) making money at B.

            Yes, because nothing commercial affects anyone else in the world, and so everything is entirely equivalent. Most of the projects that I work on - including some that I work on for free - have a net positive impact

    • For me, it was pretty straightforward: Money, and opportunity. Software companies here in Australia tend not to gamble so much on fresh graduates, so the majority of opportunities that were attractive financially were in the consulting (Accenture, IBM, Infosys) or Financial space. As a poor student coming out of university, when I'm given the offers of $45k/yr vs. $60k/yr, plus bonuses, it's a pretty easy decision. Work hours tend not to be onerous, workplace conditions good, and the majority are in the mai

      • there are people who live off the difference in those salaries.

        what did you buy with that exra fifteen grand? did you need it?

        • by Nursie (632944)

          Not that this justifies it, or that I disagree with you, but here in Australia the cost of living is vastly higher than the united states.

          The US and AU dollars are at rough parity, but it goes a hell of a lot further in the US.

        • Are you working for minimum wages? If not, why are you so greedy? Or even why are you employed instead of volunteering in a 3rd world country? [/sarcasm]
        • by nschubach (922175)

          It doesn't matter what (s)he buys with that extra $15k... it produces jobs to make the goods that they bought. Granted, those jobs may be Chinese jobs depending on what they are buying, but that's not for you to decide for the person making the money.

    • by tehcyder (746570)

      A friend in the industry once remarked that some of the best and brightest in software engineering have been going into the financial industry as of late. It's hard not to wonder what they might have achieved in more productive areas of work...

      I agree. Nationalise the so-called financial sector entirely and stop them inventing new derivatives-of-derivatives-of-derivatives, and just to stick to nice simple money going in and out of acounts. Then re-allocate the best minds into fields like medicine and space travel.

  • by PJ6 (1151747)

    The company also supports Excel and all different versions of Linux.

    Riiiight.

    • Not that hard. Excel, front of house. Linux, back of house.

    • FYI there's a crazy number of front-office traders running their own "programs" if you call their VBA that. Whatever you think of it, Excel is an easy way to custom-build financial calculations, as well as a simple front-end for displaying information. Usually the guys are too busy to turn their sheet into something more rigorous.

  • by ka9dgx (72702) on Monday July 11, 2011 @07:06PM (#36727792) Homepage Journal

    Information asymmetry makes even the fastest analysis on the planet irrelevant as the data input is garbage. It is this lack of transparency which resulted in the housing bubble, etc.

    The "too big to fail" banks regularly hide data from customers, regulators, and other branches of their own organization.

    This is interesting because of the speedup of FPGAs, but don't be fooled by a second that it addresses an actual business need, other than PR.

    • It is rather curious that a company of JP Morgan's size would choose to address risk with computers rather than lobbyists. It has already been demonstrated that enterprises who matter can simply play "heads I win, tails you lose" with the public purse.
    • by khallow (566160)

      Information asymmetry makes even the fastest analysis on the planet irrelevant as the data input is garbage.

      Even garbage data tends to have real information hidden in it. And most financial data tends to have a quite a bit of real information in it.

      Plus, what side of the "information asymmetry" is JP Morgan on? I bet they tend to have the advantage.

    • Human stupidity, greed and arrogance resulted in the housing bubble & subsequent crash. Banks such as Goldman Sachs and Deutsche Bank who had clever people that saw the whole thing coming got out while the going was good and made money on the way down - powered by intelligence, greed and arrogance. Transparency had nothing to do with it.

      • by decora (1710862) on Monday July 11, 2011 @10:23PM (#36729230) Journal

        if Morgan Stanley hadn't got bailed out by a Japanese bank, and if Bank of America hadn't bought Merrill, then Merrill would have failed, the Morgan, and Goldman and JPM would have fallen because of it.

        Goldman's credit default swap business with AIG was also basically 100% bailed out by the taxpayer. Goldman would have lost massive amounts of money if it hadn't been for the deal the government gave them when it took over AIG.

        .

  • ...starbridge wants their concept back.

  • This is a good idea. A hardware [austinchronicle.com] implementation of a risk analysis algorithm is always faster than software.

  • Reprogram this risk analysis computer into a Bitcoin miner and finish the remainder of the 26M BTC?

    • by zill (1690130)
      Actually it's 21 million BTC in total.

      There are currently 14.2 million BTC left, which are worth $203 million USD in today's market values. JPmorgan made $17.37 billion last year. You do the math.
    • by makomk (752139)

      I wouldn't be surprised if someone already has, to be honest.

  • So they have sped up to computation but have they spent any time and effort on getting a sane algorithm that, for example, can identify a house-of-cards situation like the recent collapse?
    • by NateTech (50881)

      Who says they didn't see the collapse coming and knew the government would bail them out at the expense of the taxpayer?

  • by nimbius (983462) on Monday July 11, 2011 @07:41PM (#36728118) Homepage
    performance of the system, its architecture or its value to the company
    and more impressed that one of the worlds largest financial service providers
    partially responsible for the worlds second largest economic collapse has found, despite their
    prior record with the concept of 'risk', the objective, quantifiable definition of the amorphous and
    highly elusive concept of said 'risk.'
  • Note that Maxeler sold 20% of itself to JP Morgan earlier this year.
  • A purpose built machine will often vastly outperform a general purpose computer.

    The nice thing about an FPGA machine is that creating the purpose built machine is much faster.

  • Since they closed their proprietary trading operation, this will no longer be useful in evaluating bets against the "investments" they recommend to their customers. Is this really useful, or is it just another complicated black box they can show marks and say "this is too complicated for you to understand, so just trust us."?
    • The result of this will be that they'll get a number summing up their credit risk in 5 minutes instead of 8 hours. If there's anything way out of whack, they'll be able to react during the day, rather than having to react effectively 24 hours later.

  • "For the new Maxeler system, it flattened the C++ code down to a Java code."

    ???

  • by zill (1690130) on Monday July 11, 2011 @08:13PM (#36728382)

    For the new Maxeler system, it flattened the C++ code down to a Java code.

    I hope to God that's a typo. C++ -> Java -> Java Bytecode -> Native code almost sounds like a programming language Rube Goldberg machine.

    How would the pointer operations even translate?

    • Re:C++ to Java? (Score:5, Informative)

      by Anonymous Coward on Monday July 11, 2011 @08:28PM (#36728516)

      That's not how it works. The Maxeler FPGA compiler is actually a Java library for describing dataflow machines. They call it "stream computing" - basically just very deep pipelines. You describe your algorithm in the dataflow/streaming style, using the Maxeler Java library, then when you run your program the output is an HDL design to load onto the FPGAs. There's no automatic conversion from a serial program (in C++ or anything else) to a dataflow/streaming program. The developer reimplements the algorithm manually.

    • by bunratty (545641)
      Java references are pointers.
      • Java references are implemented with pointers.

        FTFY. They are actually closer to C++ smart pointers in that you can't do crazy pointer arithmetic with them, but they will do garbage collection.

        This means that while you certainly could implement Java references using pointers (and other stuff like GC), and I doubt there are any serious implementations using anything else, you can't reasonably implement pointers using Java references.

  • by bill_mcgonigle (4333) * on Monday July 11, 2011 @08:15PM (#36728392) Homepage Journal

    The devs are going to run jobs while the machine is idle to corner the Bitcoin market.

    But, wow, from the perspective of getting the Boss to buy awesome hardware for your pet projects - hey, we're not worthy.

  • by decora (1710862) on Monday July 11, 2011 @10:15PM (#36729174) Journal

    JP Morgan, along with every other mega-bank, has no idea what is actually on it's balance sheet, and hasn't for 10-20 years.

    The Shadow Banking system is too big, too complicated, and too interconnected for any of these risk metrics to mean anything.

    JP Morgan did the same business as Goldman Sachs and the others, loading up with CDOs and credit default swaps and CLOs and the rest of it. JPM is portrayed as 'wiser' than the rest in the books and the articles about the crisis, but its not really true. They were less stupid than the stupidest people, that doesnt make them smart.

    They got bailed out just like all the other megabanks. Why? Because they had no idea what was on their books. They are running a black box. All the supercomputers in the world cannot make up for a complete and utter lack of transparency. And that is what the world of Credit Default Swaps (invented at JPM no less) are. A gigantic black box. The rest of the Shadow Banking system is the same, and JP Morgan (and the rest of the big banks) are up to their necks in it.

    Just because you can calculate lies faster, doesnt mean they aren't lies.

    • by devent (1627873)

      But that is the whole point of the system. To have a system that can calculate the risks, so you have an excuse. "But the Supercomputer we had in the closet told us the risks were acceptable, and we can prove it with the algorithms we used" "Ah ok, here are another 500$ Billion bailout money".

    • When your business is selling lies, plausible deniability is a big part of your business strategy. That unfathomable black box is pure gold.

  • can steal more money than any man with a gun.

    With apologies to Don Henley.
  • 8 hours (or 28,800 seconds) to 238 seconds!

    So it's 121x faster, in reality, not in some theoretical limit.

    Either that was some shitty software they were running, or they were trying to do the processing on that spare 286 they had collecting dust in the corner. Seriously, I'd like to know what they were using for comparison.

"A mind is a terrible thing to have leaking out your ears." -- The League of Sadistic Telepaths

Working...