Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
The Almighty Buck

Cryptocurrency Exchange Kraken Suddenly Goes Dark For Two Days (sfchronicle.com) 118

Posted by EditorDavid from the releasing-the-kraken dept.
An anonymous reader quotes the San Francisco Chronicle: One of the biggest cryptocurrency exchanges was down more than 40 hours this week, causing clients to freak out... San Francisco's Kraken went offline at 9 p.m. on Wednesday for maintenance that was initially scheduled to last two hours, plus an additional two to three hours for withdrawals, according to an announcement on the company's website. "We are still working to resolve the issues that we have identified and our team is working around the clock to ensure a smooth upgrade," according to a status update on Kraken's website posted early Friday. "This means it may still take several hours before we can relaunch." Shortly after noon, the company said it was "still working to track down an elusive bug which is holding up launch." It promised customers "a substantial amount of free trading" after the problem was resolved. In previous updates, Kraken mentioned it is working on "unexpected and delicate issues" and assured clients their funds were secure, adding that "Yes, this is our new record for downtime since we launched in 2013. No, we're not proud of it."
It's 53 hours after the downtime began, and their web page is still showing the same announcement.

"Kraken is presently offline for maintenance."
This discussion has been archived. No new comments can be posted.

Cryptocurrency Exchange Kraken Suddenly Goes Dark For Two Days More

Cryptocurrency Exchange Kraken Suddenly Goes Dark For Two Days

Comments Filter:
  • They're at ~99.5% service availability for all of 2018 if they're at the 45 hour outage mark and come back into full service right away. That's a lot, lot worse than Citibank and the Visa network, as examples. Anyone want to place odds on that number falling to ~3.5% for the year?

    • I think you need a new calculator

      You can do the approximate math in your head. Down 45 hours is approx 2 days and we are 13 days into Jan. 20% of 13 days is 2.6 days.

      • Re: (Score:2)

        by gweihir ( 88907 )

        You do this wrongly. Uptime/availability is always calculated for a fixed period that is selected in advance. It is never done as downtime vs. elapsed time. That is just something amateurs do.

        • He's not doing Enterprise Software like we are. We are metaphorically flying around the Universe in the NCC-1701-D while he's metaphorically stuck on Turkana IV avoiding the rape gangs.

          Data! Engage!

    • Re: (Score:2)

      by gweihir ( 88907 )

      The risk that they take the money and run is far, far lower with Citibank or Visa....

    • Hmmm... the Kraken has gone dark,. Most be a Kerbal attack.

    • Re: (Score:2)

      by HiThere ( 15173 )

      If it's a cryptocurrency exchange shouldn't you be comparing it to the stock market rather than to a bank?

  • Maybe they found something missing (Score:3)

    by gweihir ( 88907 ) on Saturday January 13, 2018 @05:12AM (#55920849)

    And are currently crafting a "message" and getting legal help....

    Would not surprise me one bit.

  • Check their actual status page (Score:4, Informative)

    by wyoung76 ( 764124 ) on Saturday January 13, 2018 @05:19AM (#55920869)
    Their status page (https://status.kraken.com/) claims that they're preparing to relaunch the site soon.

  • Kraken not working this is news? (Score:1)

    by Anonymous Coward

    So it's been down for 48 hours seems like normal operation to me as every time I used kraken when it's been "up" it takes 2days to make a trade or at least that's how it feels! so painful kraken

  • Sea monsters occasionally do that.

  • FUD (Score:5, Informative)

    by Anonymous Coward on Saturday January 13, 2018 @05:48AM (#55920927)

    While the facts seem to be in order, this is written as if it was to fuel FUD. Kraken planned to change/upgrade their trading platform for a long time; now that they do, they ran into some technical glitch that they are trying to solve before relaunch.

    The relevant link is status.kraken.com [kraken.com]

    • This site is full of sysadmins. Technical glitches would either be dead hardware (during an upgrade? plug the old stuff back in!) but they could have already replaced hardware. Or it could be software, which they could have restored from backup.

      Multiple days of downtime tells us definitively that the problem isn't some sort of routine "technical glitch" but something more serious, like, "oh, hey guys, all the cookies are gone. I checked the jar and it is just crumbs."

      And actual technical glitch that you wer

      • Re:FUD (Score:5, Insightful)

        by Cederic ( 9623 ) on Saturday January 13, 2018 @08:48AM (#55921385) Journal

        Or it could be something like "Shit, we just fried the drive controllers, the RAID array is toast, where are the backups? What do you mean it takes three days to ship them back, build new arrays and restore the data? Get on with it!"

        If you're not running fully resilient (and that's expensive, so most people don't) then it's quite easy to lose a couple of days.

        • No, actually, if that was the situation they'd be able to give a clear update like, "Hardware problem: parts in the mail; estimated downtime 5 days" or something. It would not be a, "golly I'm so sorry it will be up in a few hours" situation.

      • Re: (Score:2)

        by plopez ( 54068 )

        You sound like like you are an experienced sysadmin /s

        If management went cheap on backups and recovery, they're screwed

        If it was "cloudified" depending on TOS they could screwed royally. Sorry backups and recovery aren't in the TOS (Terms of Shit).

        If they cut costs and hired monkeys to run the thing they are screwed. See also outsourced it, see also cloudified.

        This is more than a glitch as you so glibly put it, train it is a wreck.

        • If it was "cloudified" depending on TOS they could screwed royally. Sorry backups and recovery aren't in the TOS (Terms of Shit).

          What the HELL is the point of the cloud if it doesn't include quick recovery from backups?

      • This site is full of sysadmins. Technical glitches would either be dead hardware (during an upgrade? plug the old stuff back in!) but they could have already replaced hardware. Or it could be software, which they could have restored from backup.

        Multiple days of downtime tells us definitively that the problem isn't some sort of routine "technical glitch" but something more serious, like, "oh, hey guys, all the cookies are gone. I checked the jar and it is just crumbs."

        And actual technical glitch that you were trying to solve before relaunch would get abandoned after some number of hours (days) without a successful fix. You don't have to leave the site down for a "technical glitch," you leave it down because the sky fell on your head and there is no reason to turn the server back on. Like, if all the money is gone and you turn the server on, people try to get their money, and can't, and want to know why.

        Exactly.

        And I repeat, the fact they cannot even put a decent splash page (which is among the first things one must implement), that is telling.

      • Re: (Score:2)

        by HiThere ( 15173 )

        My actual thought was "I don't care what the reason is, bitcoin is so volatile that being down over two days is ridiculous. Isn't everyone who uses them going to abandon them ASAP?".

        There are too many possible hitches, so I didn't even try to guess what they were. But it sounds as if they put up new software without running it in parallel for awhile first as a trial. (On duplicate hardware.) We used to do that for matters a lot less sensitive to our clients.

    • While the facts seem to be in order, this is written as if it was to fuel FUD. Kraken planned to change/upgrade their trading platform for a long time; now that they do, they ran into some technical glitch that they are trying to solve before relaunch.

      The relevant link is status.kraken.com [kraken.com]

      Technical glitch? They cannot even put a decent splash page with the company log on it when you hit the site. That shit was ok back in 95-96. But who does that in 2018? It does not instill much confidence.

  • I hate to disclose this but... (Score:5, Interesting)

    by slashmydots ( 2189826 ) on Saturday January 13, 2018 @05:50AM (#55920935)
    Recently the third party that does domestic wire transfers to a bank account made a mistake communicating with Kraken's systems and they paid me around $400 twice. I told them about it and offered to reimburse them but their support staff never got back to me about how to do so and basically just locked one of my latest deposits and most of my accounts. I even offered to straight up paypal them the funds to get my account unlocked. No response. So if this happened on a larger scale to more people, they just lost A LOT of money. Btw wire transfers are expensive at most banks so just use Coinbase, they do it as a free direct deposit. Oh and also their system actually works.

    • they paid me around $400 twice. I told them about it and offered to reimburse them but their support staff never got back to me about how to do so and basically just locked one of my latest deposits and most of my accounts.

      Lesson: don't complain if you get free money. Of course, be ready to return it later if you're asked to. As the general principle goes, it's easier to ask forgiveness later than get permission up front.

      I assume you have enough cryptocurrencies locked up in the account to care. That's the topic for another lesson (don't trust exchanges for any substantial storage) though I understand issues like high withdrawal fees making small withdrawals infeasible.

      • Re: (Score:2)

        by plopez ( 54068 )

        like drop it into some bonds or something and earn some non-refundable money from it.

      • You take a look at my electric bill and call it free money. Anyway, I have accounts at Poloniex and Coinbase and Bittrex and local wallets because I'm not exactly new to this.

        • You take a look at my electric bill and call it free money.

          By "free money" I was referring to the glitch where they paid you twice for a single transaction. I'm sorry if I've misunderstood the situation.

          Anyway, I have accounts at Poloniex and Coinbase and Bittrex and local wallets because I'm not exactly new to this.

          I remember when Satoshi was on bitcointalk.org, and I co-developed the first opensource FPGA miners. But whatever.

  • Most Interesting Coder (Score:5, Funny)

    by mentil ( 1748130 ) on Saturday January 13, 2018 @06:15AM (#55920975)

    I don't often test my code; but when I do, I do it on production server.

  • Way to keep fucking up the market (Score:5, Insightful)

    by Anonymous Coward on Saturday January 13, 2018 @06:28AM (#55921001)

    It's amateur hour stuff like this that keeps fucking up the market. Coinbase and their sketchy release of BCH trading just as Bitcoin had peaked, tanked the market a month ago. Now I fully expect this news to do the same once it becomes widespread.

    If you're going to be running a major currency exchange, why the hell are you taking down and deploying into production systems?

    With the world of VMs these days, duplicate your entire fucking production environment over to new VMs. Leave the original environment up and running to keep serving customers. Deploy into the new duplicated VM environment. If all passes sanity testing after deployment, cut the traffic over from the old production environment to the new production environment. With VMs, all they should need is a shitload of RAM and disk space. CPU usage will be low on the new environment that deployment is being done on, and once customers cut over that usage will just drop on the old VMs and pickup on the new VMs.

    If the whole thing goes tits up, you still have your old prod environment you can cut all the traffic back over to. if it all goes well, after some amount of time you can decommission the whole old VM environment.

    Aside from being able to safely deploy, having a duplicated prod environment, hopefully in a separate datacenter gives you redundancy during the time between deployments

    • Re: Way to keep fucking up the market (Score:5, Funny)

      by Reverend Green ( 4973045 ) on Saturday January 13, 2018 @07:09AM (#55921109)

      PHB: How long have you been working with Crypto(tm)?

      Devops nerd: cryptocurrency processing is not especially different from other workloads, and benefits from the same modern best practices.

      PHB: get out of here, nerd. Next!

      Wannabe: Hi! I love Agile(tm)! I'm willing to sell my soul for cheap!!

      PHB: Awesome! How long have you been working with Crypto(tm)?

      Wannabe: I've been doing CRYPTO for five years! I'm CRYPTO Certified(tm)!

      PHB: double awesome! Now a technical question: describe the basics of how to set up a resilient production environment for a web application.

      Wannabe: huh? What's resilient mean??

      PHB: I don't know either! You're hired!

    • With that, you need to ensure any database changes can be made independently of the code in your prod and staging environments - ensuring both forward and backwards compatibility - or anything short of a cut-over can lead to data integrity issues. If you can ensure referential data integrity, that business logic is not simultaneously executing on the prod and staging instance, and that your load balancing instances are properly configured, than yes, itâ(TM)s a great architecture and development methodo

    • Re: (Score:2)

      by Megane ( 129182 )
      Good. So how do you "duplicate" the other end of a transaction to a bank for your testing? This isn't supposed to be a trick question; you have dummy accounts set up specifically for testing purposes, and/or a simulator. I wouldn't be surprised if they didn't have the first, and had an insufficient in-house written version of the second.

    • It's amateur hour stuff like this that keeps fucking up the market. Coinbase and their sketchy release of BCH trading just as Bitcoin had peaked, tanked the market a month ago. Now I fully expect this news to do the same once it becomes widespread.

      If you're going to be running a major currency exchange, why the hell are you taking down and deploying into production systems?

      With the world of VMs these days, duplicate your entire fucking production environment over to new VMs. Leave the original environment up and running to keep serving customers. Deploy into the new duplicated VM environment. If all passes sanity testing after deployment, cut the traffic over from the old production environment to the new production environment. With VMs, all they should need is a shitload of RAM and disk space. CPU usage will be low on the new environment that deployment is being done on, and once customers cut over that usage will just drop on the old VMs and pickup on the new VMs.

      If the whole thing goes tits up, you still have your old prod environment you can cut all the traffic back over to. if it all goes well, after some amount of time you can decommission the whole old VM environment.

      Aside from being able to safely deploy, having a duplicated prod environment, hopefully in a separate datacenter gives you redundancy during the time between deployments

      Exaaaaactly. That's how you do it. Unfortunately, it is all the obvious that many of these exchanges are being run by amateur recent grads, code monkeys and keyboard cowboys that program for the happy case only.

      Then shit hits the fan, and they cannot even put a semi-decent splash downtime page.

      • Re: (Score:2)

        by unity ( 1740 )
        Life experience is a great educator; it sounds like the amateurs in charge of this thing just got a big dose of on the job training.

    • If you're going to be running a major currency exchange, why the hell are you taking down and deploying into production systems?

      https://en.wikipedia.org/wiki/The_Sting

  • Works fine for me now (Score:1)

    by Anonymous Coward

    I guess they fixed it

  • and did say they were going to fix it this week,
    Looks like the fix, fixed them

  • NodeJS My bet they got fucked up with dependencies or even got hacked in the process of upgrade.
  • And do you know where your BTC is?
    AKA: will you have 'money' during (and after) an unexpected bank holiday?
    If you spread your risks you will have.
  • it's online now, what can i do know with my pitchfork and my torches?

  • ... a status update on Kraken's website...

    They don't even have a status site available that does not go down with their main site?

  • What does Cryptocurrency Exchange Kraken run on?

  • Update - We've cleared the order book and resumed trading. If your order book data looks stale, try refreshing. We're continuing to closely monitor the service.
    Jan 13, 15:47 UTC
    Update - Order book data is not being displayed properly. Pausing trading while we investigate.
    Jan 13, 11:54 UTC
    Update - Orders appear to be filling in and executing as expected. The aforementioned blog post can be found here: https://blog.kraken.com/post/1... [kraken.com]
    Jan 13, 11:40 UTC
    Verifying - Systems are back online. The new trading engi

Slashdot Top Deals

No spitting on the Bus! Thank you, The Mgt.

Close