Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Slashdot.org News

Slashdot Back Online 346

Posted by CmdrTaco from the well-that-was-a-mess dept.
I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it, thus triggering the end of several nearby worlds as well. Props to Yazz, KurtG and Scott from Cisco for managing to help get us back online. We'll post more when we know it.
This discussion has been archived. No new comments can be posted.

Slashdot Back Online More

Slashdot Back Online

Comments Filter:

  • Re:Original Story (Score:4)

    by Anonymous Coward on Sunday June 24, 2001 @08:06PM (#130840)
    Phrases like "if you don't get this motherfucking network up within the next hour..." and "were you born clueless, or did your Mom drop you on your head when you were young..." didn't help to keep me there. I'm a contractor, not a therapist.

    Jeff and Rob are not easy to work with. Best wishes to my replacement.

    - Anne Tomlinson

  • I've never heard of you anne, but thats a pretty impressive troll. No doubt I'm hard to work w/. I'm a perfectionist. But neither of those phrases sounds like Jeff or I. We'd be much less polite.
  • If you're a perfectionist, I'm Miyazaki.

    --
  • If you're going to be a grammar pedant, be pantically correct. "I" would be proper at the end of that sentence because it preceded an understood verb, and therefore was a subject pronoun and not an object pronoun.

    --

  • Re:I went Outside!!!! (Score:4)

    by Trepidity ( 597 ) <delirium-slashdot@@@hackish...org> on Sunday June 24, 2001 @04:11PM (#130846)
    nothing is funnier when user friendly does it.
  • Thought this had something with the 50% staff reduction VA is having on Wednesday. For a moment it looked like CmdrTaco got fired and crashed the server on his way out or something. Fortunately now we can read all about the latest conspiracy to spam email.

  • they're all OSDN stuff. it was an OSDN cisco router that melted, see http://usw-sf-log.sourceforge.net/

    That's fine... Kuro5hin was up all weekend. Are they somewhere physically (and logically) different?

  • I knew you could!

    This being /., I would think there would be more redundancy on the hardware level as well...
  • The major problem would be IO bandwidth, so you'd have to use something other than multiple cards in a 32bit 33MHz PCI bus. That has a theoretical limit of 132 MB/s, which is too low a bisection bandwidth for any respectable gigE switch. NVidia's nforce chipset has two PCI busses, just like my old Mac clone from Daystar, so that would help some. Still, with a server mobo and some 64bit 66MHz PCI busses, you'd have 528MB/s per bus. That's decent, but still nowhere near what you want for 24 GE ports. You'd also need to buy several 4-port NICs, and find slots to put them in. Switches really are hard to do with software on commodity machines. Routing is another matter, unless you have a lot of subnets and need a lot of bisection bandwidth, I guess. (If you were going to be limited by the fact that all the traffic was going up or down through one of the interfaces, and not between two internal ones, then a computer with NICs could probably do the job reasonably well.
    #define X(x,y) x##y
  • Well, for the past 36 hours at least Mae West has been un-responsive to the rest of the world. Basically, for those of us down under, that's let about 40% of the USA sites unreachable. Didn't even realise /. was dead, just thought the whole of the USA was dead (and started rejoicing at the thought.... :P )
  • I beg to differ. You may wish to have a look at the online traffic analysis reports at various places that map network traffic. The one I go to is the Internet Traffic Report [internettr...report.com] It quite clearly shows that Mae West has been generating 100% packet loss for well over 24hrs now. Here's the North American [internettr...report.com] page. According to the 7 day report, it dropped off air on the 21st and hasn't been back since. Means that I still can't get to most USA based sites that I like to frequent. Sure the 'net routes around the problem, but when something like 70% of the asia-pacific traffic goes through it, that puts a hell of a lot of strain on all the other points.
  • Shouldn't an update be appended to the complete original text of the story and clearly marked as such, rather than appear to be an attempt to cover up something you came to regret having said?
  • I had no idea that there was a Japanese version of Slashdot but one look at it goes a long way toward explaning why so much of the spam I get is composed of (to me) undecipherable characters. How many "foreign" Slashdots are there?
  • Welcome back Cmdr... We missed you...

    Lando
  • Nod,
    It was pointed out to me a couple of years ago that bind doesn't really implement the RFC for dns. There are a lot of problems with the bind implementation, but since bind is the default, it's hard to get a "proper" dns working since the standard is bind's implementation.

    Lando
  • Thanks,
    Will check it out.

    Lando
  • Well myself, I went jogging, and I noticed the birds that were flying around actually had realistic "surround" effects, so that when I closed my eyes, I could almost feel them moving around me. Very lifelike.

    The attention to detail was truly amazing. Car horns honked and the sound echoed just the way you'd have expected in a ultra-realistic video game-- only better.

    Very hard to describe, but definately worth it trying again in a couple months.

    W


    -------------------
  • I wonder why /., the champions of open source, aren't using a Linux box with a few NICs and some hacked-together code as a load balancer?

    If you'd read teh post on newsforge, you'd know that it was EXODUS's routers that went down, NOT OSDN's. RTF News Articles.
  • Mae West has been unresponsive to the west of the world for several years. She died in 1980, so that's a bit more than 36 hours. Seriously, though, what are you referring to by Mae West?
  • Rob: Okay. Just let me know when youre ready.

    Rob would never use "you're" and "your" in the appropriate situation.

  • There is a big difference between covering open source and free software, and actually advocating and believing it. Slashdot does a pretty good job of the former, but is .. uh .. "objective" enough to avoid doing the latter.

    Although Slashdot seems to conform (mostly) when you look purely at the source-code aspect of things, they very clearly work directly against open source interests when you look at file format and multimedia issues. For example:

    • Slashdot uses GIFs for images
    • Slashdot uses MP3 for it's "radio" program
    • Slashdot editors advocate the purchase of DVDs
    You can't use interfaces/formats that require licenses, and stay free (and in many cases, you can't even have open source). Content providers that use/advocate this stuff, indirectly work to encourage people to stop using and developing free and open software. The use of a user-unmaintable router is very minor and insignificant by comparison.

    Slashdot does overall have a positive influence, though, and from the selection of stories, it is very clear that the editors are at least very interested in the topic of open source. But calling them "champions of open source" is completely inappropriate.


    ---
  • One little question: Who cares?

    If you by any chance think its a valid comment, then drag itover ffs. You cannot expect everybody to read both places.

    And listen you idiotic pup. Not everybody thinks that "Karma" is life, death and everything.


    --

  • So why is it that most ISPs hire less qualified people? Is it because you get what you pay for?

  • Yup. But DNS seems to be one of the most poorly managed things on the net. When it's only done half-right, most people can still get to most places and they think it's correct, but subtle problems do exist in many ways, especially with caching, and most admins have no clue about what or why or even realize it's a DNS problem, saying "It can't be DNS, I can get there".

  • Woo ... those deep deep technical terms again, like "melted". OMG, that's so deep it's not even in my CCIE books. Well at least it's back to solid form now.

  • I had a Cisco router lose the smoke built into it's power supply once. Fortunately it was one of two routers running in parallel, so for the 4 hours it took to get a replacement, we were actually up all the time.

  • A choice today is djbdns [djbdns.org].

  • Oops. Read that too fast, parsed that as "perl autistic," and my first thought was, "How appropriate."

    --
  • Or someone put a link to /. up on a certain Christmas Island website. [goatse.cx]

    --
  • Homo Sexual Routing Protocol?

    --
  • Don't forget the dangling, abbreviated participle.

    --
  • Hemos decided to test Exodus' claim that the colocation cages were sharkproof. He had them lower the cage with him inside.
    Poor bastard...

    Join the Great Fujisan Expedition! [mmdc.net]

  • Re:Tee hee... (Score:3)

    by Surak ( 18578 ) <surak&mailblocks,com> on Monday June 25, 2001 @03:58AM (#130900) Homepage Journal
    I remember how stressed out my cat, Kyoto, was when she became Cisco Certified.

    Well, for all of you wondering whether or not this is the real Sarcasta, as in Rob "Cmdr Dorko...err..Taco" Malda's other half, Kathleen Fent...here is your answer right here, plain as day.

    If you check out this page [sarcasta.net]you'll quickly see that Aunt Kathy here doesn't have a cat named Kyoto .

    Combine this with the fact that for a Mac using graphic designer (and I know many :) "she" knows a little too much about PostgreSQL vs. MySQL [slashdot.org].

    Add to that the high user number (c'mon, if this was Taco's chick, she'd be like a high user number, right? :).

    No way. Taco: If this user is really your chick, tell us. :)

    And to the holder of this Sarcasta account: Do try to do a little research before you troll as Sarcasta, 'k?
  • Try the quarterlies [yahoo.com].

    Particularly enjoy

    "OUR SUCCESS DEPENDS ON DEVELOPING NEW SYSTEMS THAT ACHIEVE MARKET ACCEPTANCE AND ON THE SUCCESS OF OUR PROFESSIONAL SERVICES ORGANIZATION" (their shouting),

    "WE COULD BE PREVENTED FROM SELLING OR DEVELOPING OUR PRODUCTS IF THE GNU GENERAL PUBLIC LICENSE AND SIMILAR LICENSES UNDER WHICH THE OPERATING SYSTEM INCORPORATED INTO OUR PRODUCTS IS DEVELOPED AND LICENSED, ARE NOT ENFORCEABLE"

    and especially for today "WE ARE VULNERABLE TO UNEXPECTED NETWORK INTERRUPTIONS CAUSED BY SYSTEM FAILURES, WHICH MAY RESULT IN REDUCED VISITOR TRAFFIC ON OUR NETWORK, DECREASED REVENUE
    AND HARM TO OUR REPUTATION".

    FWIW I think they'll actually be fine. No zillionaires, but afloat none the less.

    Dave
  • When's the last time Slashdot had 10 employees on the list of "World's richest people"?

    When's the last time Rob's gross was measured in the billions?

    When's the last time Slashdot's DNS servers were down?

    Get the hint? I could go on...

    When's the last time Slashdot attempted to publically humiliate Microsoft?

    uh, scratch that one.

  • Re:Exodus (Score:5)

    by gbroiles ( 22589 ) on Sunday June 24, 2001 @04:51PM (#130906) Homepage
    You're right, the Slashdot guys screwed up .. I want my money back, right now, you cheap Slashdot motherfuckers. How dare you run a free website without online fail-over routers? I'm goin' down to the Salvation Army and getting some free lunch and when I get back I want to see TWO rows of blinky lights in that rack, or I want a refund on my subscription, pronto.

  • Am I the only one that doesn't think this post should be taken at face value? To believe that this is really the tech in question, you have to believe one of these things:

    • This VA/Slashdot employee didn't have/couldn't get a /. username and thus needed to post as an AC.
    • This person had an account, but posted as an AC (on a message that could have come from nobody else) and then signed her name.

    Does anyone want to buy some Man Beef [manbeef.com] or a Bonsai Kitten [bonsaikitten.com] kit?

    Greg

  • Funny, isn't it?

    And people wonder why Linux users buy less software than Windows users... If mtr was a standard program on all Windows (even server-level only) boxes, people wouldn't have to buy a shareware program to do the same thing..

    And what admin in the middle of a huge crash cares about the pretty path the packets take? They want to know which router is down, but they don't care about its ICBM address (unless the problem is really bad.)

    * The other reason Linux users don't buy software is that 1) many Linux boxes don't have users, 2) many Linux users are in 'poor' countries where the purchase price of MS Office would be a year's wages, or 3) is it worth paying $30 for a shareware program when the free one is just as good and maybe just not quite as pretty?

    I buy windows software but not Linux, not because I don't think Linux developers need money, but because I haven't found a Linux program that I'd want to buy (and had to buy).

    Now, if Q3 for Linux came out as soon as for Win32, I'd have bought that, but I'm not waiting a few weeks for a political statement.
  • Unless perhaps he thought better & changed it himself? There is no evidence to support your theory, and considering how frequently people send an email in a fit of rage & promptly wish they could get it back, I see nothing suspicious in the change.
  • Yes, I have to admit. That was rather entertaining. Can't replace slashdot, but it was a decent temporary alternative, and almost as fulfilling. :)

    +1 Pimp (those that were there will know.... )

    -Restil

  • If in fact the original message was changed (I wouldn't know) it would be because they are now can be held libel for the technician being forced to quit. (Harrasment is uncalled for).

    The presedent of the company I worked for would come running down the hall if the server crashed, he would immediatly start asking us what happened. Of course we would not have an answer yet. It one instance he came running down the hall so fast we did not even have the chance to stand up and walk over to the server 5ft away from us. (Then he started questioning everything we did and simply insisted we call HP. Even though it was obvious we had to cycle the power since even the console was not responding. Server came up just fine after we cycled the power.)

    People often over react to the situation. Yes, it is a very bad thing. (Server crash, connection down, etc. But those people most effected should probably NOT be allowed in the room unless they are skilled enough to do the job themselves.(And being vaugly familiar with routers IS NOT being skilled with them).

  • Currently at f*company there is a bit about Exodus not going down however... I was there becuase /. was down and I was wondering if VA had let everyone go. Was there a spike in the sale of VA shares offered? It was the weekend so the major stock exchanges were closed and I don't know if VA trades anywhere after hours but it would be interesting to see if a bunch of geeks start bailing out of socks when the company goes offline.

    Mabe a stock price drop makes that spare router a bit more economically feasible.
  • Try this:

    1. ipchains -I output -d 0/0 80 -y -l -j ACCEPT # i.e. log the IP address of outbound web connections; this is for Linux 2.2, YMMV

    2. Pull up a site which has round robin A records with multiple IP's (IIRC www.yahoo.com will do; if you want URL for one of ours, send me email - I don't want to Slashdot my customers :-)

    3. Find out the IP address the browser is using in the logs. Observe that since the browser caches the result of gethostbyname() it keeps using the same one.

    4. ipchains -I output -d xxxx -j REJECT

    5. Hit another link and see what happens

  • Exodus (Score:5)

    by RallyDriver ( 49641 ) on Sunday June 24, 2001 @04:17PM (#130921) Homepage
    You are talking out of your rear.

    We use Exodus, and they provide us with two separate ethernet feeds, down separate cable runs, from two separate routers in different parts of the internal NOC. No need for any routers at all; we have separate endpoint hardware on each feed and just do a rough load balance across the feeds with round robin DNS.

    The recommended (by Exodus) alternative is to have a pair of peered routers which actively load balance across the feeds at the IP level, and back up each other if one goes down. I didn't do this as we're a startup I didn't want to pay for an extra pair of routers.

    Either of the above will ensure that there is no single point of failure on the front end. This is referred to as a dual-homed configuration. Exodus' WAN will ensure there is no SPF further out; making your own equipment cluster and software fault tolerant is your problem :-)

    It sounds like Slashdot is running with a single-homed connection, and that the router which failed is their own kit in their own rack. $$ permitting, they could have either (a) done a proper dual-homed setup, as per one of the above, or (b) had a spare router sitting in the rack and lease Exodus' managed hardware monitoring service, which would have meant Exodus techs switching it out when it failed.

    I don't know what Slashdot's budget for hosting is, but we are a much smaller company than Andover and dual-homed service is not exactly killing our budget. I would conservatively assume that bandwidth is Slashdot's biggest expense.

    You cannot throw pies at the co-lo provider for your own failure to have a robust setup and make proper use of the facilities they offer.
  • Actually they're using an Arrowpoint CS-800 (can't recall offhand if was a single box or an actual redundant pair). Sweet box.. until Cisco bought Arrowpoint and then proceeded muck it up by "migrating" the CS OS to an "IOS-friendly" standard.

  • "(yes, both were down. However, sebastian.slashdot.org (AnimeFu) was up. How mysterious...)?"

    Not at all, if you'd take less than 5 minutes to look at why. But here, I'll do it for you.

    Let's sart with slashdot.org's IP address as a base, so we know where "home" is -- 64.28.67.150 (Exodus). Now, looking up slashdot.org's info in NSI's database tells me that the first name server (64.28.67.55) is in the same netblock, but the second one (209.192.217.105) is somewhere else entirely (belonging to Shore.net).

    So we've established that even if our link to "home" is severed, we can still do DNS lookups.

    Now let's look up sebastian.slashdot.org -- 206.170.14.75, yet another netblock entirely (appears to belong to Up Networks [upn.net]).

    So in a nutshell, router to Exodus goes down, so the dns lookup for sebastian.slashdot.org goes to Shore.net instead, where it gets pointed to Up Networks.

    nslookup and traceroute are your friends. Use them.

    --

  • Damn you Slashdot, due to your outage, I actually went outside!

  • Yeah, but why would you spend a few zillion buck$ on an S/390 when you could buy a specialized Cisco router which is optimized for the job?

    (Other than the fact that /.'s seemed to have crash&burned)
  • For those who don't know, Anne Tomlinson is the wife of George P. Burdell.
  • then why was freshmeat.com and others down? including thinkgeek.com

  • Cisco routers are good. Cisco IOS has tab completion too so THEY HAVE to be good :)

    --
  • As a comrade in the gateway buisness, I know full well how hard it can be to work with a customer. Especially a customer who thinks that the world revolves around them. If I sat down long enough to write down all the bad things I have been called, a large portion of my week would be shot. Knucklefuck was my personal fav, but anyway.

    Don't sweat it too much. One good thing about our job is we don't have to put up with the assholes we are currently working with. We can easily find a job with a completely new set of assholes.

    On a side note, if you are in the process of doing your job and a customer is hovering over you, that qualifies as harasment. My current contract obligates me to hourly updates for priority customers. Anything more than that, and I cut them off. I would rather be elbows deep in a 7507 than in a meeting describing what I'll be doing when I am elbows deep.
  • Here's a thought. Next time you can't get to /., before you rewire your entire town trying to get a good route, please try some other site. I hear there's a few other websites around.

  • Very. (Score:4)

    by Inoshiro ( 71693 ) on Sunday June 24, 2001 @04:57PM (#130945) Homepage
    Kuro5hin is only loosely associated with OSDN. Yeah, we run their ads and get revenue that way, and we are listed in some of their material (but not all, and not often). We're not owned (even in part) by VA, and I don't know of any plans to 'buy' us (which makes sense, since they don't have to cover any costs).

    I do all system administration -- DNS, mail, etc., whereas the VA owned sites all share the same pool of cool admins (like Yaz, Alliecat, etc).

    Rusty and I are happy with our current colocation service (vhosting [vhosting.com]). We've never, ever had problems of connectivity (only of perl/admin error :)).
    --
  • they're all OSDN stuff. it was an OSDN cisco router that melted, see http://usw-sf-log.sourceforge.net/
  • Actually, Ypsilon, Now acquired by Nokia, produces routers with a FreeBSD kernel. These are real FreeBSD machines, on Intel CPUs, with a very limited userspace, and some hacks to the motherboard. The thing is really fast. There was a dude saying that with Cisco hardware the packets don't hit the CPU. That's just half true. It depends on the header. If the roting is straightforward, then it's done in shared memory, but as soon as parts of the header contain pertinent information, the CPU is, indeed, involved, with Cisco routers, too. And the problem with Cisco routers is, that they don't have very fast CPUs. Of course, this depends on the model, but noone of them is as fast as, say, a 1 GHz Athlon.

  • And when our qualified personel arrived, we discovered that she wasn't actuually as qualified as we had hoped. Then she quit


    I see you are checking your employees references just as much as you are checking the submission hyperlinks.....


    Sorry, I had to take that stab. :)

  • The lameness filter turned on itself.
  • I had written:

    Sometime shortly after midnight on Saturday morning, I posted the following (with some edits since this version is from a copy I sent to my private email list). I verified it appeared on Slashdot but it is now gone from the Cyc story to which it had been posted and within which it did appear prior to the outtage, indicating there may need to be an investigation to ascertain the problem's full extent:

    Fanatic responds:

    What this has to do with Slashdot being down is anyone's guess

    Are you referring to the idea, apparently shared by a moderator, that missing data is "Offtopic" when discussing a system outtage?

  • I was referring to the huge amount of other text, dealing with the Plato system and other stuff.

    Ah, I see, so the missing data, itself, would not be helpful to those who might wish to determine what had happened to it?


  • The original to this was a reader's letter submitted to GameFan[1] years ago. Circa 1996, I'd say.

    1. The best console gaming mag in existance, until Imagine publishing killed it.
  • Made the mistake of goin' camping this weekend, and, got a little confused when I rigged up a Ponderosa pine that had a telephone line a couple feet from it with a WAP device a local squirrel was friendly enough to wear for the exercise after I gave him one of my nuts (not those nuts, you deviant; I had some planters in my napsack). Picked up e-mail through a pine cone; checked on my primary server through a knot-hole, and then slashdot. No dice.

    Kinda feel bad, now that I'm home, that I didn't trust that squirrel and re-negged on the nuts. Guess I'll have to go back up there and make good on it. Sh|t, he's still got my WAP device! No wonder I don't have any mail.


    Linux rocks!!! www.dedserius.com [dedserius.com]

  • By the number of people on #slashdot, I thought the whole thing was a clever ruse intended to see if they could bring down slashnet (as six thousand trolls go to see why they couldn't get their fix).

    Steve

  • What this has to do with Slashdot being down is anyone's guess...Amazing to hear about PLATO after all these years. For those who never had the pleasure (which is to say almost everyone) this was work on computer-aided education, started in the 70's (60s?) at University of Illinois Urbana. The terminals I worked with had translucent plasma screens. When photographic quality was needed, a projector on top of the unit was used - someone would insert a microfiche that had 256 different images on it and computer controlled pneumatic cylinders were used to select which image would be projected. I kid you not - I spent a couple of semesters making the electro-pneumatic assemblies. This was about 1973-74.

    --
  • Actually, I was referring to the huge amount of other text, dealing with the Plato system and other stuff. It could have been reposted to the thread it was originally posted to ("the Cyc story") rather than here. Of course, so could have mine, but at least I said mine was offtopic.

    --
  • Ah, I see, so the missing data, itself, would not be helpful to those who might wish to determine what had happened to it?

    would have been just as helpful on the other thread, with a pointer here. Especially since the folks 'who might wish to determine what had happend to it' (yeah, right) aren't the readers of either thread, so he could have just emailed it to whoever. Why are you so sensitive? I didn't moderate him down, I just made an offhand comment.

    --
  • With all those people downloading "Thong Song", it was only a matter of time.

  • Original Story (Score:5)

    by jhealy ( 91456 ) on Sunday June 24, 2001 @02:57PM (#130975)
    the original statement was a little bit different:

    I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it, thus triggering the end of several nearby worlds as well. And when our qualified personel arrived, we discovered that she wasn't actuually as qualified as we had hoped. Then she quit, thus terminating 3 local star systems. Hemos or I will update this story as soon as we know what the hell happened. But apparently creds go to Kurt Grey and Cisco tech support. Hopefully we'll have more info soon.
  • Damnitall... Just toss the router and hook up a 12 volt battery to your upstream connection.
  • This is redundant, but I don't care. Some new person male or female rushes into to this huge slashdot meltdown pile of shit on a weekend... sees how totally fucked up their system's configuration is... sees what a collection of morons work for slashdot... realizes that this could lead to days of pure hell in close quarters with the morons... decides that quitting is the only option available to retain sanity.
  • Cisco gets called all the time. It doesn't make it particularly hard, either.
    Unfortunately, Cisco's support has gone the way of everyone else's. Most of the time they seem to wait for you to figure it out and then find out what you did. Don't think it's *taht* amazing because they got called. But hey! You can't expect a tech to know *everything!* So what if they call Cisco? Cisco gear melts down all the time, and it's not as reliable as all that. We used to have a 25% DOA rate on access servers at my old job. These thigns cost $60,000 with a 75% discount! Their quality control has gone down the drain to meet the demand of their manufacturing line.
    Dunno who did or said what, but Cisco ain't that great, and they're surprised by nearly everything you ask them. You say "every router has this config and it works on all of them but this." They say "then the others are all broken, this one works to specs." SOunds funny, but it's happened to me.
  • Glad that's over. When I first noticed the outage, I thought it'd be brief. When it continued, I had a sinking feeling and started to consider the entities that would like to shut down slashdot legally.
    It was like a mystery story that begins with the death of a controversial man. Everyone had a motive, from Microsoft to the MPAA to the Scientologists.
    I had a surge of unhappiness at the thought that if Slashdot were really the victim of legal asault, we probably wouldn't learn the truth for a long time. VA's attorney would have told the employees not to comment. The whole community revolving around this site really counts for less to the legal system than some shopkeeper's (in the last analysis) claim of malicious interference with his tomatoes.
    It made me think of the moment in The Hacker Crackdown when AT&T pulled the plug on a machine hosting an online community.
    I tried reading kuro5hin. Everything is more reasoned, grammatical, lucid, correclty spelled -- and yet strangely lacking vitality. It was like walking through a clean and quiet museum, with 'do not touch' signs everywhere. There were no street urchins chalking obscene sketches on the marble walls. Maybe I'm a denizen of the lower depths of the internet and not suited to such musem atomspheres. Slashdot is like a real city, complete with beggars and drug addicts, while kuro5hin is perhaps like a mall.
  • (Most/all?) cisco routers do most of the work of moving packets from one interface to another entirely in hardware. The packet never hits the CPU at all. The CPU is there mostly to maintain the routing tables. It doesn't DO the routing at all. On a (PC) linux box, not only must every packet be routed explicitly by the CPU, but all traffic must go across the PCI bus, twice. A PC is just not the right architecture for a high traffic router.
  • Thank god you are back on line. I was losing it. I even started reading kuro5in.

    Even more important, freshmeat is back. A weekend without freshmeat is no weekend at all.

  • How to be a karma whore. (Score:3)

    by Carnage4Life ( 106069 ) on Sunday June 24, 2001 @02:56PM (#130992) Homepage Journal
    Q:Mommy, How can I become a good karma whore.

    A: Steal posts from kuro5hin.

    I wonder why /., the champions of open source, aren't using a Linux box with a few NICs and some hacked-together code as a load balancer?

    Exhibit A [kuro5hin.org]

    Funny that when Microsoft's router failed (probably a Cisco also) it was catastrophic incompetence but for you guy's it's just bad luck.

    Exhibit B [kuro5hin.org]

    --
  • And for those that missed that part, the story was originally:

    I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it. And when our qualified personel arrived, we discovered that she wasn't actuually as qualified as we had hoped. Then she quit. Hemos or I will update this story as soon as we know what the hell happened

    Hope this clears up what that poster was trying to say for all those late-comers.

    /Mattias Wadenstein

  • Re:Story keeps changing? (Score:5)

    by maswan ( 106561 ) <slashdot2&maswan,mw,mw> on Sunday June 24, 2001 @03:41PM (#130994) Homepage
    Well, so far I have seen 3 different stories. And a 4th one in a comment (that looks like it fits along with the other three comments).

    All the versions I've seen personally in chronological order:

    I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it. And when our qualified personel arrived, we discovered that she wasn't actuually as qualified as we had hoped. Then she quit. Hemos or I will update this story as soon as we know what the hell happened

    I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it. Hemos or I will update this story as soon as we know what the hell happened.

    I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it, thus triggering the end of several nearby worlds as well. Props to Yazz, KurtG and Scott from Cisco for managing to help get us back online. We'll post more when we know it.

    The one from another comment:

    I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it, thus triggering the end of several nearby worlds as well. And when our qualified personel arrived, we discovered that she wasn't actuually as qualified as we had hoped. Then she quit, thus terminating 3 local star systems. Hemos or I will update this story as soon as we know what the hell happened. But apparently creds go to Kurt Grey and Cisco tech support. Hopefully we'll have more info soon.

    /Mattias Wadenstein

  • Story keeps changing? (Score:3)

    by Mahy ( 111194 ) on Sunday June 24, 2001 @02:55PM (#130997) Homepage
    In the past 10 minutes I've seen 3 different versions of this story of varying lengths... Is that normal?

    Version 1: I'm still not exactly clued in as to why we're back online, but hey, we are. Sometime saturday morning our Cisco router melted down. Ordinarily this would only be the end of the world, but none of our qualified personel were available to fix it. Hemos or I will update this story as soon as we know what the hell happened.

    Version 2: Inject some stuff about the qualified person being a she, and not being very qualified, and quitting.

    Version 3: Inject some weird stuff about neighboring starsystems also being blown up.

    Am I losing my mind?

  • Interesting (Score:3)

    by talonyx ( 125221 ) on Sunday June 24, 2001 @02:46PM (#131010)
    I wonder why /., the champions of open source, aren't using a Linux box with a few NICs and some hacked-together code as a load balancer?

    Your qualified personell would be just as qualified as the woman who quit, and it would have cost less. Plus, you could appeal to the masses with an Ask Slashdot about the best way to set it up!

    Funny that when Microsoft's router failed (probably a Cisco also) it was catastrophic incompetence but for you guy's it's just bad luck.
  • It really pisses me off that the post was changed.

    I don't know whether the stories of Rob and Jeff flaming the tech for not being able to her job are true but the fact remains that:

    • Taco posted the story as he saw it (rightly or wrongly).
    • The Powers That Be didn't like the wording and made him change it.

    Makes me wonder where editorial control heading.

  • Re:Interesting (Score:3)

    by Hellcheese ( 128336 ) on Sunday June 24, 2001 @07:04PM (#131017) Homepage Journal
    Most low-end routers (1700's, 2500's, 3600's, etc) do packet forwarding in software. It's when you've got something like a Catalyst 5000 with a route supervisor engine card that it can do switching in hardware. If routes change however, the first packet gets routed in software. This establishes a path for all subsequent packets to follow in hardware. 7500s and 7200s can do this sort of stuff as well. There are two types of forwarding the Cisco's can do - CEF (Cisco express forwarding and multilayer switching).

  • Re:your cisco? (Score:5)

    by alanjstr ( 131045 ) on Sunday June 24, 2001 @03:57PM (#131020) Homepage
    More from Roblimo over at NewsForge [newsforge.com] Sunday June 24, @07:30PM
    - by Robin "Roblimo" Miller - On Saturday, June 23, the primary controller in the router that controls access to all OSDN servers hosted at the Exodus facility in Waltham, MA, suffered a catastrophic failure. The sites affected were Slashdot, freshmeat, NewsForge, and Mediabuilder, among others. The secondary controller did not automatically take over as it shoud have. It did not work when activated manually, either. The first Cisco support people contacted professed to be "amazed" at the situation, saying it was the first time they had seen a failure of this kind. OSDN and Cisco people, working through Saturday night, were unable to cure the problem. Sunday afternoon, OSDN employee Kurt Gray and Cisco rep Scott, working by telephone, were stepping through the router's configuration and, says Kurt, as they worked to undo other changes that had been made, "on one reset everything came back." OSDN network operations were already in the process of rebuilding the company's network to eliminate the router as a potential single point of failure. As of 7 p.m. US EDT most of the sites were available at least part of the time, but full service was not yet restored. There may still be slowdowns or intermttent failures until a permanent fix is made. We'll have a more complete story within a few days. Right now, OSDN network operations staff members are too busy working to talk.

  • Interesting (NOT!) (Score:5)

    by rgmoore ( 133276 ) <glandauer@charter.net> on Sunday June 24, 2001 @03:19PM (#131027) Homepage
    I wonder why /., the champions of open source, aren't using a Linux box with a few NICs and some hacked-together code as a load balancer?

    IIRC, they are using a Linux box for their load ballancer. It was their router that got fried, which is a completely different beast. Heavy duty routers remain specialized boxes, and Linux hasn't really serious inroads into that market yet.

    Funny that when Microsoft's router failed (probably a Cisco also) it was catastrophic incompetence but for you guy's it's just bad luck.

    Not funny at all when you get the facts straight. The serious problems that MS had were with their DNS servers- which were running Windows- not their routers. IIRC the DNS servers were later cracked, too, which was rightly seen as an indication of poor security. When Microsoft uses its own products, they don't stand up to the use they're being put to, and then Microsoft has to use *BSD based systems to get working again, that's very different from when a Linux site has its non-Linux hardware melt down (and the description did make it sound like a hardware, not software, problem).

  • Maybe she didn't quit... (Score:3)

    by Salsaman ( 141471 ) on Monday June 25, 2001 @04:05AM (#131038) Homepage
    Maybe she touched the incoming cable of the router and got Slashdotted.

  • Everything else on /. is redundant--the stories, comments, lame jokes--so why don't we have redundant hardware too?

  • I called my ISP! (Score:5)

    by ellem ( 147712 ) <ellem52.gmail@com> on Sunday June 24, 2001 @05:30PM (#131041) Homepage Journal
    Me : Dude it's like Yahoo, it's never down.
    ISP : Sa-lash dot?
    Me : Dude slashdot.org!
    ISP : www.
    Me : No no no... listen 64.28.67.150
    ISP : Uh... www
    Me : Damnit I'm down can't you see I'm down?
    ISP : We're like up and stuff. Is this a Macintosh?
    Me : I am calling my lawyer! I'll sue you blind!
    ISP : Uh I have to get my supervisor.
    Me : -click-
    ---

  • Why did she quit? (Score:3)

    by bear_phillips ( 165929 ) on Sunday June 24, 2001 @02:49PM (#131057) Homepage
    Can you give us any info on why she quit? From the short blurb, I can only think that she came in, didn't know exactly what to do, got chewed out, then ran out of the place crying. Hopefully I am wrong. This would make a great ask slashdot "how much leeway do you give a tech employee." I know when I started, I could do anything given enough time, but my employeer stuck with me. Granted in this case, there wasn't any time to waste.

  • Re:Original Story: Who are you? (Score:4)

    by streetlawyer ( 169828 ) on Monday June 25, 2001 @07:32AM (#131060) Homepage
    "I'm a perfectionist"

    As a perfectionist, you'll of course want to know that you use "me" when the reference comes after the verb. "Neither of those phrases sounds like Jeff or me".

    Also, given that you're a perfectionist, you'll be appalled to hear that someone has been using the "CmdrTaco" identity to post poorly spelled, ungrammatical crap all over Slashdot for the last three years. That same person has always tried to justify himself by whining that he "doesn't care about that stuff" and "doesn't want to be too fussy". This may or may not be the person who wrote the notoriously buggy first release of Slash, and said that it was "close enough".

    But of course that couldn't be you

    Because you're a perfectionist.

    Ha, ha, ha. I think I'm going to have to give up satire.

  • More typically it's like this.

    You: I can't reach my favorate site:
    ISP: Ok, can you reach other sites?
    You: Yes, but I can't reach this one. It must be your fault.
    ISP: Well, what site are you trying to reach?
    You: Slashdot. It's never down. It always works
    ISP: I can't seem to reach it from here either. My guess is it's down.
    You: No, it's your fault. I want to see my slashdot.
    ISP: Let me do some tests. No, I'm sorry, that sight is down.
    You: You are lying to me!! Just like AOL did. Your ISP sucks! I want a refund.
    ISP: Well, you can reach every other site, right?
    You: Yes.
    ISP: Then most likely, the problem is with the website.
    You: *explitive deleted* I want you supervisor.
    ISP: Fine.

  • *ahem* (Score:5)

    by Technodummy ( 204943 ) on Monday June 25, 2001 @01:10AM (#131100)


    if that was a post about a guy, and he was thought to be less than qualified, would you be posting this?

    sexism goes both ways, assuming someone isn't incompetent due to their gender is just as stupid as assuming they are

    equal rights for incompetent people dammit! *L*

    and yes, I am a chick

    on a personal note, maybe that post was taken down due to it's rudeness, rather than the sex of the person involved...

    my 2 cents...

  • Re:What really happened (Score:5)

    by DickBreath ( 207180 ) on Sunday June 24, 2001 @04:01PM (#131101) Homepage
    I respectfully disagree. Give Hemos his dignity man. I conjecture that what happened was more like this...

    CiscoChick: Hi Rob. It's that time again. I came by to check on your equipment.
    Rob: Equipment!?! Okay. Just give me a minute to get my pants off.
    CiscoChick: No, no! I meant your Cisco router. I'm here for a scheduled routine preventative maintainence checkup.
    Rob: Oh! That equipment.
    CiscoChick: Yeah, the router. But when I'm finished, I could check out any other hard ware you have around. <wink>
    Rob: Okay. Just let me know when youre ready.

    later.....

    CiscoChick: Okay, Rob. I'm done checking the Cisco router.
    Rob: Okay. Cool.
    CiscoChick: Wow! Look at that equipment!
    Rob: Yeah.
    CiscoChick: I mean, it's so small!
    Rob: Yeah, it's the latest new thing in miniaturization.
    CiscoChick: Okay, well.... Let's not focus on the size. What is the uptime on that thing? Does it go down very often, like Windows?
    Rob: Ummm... Have you ever done it in a co-location cage?
    CiscoChick: No, but there's a first time for everything.


    3 minutes later...

    Rob: Ahhhhh! I needed that.
    CiscoChick: Oh, no! What's happening!
    Rob: Eeeeeeiiiiiiieeeee!!!! The router is melting!

    --
    "Linux is a cancer" -- Steve Ballmer, CEO Microsoft.

  • Oh man... (Score:3)

    by kgutwin ( 243912 ) <kgutwin@yahoo.com> on Sunday June 24, 2001 @02:46PM (#131123) Homepage
    ... And I was just about to get some real work done!

    -Karl
    ---------
    [root@kgutwin /dos]# file msdos.sys

  • Hmmm. (Score:4)

    by sachachua ( 246293 ) <sacha@@@free...net...ph> on Sunday June 24, 2001 @02:56PM (#131124) Homepage Journal
    Slipping standards in the open-source world.. ;) Didn't Microsoft catch a lot of flak for having a single point of failure that other time that most of its sites went down?

    When I couldn't get my Slashdot, I assumed the worse. High-profile hijacking. Aliens beaming up the OSDN headquarters. Servers sneakily migrated to Windows, which then promptly crashed.

    Kidding aside, I'm glad Slashdot is back up.

  • I went Outside!!!! (Score:5)

    by V50 ( 248015 ) on Sunday June 24, 2001 @03:06PM (#131126) Journal
    I was just leaving my house to go demand my News for Nerds and/or Stuff that Matters, when something hit me: THIS IS SPECTACULAR RESOLUTION!!!

    When I stepped outside it looked like everything was being generated by 500,000,000 GeForce3s!!! The trees looked REAL!! It must have been at least 1,600,000,000 x 1,240,000,000!!! I couln't even see any jaggies! Talk about anti-ailiasing!!

    After spending 2 days outside sue to lack of Slashdot it's hard to come back to my Power Mac 6100/60 with a 14" monitor at 640x480. I wish I had reality's 3D card...

    And it seem's Slashdot has slashdotted itself. How did that happen??

    --Volrath50

  • Re:Original Story: Who are you? (Score:5)

    by V50 ( 248015 ) on Monday June 25, 2001 @06:11AM (#131127) Journal
    Not anouther fake identity troll!

    I mean who is this CmdrTaco guy and how the heck did he get UID #1???

    It's obvious that this isn't the real Rob Malda, we all know that Slashdot editors NEVER post at Slashdot....

    --Volrath50

  • Re:Interesting (Score:4)

    by raju1kabir ( 251972 ) on Sunday June 24, 2001 @11:38PM (#131130) Homepage
    I think a better comment to make would be on the point of connection redundancy over the internet - wasn't the original military plan for the net to make sure information could still get around even following a nuclear strike......one fried router and suddenly a whole shedload of sites go down

    That would be a pretty dumb comment to make in this context since the router that went down was on the premises of the customer whose sites went down.

    I mean, if you step on the modem in your house, you lose your link to the internet, but that doesn't mean you've identified an Achilles' heel in the internet's infrastructure - "Aha! This single modem controls access to the ENTIRE INTERNET! By stepping on it I have rendered the whole network inaccessible to EVERYONE in my house!!"

    Seems that should be pretty obvious.

  • Translation (Score:5)

    by jstockdale ( 258118 ) on Sunday June 24, 2001 @11:55PM (#131139) Homepage Journal
    Sometime saturday morning our Cisco router melted down. We were making smores, using the excess heat generated by the improperly ventilated Cisco router, Saturday morning after finishing a 16 hour coding spree when all of a sudden CmdrTaco dropped his. The ohhh so gooey marshmello and yummy chocolate dripped into the air vents and all of a sudden the Cisco smoked and all the led's went out. Realizing there must be a fault, we promptly panacked and ran around the room screeming. but none of our qualified personel were available to fix it Our Cisco expert wasn't up to the task due to eating too many smores on a earlier coding break. Upon realizing this, we attempted to fix it ourselves by plugging it back in, but to no avail. We then called Cisco. Props to Yazz, KurtG and Scott from Cisco for managing to help get us back online. Those great guys from Cisco, upon hearing of our predicament, instructed us to apply a half sinewave duration, several hundered newton, force laterally on the Cisco box using our lower right appenditure. Upon asking them "What the fuck are you talking about?" they replied "Kick the damn thing." Using extreme precision, we followed out their instructions and kicked the shit out of the box. Amazingly the thing reset itself and began working as usual. We suspect that the cause was a software bug, or a chocolate induced short circuit.

  • Congratulations... (Score:5)

    by Libster ( 308301 ) on Sunday June 24, 2001 @03:32PM (#131152) Homepage
    It only took you forty five minutes mins to offically retract the whole "she" thing.

    Gathering data from your May 2nd demographic evauation, Im thinking that you nearly lost five percent of your readers in the space of forty five minutes.

    Yours in disappointment,

  • Re:Interesting (Score:3)

    by roguerez ( 319598 ) on Sunday June 24, 2001 @03:03PM (#131157) Homepage
    I wonder why /., the champions of open source, aren't using a Linux box with a few NICs and some hacked-together code as a load balancer?

    Because you use the right tool for the job. Linux can can very well be used for many purposes, including acting as a router for your home/small business network. That does not, however, qualify it as a full blowing routing device which is up to the job of handling router needs of something like slashdot.

Slashdot Top Deals

The use of money is all the advantage there is to having money. -- B. Franklin

Close