Big Data's Invisible Open Source Community

itwbennett writes "Hadoop, Hive, Lucene, and Solr are all open source projects, but if you were expecting the floors of the Strata Conference to be packed with intense, boostrapping hackers you'd be sorely disappointed. Instead, says Brian Proffitt, 'community' where Big Data is concerned is 'acknowledged as a corporate resource', something companies need to contribute back to. 'There is no sense of the grass-roots, hacker-dominated communities that were so much a part of the Linux community's DNA,' says Proffitt."

So... I read the article...

[bmo]

And I have to ask...

What was the point of the article? That the trade show is like every trade show ever?

Really, I'll write a report the next time I go to EASTEC and whine about the lack of "Makers" (in the geek culture sense of the word) among the vendors of Big Machinery.

--
BMO

Re:Sorry

[Anonymous Coward]

Parent poster nailed it.
Try to get support from "the community" when you discover a bug in a code path that nobody except you encounters. Suddenly the community becomes very small indeed.
There just aren't that many geeks out there who handle petabyte datasets. Prove me wrong, dear reader.

Scratching Itches

[Anonymous Coward]

A big part of the grass-roots movement that Linux and other open-source projects benefit from comes about because hackers (in the good sense) contribute to software that they themselves want or need. There probably aren't many programmers that want (or can afford) to store and analyze petabytes of data in their free time. That's important to corporations, though, so I suspect that's why you see primarly corporate interests in open-source Big Data projects.

Re:Sorry

[Hal_Porter]

http://adequacy.org/stories/2001.10.2.33542.4010.html [adequacy.org]

The Linux Fault Threshold is the point in any conversation about Linux at which your interlocutor stops talking about how your problem might be solved under Linux and starts talking about how it isn't Linux's fault that your problem cannot be solved under Linux. Half the time, the LFT is reached because there is genuinely no solution (or no solution has been developed yet), while half the time, the LFT is reached because your apologist has floundered way out of his depth in offering to help you and is bullshitting far beyond his actual knowledge base. In either case, a conversation which has reached the LFT has precisely zero chance of ever generating useful advice for you; it is safe at this point to start calling the person offering the advice a fucking moron, and basically take it from there. Here's an example taken from IRC logs to help you understand the concept.

<jsm> Why won't my fucking Linux computer print?
<linuxbabe> what printer r u using?
<jsm> I don't know. It's a Hewlett Packard desktop inkjet number
<linuxbabe> hewlett r lamers. they dont open source drivers <------LFT closely approached!
<linuxbabe> but we reverse engineered them lol. check the web. or ask hewlett for linux suuport??<------ but avoided, he's still talking about the problem
<jsm> Thanks. I already did that. But I can't install the drivers on my fucking computer. I've got a floppy disk from HP, but my floppy drive is a USB drive and Linux doesn't have fucking USB support.
<linuxbabe> linux DOES have USB support!!!!!!
<jsm> yeh for fucking infrared mice, and for about a thousand makes of webcam it does. Get real here. For my fucking floppy disk drive, I am telling you through bitter experience it does not. Even if someone has written the drivers in the last week
<jsm> which I sincerely doubt, how the hell am I going to install them given that my floppy drive doesnt work?????
<jsm> this ought to be in the kernel. what good is a fucking operating system that doesnt operate?
<linuxbabe> Imacs dont have floppy drives at all <----- useless point, but not LFT. All apologists make pointless jabs at other OSs
<linuxbabe> so you ought to be greateful that Linux does. drivers like that shouldn't be bundled in the kernel
<linuxbabe> makes it into fucking M$ bloatware. bleh
<linuxbabe> download drivers from the web!!!! apt-get is your friend
<jsm> So everyone keeps telling me. Unfortunately the fucking modem doesn't work under Linux either, and since the Linux installation destroyed Windows, that leaves me kind of fucked.
<linuxbabe> Linux doesnt destroy windows
<jsm>mandrake installer does. It "resized" my Windows partition and now the fucker won't work
<linuxbabe> you shuold have defragmented. windows scatters data all over your hard drive so the installer cant just find a clean chunk to install into. it isn't linux fault <---- distinct signs of LFT being approached
<linuxbabe> that windoze disk management blows
<jsm> so why doesn't my fucking modem work?
<linuxbabe> what computer hav u got
<jsm> A Sony Vaio PCG
<linuxbabe> that doesn't have a modem
<jsm> I assure you it fucking does. I used to use it to check my email back in the days when Windows worked.
<linuxbabe> its got a winmodem. thats not a modem <----- nitpicking over technical terms is a sign of impending LFT
<jsm> what do you mean?
<linuxbabe> a winmodem isnt a proper modem. it just uses proprietary windoze apis. doesnt do the work of a modem at all.