Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
News

Trends in an Open Source Project 92

Posted by Roblimo from the predicting-human-behavior-is-art-not-science dept.
Doug Muth writes "On Eric Raymond's website, he has just put a graph depicting the growth of Fetchmail over the last few years. It's rather interesting that the number of participants in the project has only grown linearly - not what one would expect from an open source project. Anyone have ideas as to why this less than expected growth might be?"
This discussion has been archived. No new comments can be posted.

Trends in an Open Source Project More

Trends in an Open Source Project

Comments Filter:
  • And this proves that Slashdot provides enough eyeballs.
  • This isn't really too suprising given the nature of the project. Fetchmail is a thing, its an entity in itself. The number of coders who would contribute it would be some fraction of the number of coders interested in such a project and who have the necessary expertise. This is actually the first I had heard of it for instance, its not something I was ever looking for.

    For exponential growth I think a couple of things are needed. It needs to be a project that will explode in use by word of mouth and it needs to be the type of project that allows a wide variety of backgrounds to be involved.

    Linux as a whole is more similar to this, of witch Fetchmail can be seen as a component. The kernel itself has a number of systems. If your expertise lies mostly in networking stacks there's a place for you. If the in and outs of tuning memory architecture for SMP is your forte' then there's a place of you. If you're not so good at coding but can write good succinct documents there's a place for you. If the kernel itself has no interest to you other than as an enabling technology there are other things you can choose to work on: X11 or other graphics interfaces, themes, productivity tools, games etc.

    The interest in Linux can spread exponentially, at least for a while. When you've got 5% or 10% market share you can grow exponentially for a while. When you've got 98% market share then you can't grow exponentially unless you conquer new markets. The number of potential places a person can work and the variety of interests it matches means that the number of developers can grow as some fraction of the exponential curve.

    Now that I've heard about Fetchmail I do have some interest in it, I probably won't become a developer. That's just not where my talents lie.
  • The growth of developers on any project cannot be exponential or, for that matter, anthing much steeper than linear. The Mythical Man Month predicted this 25 years ago. The larger a development group gets, the more complex the interactions between the developers become--and the project eventually becomes self-limiting unless Herculean effort is used to overcome this effect.

    There are extraordinary exceptions of course, but these are due to management skills that far exceed anything you or I know how to deal with.

  • Far be it from anyone to think that the NCSA web server wasn't successful, but it has fell to demise. It's a project that failed to survive, even in the dawn of opensource times.

    OTOH, the NCSA web server has died out mainly because it was superceeded by Apache, which on a certain level can be viewed as the obvious heir to the NCSA throne (being originally based on the NCSA source, and being mostly upwardly compatible with NCSA). Given that, did NCSA really fail, or not?

  • >That's a really good way of thinking about it. This begs the question: what happens to the environment (the project) when the number of
    >inhabitants (the coders) outgrow it?
    >
    >Obviously, a bunch of coders will leave to start work on a new/different project. [...]
    >
    >I can't think of any opensource projects off the top of my head that have been stripped down to tree bark and dust because there were too
    >many coders.

    My guess is that a flame war would break out over the adding/removal of some feature, which would give many contributors the excuse to leave & find another project to assist.

    But I think the best example lies in the proprietary software arena: look at what Microsoft, Oracle, Sun, et cetera, all will do to keep growing their profits at a set percentage (which translates into continued exponential growth) --

    Bloated software that increasingly does a little of everything adequately (for varying definitions of adequate) & nothing well, & that is increasingly more dependant on the efforts of aggressive marketing & restrictive licensing.

    Not every desktop user needs an enterprise-strength database, nor does every server need a GUI interface dumbed down to a newbie's level. But that seems to be what the barons of the proprietary software industry are working towards.

    . . . & it will all have email, too!


    Geoff
  • whilst it's great that eric has posted some info about the number of subscribers to his lists and about the number of lines of code, i think that the relationships being drawn don't make much sense. simply plotting data on the same graph, as we teach our students, doesn't necessarily mean there is a relationship between the items plotted.
    so the subscribers increase at the time a new version of the software was released--surprise. so lines of code increase at the time of a new release--surprise.
    but to attempt to show that lines of code are related to number of subscribers, well there is nothing to show that.
  • The growthrate doesn't suprise me, because I don't understand the need for fetchmail in the first place. Everybody I know uses IMAP to access their email; if I downloaded my mail onto a non-server, I'd be crippled. I'm suprised that there are so many people using fetchmail, since it implies that:
    1. These people access their email from exactly one computer, or
    2. These people are content with only having access to their email from one computer.
    Am I missing something?
  • The number of potential programmers remains a relatively constant percentage of the population. What is not constant is the opportunities to program, and this is rising exponentially.

    So why aren't there more programmers working on fetchmail? A couple of reasons. First, consider how long it takes to get up to speed with programming. The average programmer isn't introduced to coding in January and skilled enough to do serious work in fetchmail by December. Second, the number of available projects to work on is also rising exponentially. If no new projects were introduced since fetchmail was started, then we would certainly expect an increase of fetchmail coders as the pool of coders increases. However, there are tons of new projects out there.

  • Am I missing something?

    ssh, (or telnet (or a few CGIs)), or lack of money.

    sklein

  • Can we say how many people are working/using a software if we only know how many are subscribed to the mailing lists??

    If you're using a software and you find a bug or make an improvement you don't have to subscribe yourself to a mailing list. You get the source, patch it, and send the modifications back to the authors.

    If you're using a software you also don't subscribe to the mailing list if you got it with your distribution.

    In my opinion these statistics are just _useless_.

    Am I missing something?
  • I don't think that the number of people on the mailing list represents the population of developers. If you could sanitize the data, it would be more useful to plot total number of true developers vs total number of code contributions from that population of developers.

    I would be interested to see if that graph was linear for not.
  • Part of the success in an Open Source Project is the focus of the project itself, in this case Fetchmail. Fetchmail is one of the really useful programs but not exactly a sexy one to work on. Some people are attracted to an Open Source Project because of the type of project it is, kernel hacking, GUI development, etc. The topic matter is not nessesarily more interesting or more difficult or just better it's just that people want to volunteer for those projects not others, like Fetchmail. One could even reason that Open Source Project Staffing is like a demand-side market. We the prorgammers decide which projects survive by contributing our time to them. Therefore only projects that attract people, or that people are attracted to, are likely to gain support. But since we all are intellegent human beings we can see that such behavior will lead to only gimmicky, flashy, no substance projects that have a lot of support but not much else. We know that some projects whether we like them or not need to be supported because if we don't support them then no one will. It's sad, but Fetchmail could be one of those projects.

    So what did we learn?
    Basically the C keyword auto is useless.

  • Because fetchmail isn't interesting (Score:1)

    by Anonymous Coward
    Face it, what's fetchmail? A little program to pop / imap email. I'm mean, really, how many features does such a program *possibly* need? If fetchmail had any more developers, it'd evolve into an MUA.
  • A more suitable graph would indicate number of contributors rather than constituents of the mailing lists.

    This is very true. I myself have been on the fetchmail mailing lists for a long time and while in the beginning I actually sent some stuff to the list (Note: I do not claim to have contributed, by the way. Not to fetchmail at least.), it's been a long time since I last had a need to do that.

    So if ever I had been a developer, I would certainly at some point have stopped being one, even though I'm still on the list. For quite a while now, my fetchmail related needs would just as well be covered by merely being on the annoucement list. And even that is not really needed, given the availability of Freshmeat & Co.

    So not only should one expect to see the number of contributors level off after a while, it can even be natural for it to shrink. This is so even for a successful project that is still very much alive.

    --

  • Re:It's quite simple (Score:3)

    by Black Parrot ( 19622 ) on Tuesday September 28, 1999 @03:01AM (#1654457)
    > Look at the coding curve. It's logarithmic, and approaching a constant of about 17,000. That means that the additional participants just aren't producing proportionate changes in the open source project.

    Just a pedantic point: As a project matures, there will probably be an increasing amount of reworking existing code (cleanup, bug fixes) in proportion to the amount of new code generated. Thus the logarithmic LOC curve probably does not reflect the amount of work going into the project.

    Perhaps a more interesting measure would be a plot of the rate of CVS checkins, perhaps weighted by the sizes of the individual checkins. (If anyone can check this for some project, it might provide an interesting/useful datapoint for the community.)

    But the above does not prejudice your basic thesis; I myself participate as a spectator/part-time-alpha-tester on various development lists, and I'm sure many others do so as well.

  • Fetchmail was created by Virginia Tech undergrad Carl Harris to itch a scratch he had while a sysadmin there. Eric Raymond took over maintainance of the project when Carl moved on to other things. The elegant prose of C&B was inferred by ESR after the fact, rather than the cart leading to the horse as you imply.

  • One wonders how many people like me there are, who use it but don't feel a need to find out about new releases because the old ones aren't giving us any problems?

    Quite a few i imagine. Even counting downloads wouldn't begin to indicate the actual number of users since fetchmail is included with most (or all) popular distributions.

    Certainly mailing list subscription gives no clue as to the number of users. (Nor, i suspect, was it intended to.)

    sklein

  • ?

    I don't grok your concern... he's right after all, Y2K is not a leap year. I suspect that March 1st will be much worse than January 1st 2000 because more systems are going to see it as February 29th 2000 than will see it as March 1st 1900. (of course the *really* messed up folks will see February 29th, 1900.....)
  • While I agree with the idea behind your statement, I'd like to point out that ESR did not attempt to show that lines of code are related to number of subscribers. He plotted lines of code versus time and number of subscribers versus time. The fact that these plots are presented in one and the same graph does not imply that the author is trying to show a relationship between the vertical axis.

    --

  • Am I missing something?

    Home users with... one computer (and no interest to keep their private e-mail stored on an ISP's server).

    Also, victims of sysadmins that switch off support for NFS-mounted mailboxes (hey, this really old technology already was multi-computer!) but that have no desire to switch to another MUA.

    --

  • I dunno...I look at the total subscribers plot, and I see a trend that could be linear, and that could also be a log curve that is just beginning level. Vision is simply not the best way to analyze trends in data plots.

    What would be really helpful is one or more fit lines, each with some variance data to show how closely they fit the plot data. Without this, the growth trends ESR is pontificating upon are pretty speculative.
  • Fetchmail participation is limited because the tool is limited in what it does, and because it has worked quite well for a long time. I use fetchmail; never seen any bugs worth complaining about; never wanted any features enough to act upon. It works. I'm happy with it and have been since I first used it. I'm delighted to discover new features in fetchmail when they appear, or when I browse the documentation, but I don't join the lists to keep track of improvements. Not all opensource projects require huge development communities, and many contributors don't bother with mailing lists -- they just submit a fix when they find a bug and shrug if it doesn't get incorporated into the code base.
  • It's logarithmic, and approaching a constant of about 17,000

    As a former calculus teacher in grad school, I can't let this slide. Logarithmic curves have no upper limit, though they do grow more and more slowly.

    I think there are at least two properties of shared software development that ought to be reflected by any proposed model. First, there is rapid and increasing initial growth, reflecting enthusiasm and growth of the programmer base.

    Later the growth in the lines of code will decrease - it's possible that the lines of code will approach an asymptote. The bigger a project gets, the longer it takes a new developer to get comfortable enough to start contributing, and the more careful an experienced developer must be to avoid conflicts. Also, you start getting into the phenomenon of rewriting sections of code to improve modularity, say, which may even decrease the number of lines locally.

    For an example model with an asymptote, one might consider the logistic growth model:

    dP/dt = k*P(M-P)

    where k and M are constants, and P(t) is the number of people involved. M is the maximum population. For small values of P(t), this is similar to exponential (increasing slope), but then it changes to concave down and levels off.

  • Only at living.... only at living.

  • Linear growth (Score:3)

    by Helge Hafting ( 14882 ) on Monday September 27, 1999 @09:12PM (#1654472)
    Not really surprising. Fetchmail usage may increase non-linearly, but as the program and docs improve, fewer and fewer sees a need to add patches or ask questions.

  • so many projects, so little time (Score:3)

    by mullein ( 37149 ) on Monday September 27, 1999 @09:15PM (#1654473)
    I think the sheer number of open source projects might keep the number of developers on any given project low. Another factor might be that someone looking to participate in an open source project could conceivably be overwhelmed by that number, and end up selecting more glamorous projects. Not to say that fetchmail isn't a great program, but when its already stable and full-featured, what need does it have of geometric growth of developers?

  • Try doing this for the Linux kernel and Mozilla (Score:5)

    by mce ( 509 ) on Monday September 27, 1999 @09:32PM (#1654474) Homepage Journal
    Someone with a lot of time on his or her hands (or even better: who gets payed for doing it) should try making similar graphs for the Linux kernel and Mozilla. That would yield a few interesting comparisons.

    The obvious problem is that it would be a bit hard to do (and even a bit harder to "prove correct"), but still... One thing one might consider is to look at all the Linux kernel or Mozilla releases and scan them for e-mail addresses and the like. Maybe only looking at the CREDITS files, maybe also scanning the source itself (e.g. to find driver co-authors that never made it into CREDITS and also those that are explicitly thanked by driver authors for contributing bug reports and fixes). Unfortunately, to make the results interesting, one would also be able to compare those numbers with those of the related "user" mailing lists and collecting that kind of data will be near impossible, I fear. In any case, one would have to be very careful in collecting and even more in interpreting, though.

    It probably won't be done, but I would not at all be surprised to find developer curves very similar to the fetchmail one. The curves for the number of users may look very different, but my guess is that the developer one would not.

    --

  • Well... (Score:3)

    by Larry L ( 34315 ) on Monday September 27, 1999 @09:15PM (#1654475)
    fetchmail doesnt seem like the most high profile os project. I haven't needed to use it yet, but i've read about it.

    I'd like to see figures on the linux kernel, samba, gnome, kde and other projects before I pass judgement on the scalability of os.

  • It's quite simple (Score:5)

    by konstant ( 63560 ) on Monday September 27, 1999 @09:18PM (#1654476)
    They're spectators. IOW, they're subscribed because they want news about Fetchmail, or because they're enamoured with ESR. They aren't coding.

    Look at the coding curve. It's logarithmic, and approaching a constant of about 17,000. That means that the additional participants just aren't producing proportionate changes in the open source project.

    ESR has graphed how the popularity of his fetchmail project has grown over time, which could very reasonably be linear for such a specialized application. He has not graphed how an open source effort grows. A more suitable graph would indicate number of contributors rather than constituents of the mailing lists.

    -konstant

  • Why would one expect it to be linear or more? (Score:5)

    by Morgaine ( 4316 ) on Monday September 27, 1999 @09:38PM (#1654477)
    I see no reason why it should be linear nor greater than linear, but plenty of reasons why it should be less than linear or possibly logarithmic.

    After all, as projects near a state of completion, you'd expect fewer and fewer bug fixes and enhancements from developers both old and new. If there is any pressure for change resulting from an increase in a project's overall audience, it's certainly not very much, just a secondary effect.

    What makes you think the opposite?

  • Not an logarithmic problem perhaps (Score:3)

    by fishbowl ( 7759 ) on Monday September 27, 1999 @09:24PM (#1654478)
    Is it reaching completion?
    Features are not being added to IMAP and POP3
    at anything resembling an alarming rate, and
    judging from the discussion on fetchmail-friends
    for the past year or so, most people are happy
    with it now, once they get it to work with their
    mailer. I do not fall into this category; I do
    bizarre things with fetchmail. And, ok, I can
    think of a few features that might be useful, but
    they are all better handled in them MTA.

    As a project manager, Eric does quite well, no doubt. He had the presence of mind to gather this
    data. And the wherewithal to turn out the graph.

    What I'd like to know is whether gnuplot turned out this graph as is, or if the gimp was involved.
    Getting a scatter chart is easy, but getting it to look like exactly what you want, for an ad-hoc chart, is not.

    I also wonder what would be involved in adding color support for gnuplot to PNG.

  • more graphs (Score:5)

    by joey ( 315 ) <joey@kitenet.net> on Monday September 27, 1999 @09:45PM (#1654479) Homepage

    Here's another set of graphs [kitenet.net] tracking a free software project -- I've been tracking the number of packages in Debian, as well as how many packages are built with different tools, especially my debhelper tool that most debian packages build with these days.

    I'm also seeing linear growth, though the angle is steeper.

  • Growth patterns (Score:3)

    by Lucius Lucanius ( 61758 ) on Monday September 27, 1999 @09:27PM (#1654480)
    Normally growth patterns in software usage tend to be explosive when it's a new phenomenon with a hungry need - the IBM PC, linux, netscape downloads, yahoo, amazon.com, etc. After the "revolutionary" period has worn off, the rate of growth stabilizes or becomes linear. At some point, it is destined to tend towards flatness, since the number of users is finite.

    My guess is since fetch mail was catering to a well established community, it was by definition a smaller subset of a mature growing community, and its rate of growth was likely to be smaller than the parent. Also, it was not revolutionary or filled a major hunger, so it was not likely to grow at a > linear rate.

    L.


  • Project too simple (Score:3)

    by Anonymous Coward on Monday September 27, 1999 @09:55PM (#1654481)
    So the "exponential growth" of the project levelled off early.

    Apache and the Linux kernel have much larger scopes, but Apache is pretty simple (most of the explosions in development are in third-party modules and CGI scripts) and Linux' growth in complexity is levelling off. Look at the size of the kernel tarballs over time - the complexity of the project is no longer growing exponentially. So I would expect developer interest in these projects to level off, if they haven't already.

  • Re:It's quite simple (Score:3)

    by NettRom ( 39971 ) on Monday September 27, 1999 @09:30PM (#1654482) Homepage
    Look at the coding curve. It's logarithmic, and approaching a constant of about 17,000. That means that the additional participants just aren't producing proportionate changes in the open source project.
    I agree. also, when you look at the number of subscribers to the two different mailing lists, you sett that "fetchmail-friends" has a somewhat constant number of subscribers, while "fetchmail-announce" is the one which increases. That supports your theory that they're spectators. What they wish to do is to receive news when something happens, not actively participate in the project.

    Maybe the number of people who feel they're capable of contributing is constant, even as the user base grows? Those who feel they have the skill join early and develop the project, while the others simply watch?

  • open source projects ask for talented programmers, not people who just understand the basics of cout and cin but people who really have skill at programming. While programmers are a dime a dozen, good programmers are not and then to lower that number ask for talented programmers with the free time to work on an open source project. Some do but many don't and this ratio doesnt magically change, this would be why I think the numbers on projects don't change logarithmically.
  • It's very simple if you think in terms of ecological niches. No project can grow exponentially for an unlimted amount of time, otherwise it would soon overflow its environment. Thus, such exponential growth must cease at some point.

    Some project have larger "habitats" - the excitement of Linux is that it's been able to jump from a niche of hobbyists to business applications and even some lower-level users. It has expanded its environment, thus has room for more rounds of exponential growth. The Internet itself saw this phenomenon when it jumped from only technical/professional users to ordinary people.

    A specialized mail application does not have this potential (unless it somehow manages to become indispensable, a "killer app").

    Thus exponential growth ceases fairly quickly for it.

    - Seth Finkelstein

  • growth model missing (Score:3)

    by icing ( 94825 ) on Monday September 27, 1999 @10:16PM (#1654485)
    There are two main factors behind any population growth:

    Resources: The number of people participating in such projects is not increasing exponential. I would assume quite a steady percentage of the overall human population is geeky enough to be a candidate for such a project.
    While the human population on Earth is showing geometric growth, the population of the high tech countries isnt't. So, it seems safe to assume linear growth of the overall population of possible participants.
    And, the percentage of internet users who can actually program must be decreasing during the last 4 years.

    Competition: There are more and more projects competing for resources. The number of such project has probably shown geometric growth during the last 2 years.

    If you take these two factors together, an average project would grow logarithmic or even not at all.

    For Eric this would mean that he is doing very well with fetchmail, as he is able to keep his percentage of the "market".

    As a side comment: Hats off for collecting all this data and making use of it. I've seen quite some data graveyards in project management. This one is interesting. Thanks, Eric!

  • Yes, spin the data out of cvs logs in a program like GNOME that has a bunch of people with cvs access.

    Metrics are a fascintating part of software engineering, though not to be taken too seriously. The problem with them is that if you start taking them seriously, they become the goal of the team. So if you measure lines of code produced, you end up after a while with locquacious coders stuffing code into source control. If you measure defects corrected, you end up with lots of shoddy code being checked in and subsequently fixed...

    And if you measure subscribers to devel & support mailing lists, you get lots of new subscribers. Eric, please post an update to your graph tommorrow so we can see the slashdot effect! :)

  • I surmise that exponential expansion can only be seen when you consider a large group of projects that are roughly the offspring of one project. The first successful projects inspire other related projects, which form their own teams, which inspire more related projects, and there would be your exponential growth. So, you might consider Apache and several other web servers as children of the NCSA server, and Zope, PHP, etc., as a child of Apache, etc. The relationships will be very rough, don't flame me because you don't think PHP is an offspring of Apache.

    So, you might want to look at a few different trees, with root nodes like the formation of the GNU project, Linux, the Berkeley System Distribution, and the NCSA Web Server Project. I suspect you can make a case for exponential expansion that way.

    Thanks

    Bruce Perens

  • There are, as has been said, many of open source projects out there, and the number of people available for development, while large, *is* finite. Just because an open source project can draw more developers than an in-house commercial project, as per "Cathedral and the Bazaar", doesn't exclude it from that fundamental limitation. As work falls off here and more interesting (read: newer, less defined, more challenging, more fun) projects come along, development for the current project could easily trail off. It's almost like a compromise between Brooks' Law & ESR's Loophole. I don't see why this "news" is surprising.

    (I'd give links for CatB and such, but am browsing with Lynx and don't care to mess with it at the moment. I'm sure you could dig it up if you haven't read it yet -- try Google [google.com] if you need help.)



  • What is this "state of completion" you speak of?
  • There's an easy answer, Fetchmail is a good package with very little work to do. A programmer with some time to work on a new project would be hard pressed to find something worthwhile to do with Fetchmail: it supports almost anything; for most transfers, the server is the performance bottleneck, not Fetchmail; it's stable and reasonably bug-free; it's ported to most relevant platforms. Without having something interesting to do, the programmer is going to look for a different project to work on.

    ----
  • Years divisible by four but not by 100 are leap years. Years divisible by 400 are also leap years. Thus, 2000 is a leap year, 1900 and 2100 are not.
  • Umm, Y2K *is* a leap year. Here's the rules:
    • If the year is divisible by 4, it's a leap year.
    • If it's a century (1500, 1700, 1800, 1900), it's not a leap year.
    • If it's a century divisible by 400, it is a leap year. (2000, thus, is a leap year.)

  • Y2K *is* a leap year. (Score:3)

    by armb ( 5151 ) on Tuesday September 28, 1999 @03:42AM (#1654494) Homepage
    Y2K *is* a leap year, 2000 is divisible by 400.
    Our concern is that many people still get this wrong, as you have just demonstrated.
    See also
    http://www.interlog.com/~r937/lycomplaint.html
    http://www.mitre.org/research/y2k/docs/LEAP.html
    http://www.urbanlegends.com/science/2000_a_leap_ year.html
  • Yargle! This old chestnut again!
    For the definitive viewpoint on this, the old classic DEC response to this particular "bug report" can be found right here [tuatha.org] - note that this was being thought about back in 1983..

  • At the bottom of ESRs page is a link to this image [durak.org] which displays a graph of the linux kernel with files/1000, lines/10,000, words/100,000 and source tree size (MB).

    My personal theory on this problem is that people tend to work on things that don't work for them. For me, I've never had a need to work on fetchmail, as it's always dealt with my environment and desired setup flawlessly.

    I'd love to come up with a more detailed analysis of this stuff, as you mentioned though. Perhaps somebody taking some sort of sociology class could find an excuse to write this paper?

  • This is like saying, well.. tens of thousands of people ask about apples, but them, talk about them.

    The fruit industry is going great! C'mmon, this only says that fetchmail is a great success. Far be it from anyone to think that the NCSA web server wasn't successful, but it has fell to demise. It's a project that failed to survive, even in the dawn of opensource times.

    I blatantly say this: this is a severe misuse of statistics to say something that is near topic, but not generalizable..

    -sporty

  • ARGH! Further proof that: Chris should not be allowed to code after dark. Neither should Eric for that matter....

    I just went and looked at the last time I had to code that algorythm and sure enough... I had remembered (yeah right... looked up) the third rule then ... don't know how I could possibly have forgotten it last night. (well, ok the fact that it is a fsck'ing exception to an exception to an exception to a rule might have helped ;)

    at least I know I'm in good company on this....
  • But if you think about it, all 'living things' die. But NCSA httpd's 'offspring', Apache, lives on. Seems like a pretty normal sort of life cycle to me I guess.

  • Um, then everything is an offspring of a CPU. You are taking things a bit out of proportions. Just like archie died. It didn't spawn download.com and freshmeat.net. THe idea was there, the project, gone. Though I do agree that some projects do have children, like ssh, lsh and the likes, http spun off https..
  • The difference between archie->download.com and NCSA httpd->Apache is that the Apache project really did start out as a descendant of NCSA httpd. download.com and archie are totally unrelated code bases that kinda fill the same niche. Although the Apache developers have rewritten just about all of Apache by now, originally it was A-patch-y version of NCSA httpd. As NCSA httpd died, Apache grew to fill in the gap just as organic offspring live on to further their species as their parents die.

  • Yes, but all projects don't do that you fail to realize. It's not a quality of OSS. It's a quality of evolution. Some things die out, some don't. OS/2 and NT for example. OS/2 is no longer developed for client's anymore. Yet NT was spawned and look at the world.
  • I happened to meta-mod this comment, and got interested... has anyone tried to form a family tree of projects? I'm sure, just using the examples you've given, you could get some pretty interesting results...
  • The number of relationships in a community grow exponentially (1+2+3+....+n-1) for n people.

    The bigger the group, the harder it is to organise effectivley (ie, not as a mob). More than a certain number of people and it's very hard to say 'you do this and he'll do that' without spending your whole time organising people.

    Also, the more people are allready on the list, the harder it is to find your voice on it and feel like you belong to the community. And as many people have ppointed out, the reason many people help out with Linux is because they like the community.

  • but the number of programers is growing! (Score:4)

    by Giraffit ( 33167 ) on Monday September 27, 1999 @10:31PM (#1654508)
    Wouldn't you expect the number of competent programmers will grow as the open source movment grows? Consider those points:

    1. reading good source is necessary to learn programming - now more good source is avaliable.

    2. the concept of playing with code is now open to people who might just not think of it before. (Scientists with programing skills can develop programs that were purchased before)

    3. programming toold are cheaper and better than ever. now its getting easier to code.

  • As someone wrote earlier on; Fetchmail is quite complete, hence there really isn't that big a need to add patches.

    Also, how many actually use Fetchmail? I think more and more people are online permanently and therefore don't use programs suchs as Fetchmail.

    When it comes to other, more complicated, projects, such as for example the Linux kernel, there is a very steep learning curve involved before you're able to begin doing actual work. It's not as if you can just jump in and begin coding...

    W S B - I might be dead, but my opinions are just as valid as when I used to live!
  • I think the assumption that the number of participants - I mean programmers + mailing list subscribers - grows exponentially is unfounded.
    Why should it be?
    There are two fundamental limiting factors
    1. The growth of the internet may be exponential,
    but it's not clear that this means exponential growth of the number of people who are _interested_ in participating (even with subscribing to a mailing list) grows exponential.
    One have to remember that the internet doesn't create "technical" people but in reality eats its way into "normal" population. Naturally the percentage of "technicians" who are newcomers to the internet will in fact drop very fast.
    2. I guess the number of open source projects grows exponentially itself.

  • community grow exponentially (1+2+3+....+n-1) for n people.
    1+2+3+....+n-1 is n(n-1)/2, which is quadratic, i.e. far from exponential. I do not see why the number of pair of persons would be related t the growth of a community.
    Exponential growth would be expected if each month, each coder would recruit two new coders. This is how epidemics spread, or algae in a pond.
    One reason it does not behave this way, as previously stated, is because fetchmail has a bounded ecological niche.
    I can see another reason: as far as i know you do not join an open source project because you know people in it but because you know about the project. So each month the project attracts a given number of developpers, which causes linear growth. Exponential growth happens when, as Bruce Perens states, the visibility of the project increases, for instance when a project gives birth to many sub-project.

    There is a flaw in this: it would work if the population of potential developers was a constant. The number of Open Source users follows a pretty steep increase rate. what about the number of open source coders?

    laurent
    ---

  • The answer is very simple - Fetcmail is SO evoloved and so well covering it's ground that there is very little one might add to this project before it over bloats (something I don't think ESR is likely to allow to happen). Therefor, the project is simply not very interesting in itslef.
    The linear growth we DO see is due probably to things people have been doing WITH Fetchmail rather then "IN" Fetchmail.

  • Others have said it, but probably one of the reasons fetchmail's code growth is leveling out is that it's already doing what it needs to and doing it well. More features would just be bloat, and without new features being added you would expect the code growth to level out.

    Eric did mention about the growth of the number of users being linear instead of a more expected logistic or exponential growth. One explanation might be that he's not counting users, just mailing list subscribers. I, for example, use fetchmail but don't subscribe to the mailing lists, not even the announcement one. For me, fetchmail is doing what I need nicely and without hitting any bugs. I check for major updates once or twice a year, and otherwise just let it run. One wonders how many people like me there are, who use it but don't feel a need to find out about new releases because the old ones aren't giving us any problems?

  • You're right.
    Fetchmail does what I want it to, it fetches my mail. If it works, and I suspect most fetchmail users are like me and have never had a problem with it. Why would there be a need for more developers? If anything the nuber of developers on a project like this should build, peak, and then drop off as the product becomes closer to complete. Not continue to build. This would be much more interesting if it were looking at something that isn't as close to being 'done'.
  • Well he carries a gun and wears a moustache. I am not sure that would make me want him to date my best friend, but I do not see how it matters when all you want is sending him bits and pieces of code

    ---
  • Most Open Source users seem to think programming is just something you pick up on the weekend and then you whip out 9 or 10 professional-grade apps whenever you want to. It doesn't happen like that. Programming is an extremely complicated skill. The reason most people don't program is because it requires an entirely different way of thinking about problems and effort.
    Personally, I would expect something like the programmers to grow linearly, or, as is true of the _vast_majority_ of Open Source projects, I would expect all programmers involved to lose interest and have the project die. Eventually, being a slave doing something you could be making thousands or millions of dollars at for free has to get old.

    Esperandi
  • Other comments in this same thread point out interesting facts.
    To those I add that another point: the target of fetchmail is limited, and tightly bound. Once it reaches that target, it is only to be expected for its codebase growth to slow down.
    The only alternative would be Creeping Featurism, and I don't think ESR wishes to go down that road...
  • I see something of an opportunity here. Could we have an "open source" set of project metrics? That is to say, create some very simple, lightweight, open source tools for tracking participation in an open source project that uses some fairly reasonable set of conventional tools (cvs, mailing lists, download mirrors) and provides some raw counts and some basic reports and graphs? It would be interesting if any open source porject could use such a toolset to keep common stats that could be used to improve the whol development model (or just to show it off to PHBs).

    I'm in the middle of an open source project right now (see the cvs browser [planetmercury.net] at my website if you're curious -- we haven't announced yet because its still too sketchy, but feel free to look) that ties up enough of my free time to keep me from starting such a thing right now, but I throw the idea out if someone else wants to run with it. Otherwise, such a simple toolset might be my next project...

    As for the rest, I guess I'm also saying a "me too" on this post. There are a limited number of good programmers with a limited amount of spare time. There are a lot of interting projects. Some projects will break the curve because they are cool or sexy, and some will be so boring (like my OS dev stats idea) that they'll be constant (at 1?). If you average the data points, I bet you get a logarithmic curve for the very good reasons cited above.
  • My interpretation is:
    • "fetchmail-announce" grows because it is a low-volume list with announcements of interest to all users, not only contributors. It must be a low-volume newsgroup so people do not feel much need to unsuscribe.
    • "fetchmail-friends" is self-limiting. Too much discussion tends to drive away those who are not interested and participating. The constant number of members suggests it is an active enough group that people are unsubscribing when they are not interested in the discussion. If there were no discussion, fewer people would bother to unsubscribe.
    • The lines of code is changing slowly because it is a special-purpose tool. It is undergoing adjustments and improvements, but its basic function is unchanged. It just works, and people use it.
    This is not something which needs dozens of modules and reports to meet different needs. You see geometric growth in a growth medium, not inside a steel girder. What you see in a steel girder is its structural support and the occasional attachment of a needed improvement.
  • That's a really good way of thinking about it. This begs the question: what happens to the environment (the project) when the number of inhabitants (the coders) outgrow it?

    Obviously, a bunch of coders will leave to start work on a new/different project. But if they stay around instead of leaving, an amazing amount of damage would occur to whatever's being hacked on.

    I can't think of any opensource projects off the top of my head that have been stripped down to tree bark and dust because there were too many coders. Anybody else?

    (Halfway through rereading the Dune novels, so the whole ecology-as-culture thing is really stuck in my head right now. :-)
  • Though I suspect that the reason for fetchmail's linear growth has to do with it's approach to fetchmail nirvana, there is another factor at work in the open source community en masse: If the community is growing exponentially, and, at the same time, the number of open source projects is growing exponentially, then we would expect to see only a linear increase in the number of programmers on any given project.

    The proof is fairly simple: suppose that the number of ideas for new projects is roughly proportional to the number of people in the community (this seems like a reasonable assumption to me) then the number of project ideas grows proportionally to the growth of the community, possibly with some constant factor involved. To phrase this mathematically:

    P=Number of Projects

    C=Size of Community
    N=unspecified exponential factor (N>1)
    K=unspecified constant factor (K>0)
    Gc=Growth of the Community
    Gp=Growth in number of Projects
    S=Average Project Size
    Gs=Growth in Average Project Size

    P = C/K (only a fraction of the community creates new projects)
    Gc = C^N
    Gp = C^N/K
    S = C/P = KC/C = K
    Gs = Gc/Gp = KC^N/C^N = K

    Therefore we see that growth in project size is linear while groth in community size is exponential.


    As I said, I don't think that this is what is at work, directly, in the fetchmail project, but it is a limiting factor on the growth of oss projects in agragate.

    We should also consider that the gowth in the oss community may not be exponential, even if the growth in networked households or Linux use is exponential. The technically savy population was networked before the rest of the folks, and was probably using Linux before now as well. I think it is possible that the growth of the oss community may be close to linear (or at least logarithmic).

    -- Jeff Dutky
  • 45,000 lines of code just to connect to a pop3 or imap server and download some mail?? Sounds like it is getting a bit top heavy... :-0
  • >>Most Open Source users seem to think programming is just something you pick up on the weekend and then you whip out
    >>9 or 10 professional-grade apps whenever you want to

    See, now if we had Visual Basic for Linux this wouldn't be a problem....

  • Measuring complexity in terms of number of lines does not seem a good idea to me. Adding drivers make the number of lines increase significantly, but not necessarily its complexity, when different drivers are just variations of the same code to suit different hardware specs.
    On the other side leaving ext2fs for a journaling file system might not cause a spectacular increase in size of the code but it will increase the complexity dramatically.

    ---
  • Previously it was mainly programmers and people whom _weren't_ clueless that started using linux. With the advent of easy to use desktop environments, I believe that the amount of programmers starting just starting to use linux are becoming less compared to the "users". Thus the types of people whom just start to contribute are a lesser percentage of the whole mass. That said, I believe the amount of people whom use linux as a simple desktop environment and nothing more will start to accelerate. A thought. Lucas
  • It would be even more interesting to have a comparison with figures plotting the usage of fetchmail. This is of course impossible, but would probably give figures that would make people upset about the linear growth in the graph feel better.


    bakes
    --

  • Tchoh! (Score:5)

    by Colm@TCD ( 61960 ) on Tuesday September 28, 1999 @12:47AM (#1654528) Homepage
    In his script [tuxedo.org], Eric has:
    # We don't deal with leap years here because the baseline day is after
    # the last leap year (1996) and there's a long time before the next
    # one (2004).

    Oh dear, oh dear, oh dear...
  • I've since discovered the incantation to make
    gnuplot do color PNGs. I didn't use GIMP at all.
  • I disagree. Fetchmail *is* one of the highest profile Open Source projects. Because, if you know nothing about Open Source, you read Eric Raymond's Fetchmail Press Release (aka The Cathedral and the Bazaar) After all, fetchmail was specifically created to test the Bazaar theory, and Eric makes that quite clear in his essay.

Slashdot Top Deals

"The whole problem with the world is that fools and fanatics are always so certain of themselves, but wiser people so full of doubts." -- Bertrand Russell

Close