Slashdot Log In
Organizational Patterns of Agile Software Development
from the swooping-about dept.
Organizational Patterns of Agile Software Development starts by describing the foundations of the authors' research. There are definitions of a "pattern" (but "your intuition about the meaning of the term will take you far") and of a "pattern language" (read the book), the history of their research, and some information about how the book is laid out. The authors recommend you read this section, and so do I; but if it's too dry for you, by all means move on.
The meat of this book is four pattern languages: how to manage a project, how to grow it over time, what can make up an organization's "style" (I'd use the word "culture"), and how the people fulfill their roles and interact with each other. These are not prescriptions or algorithms; they're elements of how successful organizations have worked.
Each pattern describes one aspect of some effective software development organizations. Some patterns are found in more than one pattern language; "Community of Trust" is common to all. Others are less general; "Moderate Truck Number" applies only to the "piecemeal growth" pattern language.
How valuable are the patterns? Some (such as "Get On With It", proceeding with an effort before the planning is considered complete) are common sense. Others (for example, "Don't Interrupt an Interrupt") are things you probably know, but might need to be reminded of ... or might need to remind your boss of. More than a few (my favorite is "Architect Also Implements") might help you understand how something could or should work. Finally, there are some patterns here (such as the "Day Care" pattern for training new members) that might be new to you.
The rest of the book puts the patterns and pattern languages into perspective. There are chapters on organizational principals and (seriously) anthropological foundations of this work. Then there are two case studies of very successful projects. On one, "[about one] million lines of code were written over a period of 31 months by about eight people (that's about 1,000 lines of code per person per week) -- that doesn't include code in the [two] prototypes." It's easy to crank out code at that rate for small bursts, or on small projects. To stay at that pace constantly for over two and half years is nothing short of astounding. The resulting product was released to great reviews. (It then did poorly in the marketplace when it went head-to-head with a directly competing product from Microsoft. Sound dissatisfying? Consider how very long people waited impatiently for Mozilla and its successors such as Firefox. More directly, look at Robert Glass's assertion of the "disconnect between managers and their programmers" as to what projects are seen as successful; it's Fact 13 in Glass's book reviewed August 30th on Slashdot.)
What's imperfect about this book? A couple of things.
First, sometimes the language gets too academic for easy reading. Example: "We have also seen a lighter though almost equally destructive form of this phenomenon, which we describe as schismogenesis.... Symmetrical schismogenesis occurs when two factions each rise in power (or in fear or distrust of each other) and form cliques or splinter groups that tend to focus inward rather than resolve issues in the dialogue with each other." Clear enough if you work on it, but a little intimidating.
Second, the book is surprisingly partisan on some subjects. The book is not kind either to ISO 9000 or Extreme Programming; it could serve as a sort of litmus test, delighting critics and coming across to supporters as unfairly harsh.
What's good about this book? It's a collection of good information, well presented, with information on how to apply it, on a topic where not much knowledge has been accumulated. For some specific circumstances, this book sometimes points out different likely alternatives, with information on when each is applicable. Don't expect Organizational Patterns for the Complete Dummy; then again, don't expect anything useful to be superficial.
How could Coplien and Harrison's work apply to open source development? For starters, they point out the value of people working physically together, and of individual code ownership; these aren't easily applied to open source, but at least it points out forces that need to be resolved somehow. On the other hand, some patterns here are hugely relevant to open source: "Work Queue," "Informal Labor Plan," "Self-Selecting Team," and "Team Pride" come to mind.
Organizational Patterns of Agile Software Development is no panacea. If your organizational practices are the opposite of what's found to be effective, you may find this book frustrating. A book can't take your organization where it needs to go; but Coplien and Harrison have put up some road signs.
You can purchase Organizational Patterns of Agile Software Development from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
James Coplien wrote a paper in 1997... (Score:5, Informative)
It's a fairly short paper and can give you an idea of his style - or at least his style as it was 7 years ago.
What makes a software development project succeed? (Score:2, Funny)
Great idea for a book. (Score:5, Interesting)
I honestly think that most of these patterns could be synthesized simply by looking at things that good small teams do. Each of the patterns described seems like a no-brainer that we've likely had some experience with -- but the same harsh criticisms have been levelled by less-than-Humble Programmers against the GoF's Design Patterns: they're window dressing, they're an attempt to claim intellectual real estate that everyone with a brain already uses .
The critic would be well-served to read The Humble Programmer again; I mean, it's ridiculous that it has taken the time it did for the notion of Design Patterns to come about when Djikstra specifially described the concept in '76.
This book, and it's potential benefits, should be looked at from the same viewpoint that intellectually honest people looked at Design Patterns.
Unfortunately, many bright (but not bright) programmers saw Design Patterns as an attempt to undercut their own intellectual conquests, an attempt to lend mass production to the concepts that they had discovered for themselves, inch by inch. It takes considerable humility and intellectual honesty to benefit from DP's -- programmers are probably one of the lines of work where those two qualities are so strongly stressed (Code Complete, etc).
I would probably buy this book when and if I went independent (likely not within the next five years), but I don't hold out a lot of hope for it for the simple reason that it seems to also require intellectual honesty, and humility.
In MANAGEMENT!
Clarification -- I mean it might sell poorly (Score:2)
Rigorous, unflinching, honest examination of one's thought processes and motivations and examination of one's ideas -- in the manner of a good programmer -- seems as though it would get in the way of most "effective management".
Re:Great idea for a book. (Score:5, Insightful)
The problem is that there are all kinds of Gurus and books out there which claim to make somebody a better "whatever" without ever once touching on what makes a better human being in the first place.
What makes a better human being? Well the lessons are not new and have been written down thousands of years ago.
Be less selfish, be more humble, help others, be kind, share more, take care of others etc. The golden rule more or less.
Large companies instead of sending their staff to certification classes or management seminars should send their employees to become better human beings. This may mean yoga classes, budhist seminars, philosophy classes or something.
Parent
Mod that ^^^ up! (Score:3, Insightful)
At its core, that is the problem.
And actually, coders have been marginalized for a long time -- I don't think that many non-programmers realize that the amount of time that a programmer spends thinking about his work, and actively working on the problems related to it, is easily on par with that of other creative problem-solvers -- architects, mathematicians, engineers everywhere.
However -- and this is something that almost NOBODY outside of programming/mathematics/compsci realizes
Re:Mod that ^^^ up! (Score:3, Insightful)
Which is true, if you're only willing to stay in the same place (in your non-coding job). Honest self-improvement, training, and personal improvement are a requirement EVERYWHERE if you want to go ANYWHERE in life/buisness/etc. If you don't care, well, that's your call. My
Re:Great idea for a book. (Score:3, Insightful)
Let's not be naïve, Mister Coward! (Score:5, Insightful)
However, it's naive to think that the team of good coders -- and by now it should be obvious that I'm not using a narrow-minded 'creative hacker' definition; I mean good programmers, good architects, good with people, good communicators -- is really better off, in terms of potential, when they have a manager -- even one running interference -- than in an ideal situation where each programmer can know the point of view of the customer, can sit down with the customer and get inside their head and their business and bend their minds to the problem of delivering software that lets their customer take over the world on the one hand, and works within the budget of and accomplishes the goals of their company.
Manifestly, an ideal situation is one where the programmers know the needs of their company -- because they know its state, possibly have some ownership. Where they know the needs of their customer -- really know them inside and out. There is a reason that some of the most successful stories when it comes to programming are those involving companies and teams of very small size. Viaweb (Paul Graham) is an example of an agile product; features were often rolled out within a day of introduction by a competitor.
In the less-than-ideal world that many (but not all
That is a huge burden. And the fact is that most managers of IT professionals do not know how to manage the special case of programmers. The fact that the programmers often have to prod their management, gouge them for any sliver of information that could lead to an actual understanding of the real requirements and goals behind a project -- this fact is not a proof that coders need to know how to "work with" management.
Programmers will often go above and beyond the call of duty to solve the problems of the organization in a very real way -- often, as we have seen in the cases of small, successful, programmer-driven companies, in a real enough way to rocket a company from obscurity. A Manager is not going to do that.
He can, however, try to present a more ideal environment for his problem-solvers to solve problems effectively within. He can communicate ceaselessly, so that he's not excluding what could be data that is crucial to a complete or correct understanding of a problem. He can educate himself by reading the last few chapters of books like Code Complete and Rapid Development (these I picked just because they're on my desk right now; there are plenty others. The Pragmatic Programmer is solid, and there are a good dozen or so more that should be requird reading, including some of Joe Celko's later articles in Intelligent Enterprise, like "The Logic of Failure"). A Manager will likely be surprised -- if he has good coders, he will find that many of them have read all of those books already, and he will be amazed at the amount of practical knowledge they have about the realities of scheduling and organizing software projects. Rapid Development alone . .
Realistically, most programmers with that kind of an understanding of the issues involved -- and, of course, knowledge of the appropriate principles of abstraction -- will probably ha
Parent
You know after taking software engineering.. (Score:5, Interesting)
Re:You know after taking software engineering.. (Score:2)
They were fine.
No.
Yes.
No.
Re:You know after taking software engineering.. (Score:3, Insightful)
Twenty odd years of doing the right and wrong things, often repeating those wrong things, leads me to respect any kind of insight that can be brought to bear. I reckon that another twenty years should take us to where the automobile industry was in the 50s.
Mal-organisation is a big headache in our IT group. There is no magic-bullet but lear
Re:You know after taking software engineering.. (Score:3, Insightful)
Writing software is never going to be like designing and building cars no matter how long you wait. Perhaps what is really holding us back is failing to recognize the unique characteristics of software and falling back on inappropriate methods from other fields. It's not that our industry or methods are primitive, it's that our goals are much more difficult to achieve than rolling out the next gas-gusler.
Re:You know after taking software engineering.. (Score:3, Insightful)
One of the hallmarks of the scientific method is the need to develop theories that are testable. These theories are then evaluated on the basis of how well their predictions meet
Re:You know after taking software engineering.. (Score:5, Insightful)
Another example is JSP (Jackson Strutured Programming) which is pretty good for traversing non-recursive data structures and transforming them or generating reports. But today you usually have the data in a database and have a nifty GUI report builder; or you use some form of XSLT. So you rarely use JSP today because the areas where it is appropriate are almost gone.
There also methods that can be used at multiple levels, eg. prototyping, which can be used as a strategy, method, or as a tool. XP has that philosophy at its core, although I fail to see how high-availability is magically implemented by interacting with users.
The waterfall model is still valid for larger government projects where they in general insist on detailed specifications and signing that contract. Iterative development involving re-analysis a no-no because that requires re-signing the contract.
One finally interesting area is maintenance. It probably accounts for more than 80% of the development resources, yet I have never seen any formal method/strategy/tool for handling maintenance/change requests/bugfixes. Is this because maintenance is unsexy?
Parent
Re:You know after taking software engineering.. (Score:4, Interesting)
It's also good not to think of software as ever being "finished". I can't think of any projects (even short perl scripts) that I haven't had to go back and work on, sometimes 2-3 years later.
Just assume you are constantly going to have to do iterations and act appropriately. At a certain point, the iterations will slow down tremendously but be prepared to dive right back in at any time.
I.e., first deliver something in 2-3 weeks. Get feedback from the customer. Deliver the next iteration in 2-3 weeks. Get feedback from the customer. Continue until they are happy. After a while they'll come back with a bug report or a feature request. Deliver something new in 2-3 weeks. Get feedback from the customer.
As for specific tools, I personally use basecamp [basecamphq.com]. It's basically a blog. No "issue tracking" or "tickets" or anything like that. Just a place for your customer and you to discuss stuff. I just create a "bugs" category and tell them to post bug reports there. Then we discuss. Then I create a milestone in 2-3 weeks and deliver the next iteration. It's all about communication.
This isn't appropriate for open source or "boxed" software, but for one-on-one with a development team and a customer.
I think the best analogy is that of a garden. First you plant, you do a lot of work, but the work isn't done. You still have to cut weeds, plant new plants, rearrange things every now and then. Rarely do you just destroy the whole thing, and rarely do you ever just leave it alone, it needs care and feeding.
Same with software projects.
Parent
Re:You know after taking software engineering.. (Score:3, Insightful)
Extreme Programming qualifies. The way they look at it, after the first iteration is complete you're in maintenance mode. A steady stream of change requests come in, and every week you do the most important week's worth.
This is a good way to look at th
Re:You know after taking software engineering.. (Score:2)
Re:You know after taking software engineering.. (Score:4, Interesting)
Clearly, you don't know much about Extreme Programming. An XP project ends up building two complete test suites. The unit tests are written from the perspective of the developers, and the acceptance tests are written from the traditional QA perspective.
On my current XP project, the whole test suite gets run every time somebody checks in a code change; any failures and the computer yelps until we fix things. When we do releases we manually poke around a bit, but that's mainly superstition; if we were really afraid that something could be broken without us knowing, that would be a sign we hadn't written very good tests.
This means you want to cause the minimal amount of change to the codebase possible. [...] With maintenance you don't EVER want to refactor- refactoring has a much higher likelyhood of introducing bugs, requiring more extensive and expensive regression testing. You want to design the change, apply it, and then never touch that part again if possible.
This is only true if changing things is likely to introduce bugs. If regression testing is automated, it's cheap. And if you have full test suites and do pair programming, you are unlikely to introduce bugs. (At my current shop, we have bug rates well below one per developer/month.) That means that there's no reason not to refactor if the code needs it.
And there's a big reason to refactor. Your approach of fearfully making localized changes to the code without refactoring means that the broad design gets more and more distorted. Over time, the system gets squirrely enough that nobody really understands it, which increases the cost of change and increases risk. Eventually, your approach leads to the abandonment of the code base and a big fresh start. That's very expensive.
Extreme Programming, on the other hand, is designed to be sustainable over the long haul. Instead of assuming that the code base will eventually become moribund, XP tries to answer the question, "How can we keep this code good enough to work on forever, no matter how many changes are requested?"
Parent
US Navy: X coding, 2X debugging (Score:2)
I read somewhere (sorry, can't find the link) that the Navy budgets X $million to write the code for a missile-control system, and 2X $million to debug the code.
-kgj
Re:You know after taking software engineering.. (Score:3, Insightful)
That's where XP comes in, or at least claims to.
I think you can get most of the benefits of XP just by implementing the Unit Testing aspects of the philosophy; I'm not sure I buy the whole package but Unit Testing really holds up.
Re:You know after taking software engineering.. (Score:3, Insightful)
The previous ways are still valid - but the areas where they are appropriate disappear over time.
I dissent.
Those 'old ways' are not 'old methodologies'. Thechnicalities dissapear over time, but experience accumulates.
Re:You know after taking software engineering.. (Score:3, Insightful)
This strikes me as a sign that the methodology in question is incomplete.
That's not to say that a methodology should be completely idiot-proof. But it should be possible for humans to do. It shouldn't solve all their problems, but the methodology should make the problems obvious as soon as possible, and give a good framework for thinking about solving them.
For me the
How to make a software project work...for real (Score:5, Funny)
2. All the soda you can drink before 5pm.
3. All the beer you can drink after 5pm.
4. A dev machine with the best graphics card, raided SATA drives, 3+GHz, and a raw, unrestricted net connection.
Free food and drink, Half-Life parties after 5pm...what more could any geek want? They'll pump out more code than you're slowbie QA people can handle.
Re:How to make a software project work...for real (Score:3, Insightful)
1. Pizza. Lots of it, on the company dime.
2. All the soda you can drink before 5pm.
3. All the beer you can drink after 5pm.
4. A dev machine with the best graphics card, raided SATA drives, 3+GHz, and a raw, unrestricted net connection.
Sounds like the
Re:How to make a software project work...for real (Score:2)
besides, a geek would work during night.. allowing some 'strange' behavior works for some.
Re:How to make a software project work...for real (Score:5, Insightful)
A couple points in defense of QA. First, if the developers were pumping out better code, instead of more code, speed of the QA people would not be an issue. Second, your slowbie QA people aren't really slow.
What happens is, Z amount of time is scheduled for a project; X development + Y QA. The developers always go over schedule and take X + a time. (I'm not saying it's our fault--PHB, feature-creep, whatever--but it happens.) This leaves QA with Y - a time to do their job.
Inevitably, especially as a approaches Y, deadline Z is missed and all eyes go to QA since they are the last ones in the chain. As a developer married to a QA person, I get to hear about this stuff all the time. Lucky me =)
Parent
Re:How to make a software project work...for real (Score:3, Funny)
Hooray! That's what I want: people pumping out code.
I think you should next take your theory to Boeing; they'll be excited to hear a cheap way to get their workers to rivet more metal onto planes. And I'm sure the customers will be excited when they hear that the cost per pound of their planes will go down a lot.
Re:How to make a software project work...for real (Score:2)
2. We have a soda machine at work, but the coin and bill acceptors have been disabled, intentionally, by the management. Anytime you want a cold soda, got hit the button and get one. Someone comes out and restocks the machine on a daily basis. I make a pretty heavy dent in the Dew supply. When I was interviewing a possible new hire last week, and pointed out the free soda machi
NEW from Ahole Press!!! (Score:3, Funny)
The Teach Teach Yourself eXtreme Test-driven Agile Development Patterns in 21 Days By Example Bible.. JAVA EDITION!
Seriously I believe strongly in Agile methods (even before they got the name "Agile") but please let's not dilute the name "Agile" to the point that I'm embarrased to use it, okay??
Me: "I think we should stay agile and use this simpler algorithm until we have some real-world benchmarks."
Coworker: "Okay.
Me: *grumble*
Leadership (Score:3, Interesting)
If you're leading a software project and reading books like these your project is already screwed.
Re:Leadership (Score:4, Insightful)
If you're leading a software project and reading books like these your project is already screwed.
I dunno.
Good leaders are constantly trying to learn new things.
Good leaders don't dismiss an idea because it is/isn't associated with a fad, they evaluate the idea on its merit.
Good leaders know the difference between "there" and "their"
Parent
Re:Leadership (Score:2)
Re:Leadership (Score:3, Interesting)
In Peopleware, DeMarco and Lister assert that "the average software developer ... doesn't own a single book on the subject of his or her work, and hasn't ever read one."
I'd be pretty happy to learn that my manager is reading a computer book, especially one without "24 hours" or "dummies" in the title.
An anti-dilbert, yeah right (Score:4, Interesting)
What I think America needs more of us is a military-influenced leadership culture. Not too heavily influenced, but enough that people understand that when you work in a team, even as the leader, your success and the team's success are one-in-the-same.
Of course schools often discredit this by having teachers who will grade down an entire team because of a slacker or two's stupidity, whereas in the real world the slackers would get fired for bad performance. I have had a class before in college where I got graded off because another team screwed up and made it impossible for my team to finish our work.
It seems to me that the best anti-dilbert solution is to instill from an early age the value that if you screw up, you'll be accountable, and that when you succeed that you share your success with those that made it possible.
Re:An anti-dilbert, yeah right (Score:2)
This goes against the very core of corporatism. The purpose of forming a corporation is the shirk responsibility in the first place. Nobody who is busy climbing the corporate ladder will share success the idea is to take all the credit, point the finger of blame, and stab everyone in
Teamwork: Untouchable! (Score:3, Funny)
Like in DePalma's The Untouchables
That's when Capone (Robert DeNiro) takes the baseball bat and sma
So to summarize (Score:2)
I usually hear things like, if we all just did 'x' then our problems would be solved. Problem is given a large enough group, say about 3 or more, they'll almost never all d
A related short article (Score:5, Interesting)
The gist of the article is that first-rate problem-solvers are not well suited for "agile" methods, because they tend to be more individualistic (and the agile methods tend to force programmers to interact continually).
I especially thought it was interesting that programmers with good people skills do better in agile methodologies, and that there are many more programmers than there used to be who have good people skills. This could be the death of the Dilbert stereotype....
Re:A related short article (Score:2)
That article uses the example of Isaac Newton as someone not well suited to Agile methods. I propose that a genius at the level of Isaac Newton is equally poorly suited to every existing software methodology out there. (The thought of Newton in a meeting, gathering buyoff for his requirements document is so depressing it makes me sick.)
What it means to develop in an agile environment (Score:5, Interesting)
Agile is meant for projects where change is a fact of life. If your project has explicit, written in stone, unchanging goals then you should consider another design methodology.
Agile development focuses on rapid delivery of working (but not polished) software. It should be lightweight, but still accomplish the functional goal.
Agile projects should provide value long before they are finished. Each increment of functionality should be useful for a customer.
Agile is not meant for large teams. Agile does not work well with customers who can not provide frequent feedback. Agile is not optimal for projets which have stable needs that are known from the beginning. Agile is not a good choice for mission critical products.
These are the basics as taught to me, they may not be the same as presented in this book.
Re:What it means to develop in an agile environmen (Score:2)
Re:What it means to develop in an agile environmen (Score:3, Funny)
Then you've drunk the Kool-Aid. Go throw up and quit trusting the customer not to screw you over. Now!!!
Agile is not a good choice for mission critical products.
Riiiiight:) Mission critical (as in "this needs to happen or the company folds") products need to spend months doing BDUF (Big Design Up Front) in order to stay in line with the 70% failure rate that business has come to expect (and resent) from software development methods.
Re:What it means to develop in an agile environmen (Score:2, Interesting)
Not all customers will screw you over. Many of the will, but it isn't always helpful to mix customer relations with extreme skepticism.
Mission critical is no so much "this needs to happen of the company folds", but more "this needs to happen or (bridge col
What about... (Score:2)
1) Idea
2) ??
3) Profit
just wondering
It's ONLINE! (Score:3, Informative)
The best and worst experiences... (Score:4, Insightful)
o Any existing utility libraries had been heavily tried and tested
o Everyone was enthusiastic about the particular area of work they were doing, as new code was being designed.
o Everyone had separate areas of work, and there were well defined API layers between each module.
o There were only two agenda's affecting everyone
- Management wanting the work complete
- The programmers/software engineers
wanting the work experience
The worst projects have been:
o No well cleared API layers, and programmers belting code into each others routines without checking with each other.
o Senior management (in head office) messing everything up by adding a third agenda and switching everyone around for the sake
of having the brighest graduate working on XXXX instead of YYYY.
Agile (Score:2)
Agile? hmmm.. why not Fittness Programming?
I can see the project leader: "OK, Move those bits boys! MOVE! MOVE!"
Pet peave (Score:5, Interesting)
Re:Mentor? (Score:5, Funny)
Mentors, the Freshmaker!
(I apologize for this, really, it's just been one of those days).
Parent