Log in

No account? Create an account
Who drives, directs and supports projects within the Apache Software Foundation - Nick [entries|archive|friends|userinfo]

[ website | gagravarr.org ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Who drives, directs and supports projects within the Apache Software Foundation [May. 25th, 2010|02:30 pm]
Following on from my previous post on CLAs and Release Votes, I thought I'd do another one on the Apache Software Foundation. This time, it's looking at different roles in directing and supporting a project, and how the ASF deals with corporate contributions.

Note - this is a personal view. While I am a member of and PMC chair within the ASF, it's my view, not an official foundation statement. It's based on lots of discussions, talks and mailing list posts, but please do shout if you think I've got something wrong...

When looking at an Apache project, people will often make distinctions between four kinds of people:

  • Users - anyone who just uses the software, without really interacting with the project beyond that

  • Contributors - anyone who helps out with answering questions, contributing simple patches or tests cases, often but not always on the project mailing lists

  • Committers - people able to commit changes to SVN, both their own changes, and those coming in from the community from Contributors

  • PMC Members - those people tasked with keeping the project on-track, and directed by the ASF board to look after the project on behalf of everyone

It's often said that there is a pyramid of people within an Apache project. At the bottom are very large numbers of users, above them a smaller number of contributors powering the project, then a smaller number of committers making changes to the code, then a smaller number of PMC members overseeing it. If you can get yourself to a The Apache Way talk somewhere (ApacheCon, Apache BarCamp etc), you'll hear a lot more on this, but the important thing is that a project's success is driven by this large base.

Two common questions about an Apache project are: who drives and directs the project, and who keeps it going?

The answer to the 2nd is easy, but perhaps not the one you may think. It's the contributors! Without all the contributors out there, answering the questions of others, helping write examples, providing test cases and the odd patch, evangelising, your project won't grow and maintain itself. This is why you see some projects (eg httpd) rewarding those people who contribute documentation and examples, and not just those who contribute code. Almost all new committers come from the pool of contributors. Most users will hit a problem at some point, and it's through the work of the contributors in blog posts, mailing list answers, documentation or examples that help them solve their issue and carry on. Of course, on every project there are committers doing the same thing, but in most cases they started doing all that before as contributors, and it's the work done as contributors that keeps the project going!

Now, what about who drives the project? On most projects, at least some of the people, and possibly almost all of them will be there in their role as employees of a company. How should committers (and pmc members) handle the need to wear two hats, those of an employee and as a committer to the project?

Within the project, new features are added by whoever does the work. This might seem obvious, but it's worth remembering that there's no magic coding fairy secretly committing while we all sleep - all new features and bug fixes come from someone spending time at a keyboard working on them. In some cases, that's someone working on it in their own time. More often though, it's someone working on company time, usually on a project that is important to their employer.

So, we can see that on many projects, the new features in the project will be those which matter to the companies providing the employees to work on them. But, does that mean that those companies get to set the direction of the project? In some foundations, the answer would be yes, but at Apache, the answer is a little more complex. It's maybe, but only if the community agrees!

How does this work then? Generally, people should announce in advance what they're working on, and seek feedback on it. That could mean opening a JIRA for the new feature, describing it, then later attaching the patch to it. It could be a quick post to the mailing list, followed by a later commit. For a very uncontentious change, it could even be committing the patch and describing it in the commit message, assuming that your project allows commit-then-review for that sort of change. Generally though, for a company-driven change, a committer will announce what the new feature they're planning to work on.

Once the new feature has been described, what then? It could be that everyone feels this is a useful addition, and +1's it. That's the ideal case - your company's needs closely match those of everyone else, and everyone's happy! Much more commonly, people will say things like "that isn't what I'd choose to work on first, but it's useful, so go for it" or "I wouldn't use that, and wouldn't code it, but I can see the value for others, so go for it". These would be more of a +0, hinting that while the change is good for the project, it isn't something that others care deeply about (which is perhaps why no-one has done it before...) In all these cases though, the community have agreed that the change that your employeer needs fits with their needs too.

In a few cases, you'll get someone saying "that's a stupid idea, you should be working on the thing that I need instead!". That's usually the point that you need someone else to step in (so as not to start a flame war), and explain that Apache is a participatory meritocracy. If there's a feature you want, you have three choices - code it yourself, pay someone to code it for you, or inspire someone to code it for you. No-one's under any obligation to write something for you, everything's done by people who want to do so!

Now, what about the case when you describe a new feature that you want to work on (either because your employer wants you to, or even just you want to in your own time), and the community isn't keen? That might be because it will have wide ranging changes, or introduce new dependencies, or will work using a new methodology, or half a dozen other things. Well, firstly, the thing to do is to hold fire on your changes. Instead, you need to make your case, probably on the mailing list, and explain why you feel your change is the right one to do. With any luck, people will suggest alternate ways of doing things without the problems, and you can get the new feature you need whilst also having the community support you.

What happens though if after that discussion, you haven't convinced the community? Generally, that's the time to try it anyway, but not on the main codebase! Let everyone know you're trying it out, and do so on a branch, or in an external SCM, depending on the scope of the change. Try to let the community see what you're up to. Then, when you're done, come back and make your case again. Hopefully now things will be easier, because you can point to real code, and say things like "there had been concern that the new configuration system would slow down the core, but if you try it with this benchmark, you'll see it's actually 5% faster". It could be that when people see the new code, they'll agree, you have a quick vote, commit the change, and everyone's happy :)

And if even having seen the code, the community doesn't agree? Well, sorry, but you'll need to maintain a vendor fork at this point. After all, your company needs it, and the community doesn't want it, so it'll be up to you to maintain. Next time, try to make the case to the community better...

In general though, the project moves in the direction of those with the time to code. When those people are working on company time, the project will tend to move in the direction that that company wants it to. However, that's only the case while the community agrees with the direction, and if the company tries to push something the community doesn't want, that all changes.

Apache - powered by the people who do the work, but directed for and by the community!