When I started to work at Rapid7 almost a year and a half ago, one of the first things I thought about was: "How can w3af benefit from all the methodologies, tools and ideas that Rapid7 uses to create NeXpose?", and without using too many brain cycles it was clear that Agile development methodologies (and more specifically SCRUM) was one of those great things.

During the first months as a Rapid7 employee it was very difficult for me to spend any time developing for w3af, and the hiring of our Python developer was still an on-going project. But in September Javier Andalia joined our w3af team as a full-time employee and we finally started to work on those very needed improvements.

Our w3af team organization was very simple. I was acting as a ScrumMaster Product Owner and Javier was the only team member of that small group that we called "Owls". We organized our backlog and wrote more specific user stories for the ideas we had; prioritized the user stories based on the users' needs and grouped them into small chunks of work (sprints). Our world suddenly felt more organized.

But we were forgetting the most important group of all in our organization: the w3af contributors.

Our first stab at the problem was to ask the contributors to commit to delivering a piece of code before the end of the sprint we were currently working on. That failed. Contributors write code for the fun of it, due dates remind them of their 9 to 5 work. We quickly realized this and moved on to a different organization for the contributors.

The next solution was to have all the contributors be team members of a different scrum team called "Athenians". That team had longer sprints (4 weeks instead of the 2 we have in the Owls) and I would still be the ScrumMaster Product Owner for that team. After creating the team and milestones in Trac, we carefully selected the most interesting tasks of our backlog and assigned them to the Athenians-1 to Athenians-3 sprints. We had everything ready for our contributors to start working on those tasks! That also failed. We were still getting code contributions, but none of them were from the list we prioritized. Contributors write code for the fun of it, someone telling them what they need to work on reminds them of their 9 to 5 work.

Understanding this last thing was a little bit more difficult than the first one, but we managed to get there and change our team organization once again. Our current organization for driving the efforts of our contributors is as simple as it can get:

  • Do whatever you feel like doing,
  • Do it whenever you want to do it.

This might sound like chaos, but it's actually not. We're giving our contributors the freedom they want, while keeping only one thing on our side: the how. By reviewing all code that's sent by the contributors we can keep the quality of it to the highest levels. If their code isn't properly tested, breaks any of our unit tests, etc. it won't make it to the trunk and that's not negotiable.

I hope this blog post helps other open source project leaders that have decided to use SCRUM or any other development methodology. My objective is far from defining the way we should all use Agile methodologies with open source projects: my real intention is to tell the community about our specific experience, which should help others that are going through similar paths.