Hello everybody, my name is Andrew Kazyrevich and I'm going to talk about distributed agile development.
When people communicate face-to-face they convey information in several ways, not only with the words they use. Their posture, the distance they sit from one another – closer or further, they way they sit, reveal their attitude. You can see if they pay attention, or if they’re interested, or if they disagree... or if they’re bored.
Because of this face-to-face discussions usually go smoothly. Of course there may be misunderstandings – like on this slide, the guy at the left talks about a black square, but the other guy thinks about, um, something different.
But it’s natural, even in face-to-face communications there can be misunderstandings. However, most of the time they’re easy to spot...
...and everything gets resolved pretty quickly.
Unfortunately it’s not always possible to have everybody in the same room, and there are different reasons for that.
- One reason is global markets. Your business expands, you buy companies in remote countries and open up branch offices, and one day – boom – you have a distributed team.
- Another reason is global talent. You may be looking for experience that is easier to get offshore. Or maybe you’ve found great developers but the cost of relocation or visa availability is pushing you towards working with them remotely.
- Then, companies often dream to reduce costs through outsourcing to countries where developers are less expensive. Whether it works out or not is a different question, but nevertheless it's one of the reasons for having a part of your team formed from cheap and genius guys that work from the middle of nowhere.
In any case, regardless of why you have decided to go for a distributed team, it’s important to remember that distributed development has hidden costs. It surely has its benefits as well, but all those benefits come at the cost of reduced performance, hampered communication, lost context, cultural challenges, increased chance to get this "us versus them" attitude and so on.
And the key to succeed in a distributed environment is to understand the impact of distribution, to avoid it where possible, and to mitigate it where you cannot avoid it.
But let’s step back a bit now and talk about agile, what does it mean – being agile? How many people in this room work in an agile environment? [One hand raised - and it was mine] Uhm.. err.. okay, so then it's a really good idea to go through the core agile values. Later on we'll see how distributed agile teams can adhere to the same principles.
So agile manifesto has four Main Values and important thing about those claims is that while you should value the concepts on the right hand side, you should value the things on the left hand side much more. It's all about preferences, not alternatives.
The first value is individuals and interactions over processes and tools. Teams of people build systems together, and to do that well they need to work together effectively, all those programmers, testers, managers, designers, customers, etc. People can adapt to changes, they can learn and evolve, but tools and processes cannot really adopt and evolve on their own.
And that’s why most important factors that you need to consider are the people and how they work together. Because if you don’t get that right – the best tools and processes will not help you.
Then agile teams value working software over comprehensive documentation. Now if you ask your customers, would they want a document describing what you’ve built or the actual software, what do you think they’ll choose?
Of course it's an oversimplification and I’m not saying documentation is bad, but agile makes a shift away from it. The plan is to produce software quickly and often, concentrating on delivering a working product...
...and the key thing here is "working". Then the product gets enhanced iteratively, instead of spending a lot of time on exhaustive documentation up front and then building a system around it.
Then agile teams believe that customer collaboration is more important than contract negotiation. It’s also kind of obvious – here are your customers and only they can tell you what they want... problem is they perhaps cannot specify what exactly they want, so working closely with customers and collaborating with them is hard.
But it pays off because you (hopefully) end up building what they need and you can adapt to the changes faster. Surely having a contract is important as well, so that everyone has an understanding of responsibilities, but a contract should not substitute communication.
And finally agile teams prefer responding to change over following a plan.
While building a system, your understanding of the problem eventually changes. Everything can change, from business model to requirements to technology. There is nothing wrong with having a project plan, but the plan should be malleable, malleable enough so that you could change it if the situation changes. Otherwise it would quickly become irrelevant. There’s a great quote from Dwight Eisenhower who was saying, "Plan is nothing – planning is everything".
So let’s look now at typical problems that distributed agile teams face, assuming that they want to stick to the same values – "people over processes", "working software over docs", "collaboration over contracts", and "ability to change over following a plan".
- The root of all problems is communication. Focus on people and communication allows agile teams to spend less time on "additional" practices, like writing detailed specs: their close contact can substitute most of the docs. All those casual conversations in the team room, all those talks over coffee, etc, can significantly contribute to the shared vision. And remote guys miss most of them and consequently their understanding suffers.
- Further issue is knowledge sharing: spreading ideas gets slower if people work remotely. Some agile practices, like Test Driven Development or pair programming, are learned better by example, by demonstrating them – you know, instead of reading through faceless documents you sit together with someone conversant with the subject, and you can discuss why something is good or bad... Remote teams have a completely different experience. Things that need a 10 minutes talk in a co-located team now require extensive communication and serious management.
- Then, if the level of distribution gets bigger, you face time zones. It’s somewhat funny to think that you’re sitting at work, fixing off some bugs – and at the same time it’s a deep night for some of your workmates, and there’s no way you can call them and say, "hey, here’s just a quick question..."
Inevitably work will be delayed because clarification is needed, or something has to be redone because your best guess didn’t pan out when reviewed by someone else on the other side of the globe. If such things happen regularly, it will definitely impact team’s health and morale.
- And don’t underestimate the culture differences. Well, this one has two different meanings. First, the major reason why organizations have problems with adopting agile methods, is the cultural change. "Command and control" model is still popular: Big Bosses make Serious Decisions and lower level workers just implement stuff. In this environment people are often discouraged from asking questions, talking about problems like impossible deadlines, or proposing alternatives to the approved decisions. So, if you want to introduce agile, you need to overcome this kind of attitude, and bring up the culture of openness, culture of giving more responsibility to people who actually do the work.
And there’s also a different kind of cultural issues. It’s not that I want to offend anyone, but some nations, people from some countries may sound authoritative, or they don’t add "please" at the end of every sentence, etc – while some other nations tend to sound very polite. It’s just cultural differences, nothing more, and it’s very easy to come across them in a distributed team. Imagine: you may be talking to someone who simply never says "no". And their polite "we think there might be a problem", it actually means "hey, this will never work!" – and if you're not prepared to this level of politeness, things can end up really badly.
So, now as we have discussed what does it mean being agile, and have looked at the issues that a distributed agile teams could face, let’s go through some practices that help to mitigate those issues.
First thing to say here is that the key word here is "mitigate". There’s just no substitute to a face to face discussion at a whiteboard, and there's no substitute for two developers pairing together on some piece of code. If your team is distributed, it misses a lot – and the impact of this should be minimized.
First thing that helps is instant messaging. Obviously you can use emails, but emails are more for asynchronous broadcasting – to tell something not extremely urgent to a lot of people. Drop off an email and don’t expect a reply too soon. Instant messaging, however, is used for urgent conversations, something you expect to get a reply within minutes, probably seconds. That’s why it’s a good idea to have display your activity status in the messenger, so that your remote colleagues can see when you’re away from your desk.
Instant messaging can also target a group of people and you’ll probably find it useful to create group chats: all people in a group receive messages that are sent to this group. One example is a chat of all developers on a project – when something goes wrong, a build has failed or part of the system isn’t working – someone who noticed that just drops a message to this "All Developers" chat and lo and behold, everybody knows the latest news. Well, I probably know what you’re thinking now. Definitely people will receive an email from your Continuous Integration system saying the build had failed. But it will take time. With the group chat it’s a matter of seconds and it could help other team members who were just about to get the latest version from source control.
You can also create a group chat per user story – developers, testers, designers, and analysts for a specific user story can raise questions, provide answers, and share knowledge in an extremely rapid way. Each time there’s a clarification of a requirement or some piece of functionality gets checked in, someone drops a message to the group, and the news gets delivered to everybody, regardless of the location.
And such kind of things makes everybody happy – you see, they’re smiling on the slide – and it feels as like the whole team is still working in one room. Sometimes it’s even better: for a distributed team, posting a message in a group chat is almost equal to shouting out in a team that is located in one room... apart from that you also don’t distract those who are not interested.
Sometimes talking is more efficient than typing – you can use hands free headphones and web cameras to facilitate voice communication. Well, everybody does conference calls, and here’s a piece of advice. If you’re not using web cameras, then until everyone recognizes each other's voice, try to introduce yourself each time before you start speaking. Just say your name, I know it sounds stupid, but it really helps people on the other end of the line, as they cannot see you. And, of course, be careful to have one conversation at a time – remote guys will probably miss everything if several talks go on in your local room. A good idea is to assign a conference call buddy, someone in the local team who will make sure that remote workmates wouldn’t miss anything from the call.
And, in general, try to always remember how hard such calls are for those who call in remotely. If you’re lucky to be in an onshore part of the team, and remote guys have to call in, have someone in your local team to call in as well, every once in a while, just to feel the same pain. So it will be like all of you are in a meeting room and one guy from your local team goes to his place and calls in, in the same way as the whole remote team does. Do it from time to time and you’ll probably find a few improvements to all this conference calls experience.
Then, most importantly, use conference calls when you absolutely need to talk. I mean, some companies use them for regular status reports. People call in, and one by one tell what they have done or plan to do. But – big news – those who are not talking on such calls are also not listening. They mute their microphone and check email and stuff. Keep that in mind every time you need to organize a conference call. People easily get distracted. How can you keep them engaged? For example, you can allow the remote team to lead the conversation and see what happens. Or, if possible, don't call everyone separately but establish a dedicated call point, say, have a separate Skype account for the whole remote team. Call on this account instead, the remote team will gather together in front of one computer, and there will be less distraction comparing to when people are sitting at their own laptops.
You can also have so called "passive" communication, like web cameras in geographically separated locations, so that members in one office can easily see how people from other offices are doing, are they at their desks or not.
Then let’s look at desktop sharing software. When people are working in the same room, it’s easy and natural to come to someone place and go through some weird bug together.
Desktop sharing makes it possible for remote team members to look at your screen and help you, and vise versa.
It’s an amazing feature – I have very little to say about it, but it’s so amazing that I have a dedicated slide. The latest version of Skype allows sharing desktop with one person, and there’s a bunch of free applications, like VNC that allow you not only to see other person’s screen but also use their mouse and keyboard.
So instance messaging and calls really help to rapidly share common information, but you also need a place to store all this knowledge. There’s a bunch of ways to do that, and lots of distributed teams put wiki as their favorite one. Wikis are easy to set up and simple to use, you don’t need to install anything to work with them – just open up a browser and start creating new pages. Any possible information can be put there: user stories, roadmaps, design guidelines, all your status reports, various tips and tricks, and so on.
It’s also amazingly easy to run discussions on the wiki: currently, a distributed discussion probably means that you send an email, a bunch of people reply, then even more people reply, all this lengthy email thread becomes difficult to follow, and information on the top is probably not valid anymore, etc. Instead of all that, you can set up a page on a wiki, where you explain the problem in detail – and just send a short email with a brief description and a link to the wiki page. Everyone can update the page, wiki keeps the history of changes and you can always see who said what and rollback to some previous version. You can also subscribe to updates and get notifications every time someone changes this page.
Wikis are not structured, and that’s a great benefit. Remember I was talking about "people being more important than processes and tools". So here, with wiki, people can evolve their own structure as the project grows. However, this also means that, as time goes, some pages will become badly structured, obsolete or just plain incorrect. Someone needs to fix that, so either have a dedicated person who would keep your wiki up to date, or assign different people responsible for that, so that every single page has someone who keeps it clean. It shouldn’t take much time, probably a few hours every month, but you’ll have some clean and correct documentation.
And it brings me to another challenge. Regardless of whether you’d be using wiki or something else to store your documentation, unfortunately on a distributed team you will need more of it.
Agile pundits often claim that your efforts put on docs are largely wasted – and that is, well, all true if you have a small local team. Things get changed pretty frequently, and anyway you can discuss everything in person anytime. But communication between remote offices is not that easy, so docs become more important. It's a just part of the price of going offshore. There’s a great quote from Matt Simons who is a Chief People Officer at ThoughtWorks so he is workingwith Martin Fowler and other übersmart guys, and once he was writing an article about offshore development, and he said, "There are two keys tosuccessful documentation on agile projects. The first is finding thepoint of "just enough" documentation... The second key … is to not get attached to it or have unrealistic hopes of keeping it updated". Usually, documents get created and then nobody has time to keep themfresh. So as Matt says it's often better to produce a new doc every time you really need it. The nice part of it is that if each time you have to start over so you don’t write long articles. You keep things brief, clean and efficient, and that’s a good thing.
But the price of more documentation is just a tiny part. There’s a bigger price you have to pay for going offshore: travelling. Even in a local team people work better when they know each other personally, and in a distributed team it’s much more evident.
If possible, at the beginning of a project you should gather the whole team for a few iterations. People would get to know each other, and then when they’d work together remotely, personal bonds will make communication a lot easier. Also, many key decisions are made at the start of a project, and it’s better to have the whole team involved in making decisions and committing to them.
Generally, there are two kinds of travelling.
It also might be a good idea to send people over when a significant change is planned, like a new system design or a major refactoring.
- Seeding visits create personal relationships. They happen early in the project (or when a new team member joins the team) – and, if possible, should be pretty long, at least two weeks or more. The point of those visits is to get people used to working with each other, but remember that it’s also about building the relationships. Make sure you reserve time for human communication between team members, because the goal is not only "completing tasks".
- Once the seeding is done, you need maintenance visits to keep those personal bonds. Such visits can be shorter but should be more or less regular: say, a one-week trip every half a year.
Then, I’d like to talk a bit about team organization. Communication within a distributed environment is hard, and some companies address that by splitting teams by activity. Offshore teams become overspecialized in some parts of the system – say, they may be developing only UI, or just one isolated sub-component, or they do all implementation while design andacceptance testing is done locally, and so on...
By doing so, you decrease the need for communication between teams, and it looks like the whole team is still doing well. But please don’t do that – such approach brings even more problems in the long run. Because your users and customers are not interested in components and architecture.
They care about user stories, and user stories typically go through all layers – from database up to UI. If your remote teams get overspecialized in components, eventually user stories will be harder to complete, because very soon you will have cross-component stories, and in order to complete them, geographically distributed teams would have to go through specific pieces of work on a critical path, and still they would need to communicate.
Software development is about completing user stories, not about creating components and assembling them into a final product.
You need to make tasks from a single story and distribute them across the whole global team, regardless of geographical locations. This is probably the most difficult thing in distributed development – as it requires communication between different teams – but it makes a consistent user experience and consistent design, especially for functionality that "falls in between", something that is not really related to a particular component.
So that's pretty much it, here on the slide you see the list of things that help to minimize the impact of distribution. I didn't discuss some of the items during this talk, so here's the place to mention them briefly.
- Group chats and frequent calls help to spread knowledge across the whole team. Pair programming is something of the same nature, and desktop sharing allows geographically separated people use it as well.
- You need Continuous Integration to be sure that your product is always builable and deployable, and you need unit tests to make sure it still works. Both are extremely useful in a colocated agile team, but in a distributed agile team you simply must have them. It's like using source control system, you simply cannot start developing without it.
- Then, a work item tracking system allows you to keep tabs on what anyone in the team is up to, without having to contact people directly.
- Short iterations and daily sync-ups keep everyone on the same page. Communicate often, to make sure you're still building the right system. Try to keep communication open, to foster an environment where ideas come from different people and get discussed collectively. Instead of writing an email to one person, send it to a group. Or, even better, set up a wiki page. Obviously, not everything can be discussed in such a way, but if you can share something – don't hesitate.
- Plan to travel, to build personal bonds and to keep relationships going. This really helps when geographically distributed people have to solve pretty much any kind of problems.
- And organize teams by functionality, assign tasks to groups of people regardless of locations. Programmers in different offices involved in the some common task will coach their local workmates and knowledge transfer in general will speed up. Remote business analysts will provide clarifications to developers on their side and thus minimize delays. Similar skills for people in distributed teams would help to avoid isolation of a team, and you wouldn't face a dangerous situation where only one remote team can implement or maintain a specific user story.
Here's some literature that you might find useful. I don't provide links: they're long, they can possibly change, and everything is freely available online anyway. Just open up your favorite browser and type the name of any paper.
Thank you for your attention, and now questions please!
[There were a bunch of questions but I don't really remember them well... If you think your question and my answer are worth putting here, please drop me a line at andreister | AT | gmail | DOT | com and I'll be glad to add them.
Also, here are the links, as I promised at the presentation. The ones not mentioned on the slides are mostly research papers and might be of interest if you already know what distributed agile is, and want to learn more.
If any of the links is broken please let me know and I'll update.
Also, the ppt itself is available here.]