12 Agile principles

“Do As I Say, Not As I Said”

Imagine you’re on a team building the very first handheld ebook reader. The hardware team delivered a prototype device that has a USB port that you can load ebooks into, and a little keyboard that lets you interact with it. It’s up to you and your team to build the software that will display ebooks to the user.

Unfortunately, your company has a long history of building software using a particularly ineffective, “big requirements up front” waterfall process. So the first thing that your project manager does is to call a giant meeting with everyone she can find. Your whole team spends the next few weeks in a room with, at various times, the senior managers of your company, a representative from a friendly publisher who wants to publish ebooks that your reader can display, a senior salesperson from an online retailer hoping to sell those books, and any other stakeholder that your project manager can dig up and get to come to a meeting.

After days of intense meetings and heavy debate, your business analysts were able to piece together a large specification with requirements from all of the different stakeholders who were consulted. It was a lot of work, but now you have a spec that everyone thinks is great. There are extensive features for the user that would make it the most advanced handheld reader software available. It includes features to capture marketing statistics for publishers, it provides an entire Internet storefront to make it easy to buy books, and it even has an innovative feature for authors to preview and edit their books as they write them, streamlining the publishing process. This would be truly revolutionary software. After you sit down with the rest of your team to estimate, you’ve come up with a schedule of 15 months. It seems like a long time, but everyone is excited, and you’re confident that you and your team can deliver.

Fast-forward a year and a half. The ebook reader team has worked unbelievably hard, putting in many late nights and weekends, and putting stress on more than a few marriages. It was a huge effort, but the project is done, and you’ve delivered exactly on plan, practically to the day. (Yes, that seems highly improbable! But for the sake of this thought experiment, try to suspend your disbelief and imagine that it happened, just this once.) Every requirement in the spec has been implemented, tested, and verified as complete. The team is really proud, and every stakeholder who’s seen it agrees that they got exactly what they asked for.

So, the product hits the market…and it’s a flop. Nobody buys the reader, and none of the stakeholders are happy. So what happened?

It turns out that the software everyone needed a year and a half ago is not the same software that they need today. In the time since the project began, the industry standardized on a new, industry-wide format for ebooks, and because that format wasn’t in the spec, it’s not supported. None of the Internet retailers want to publish the nonstandard format that the reader uses. And even though the team built that great Internet storefront, it’s now far less advanced than the storefronts that the retailers currently use, so it’s not attractive to them anymore. And all the work that you and your team put into building the special preview feature for authors isn’t nearly as useful as a competitor’s support for authors to email MS Word documents straight to their readers and display them.

What a mess! The spec that your team started with had a lot of value to all of the customers, both inside and outside the company. But now the software your team agreed to build a year and a half ago is much less valuable. Some of these changes could have been discovered early on in the project, but many of them would not have been clear from the beginning. The team would have had to change course very quickly at many points during the project in order to take them into account. The “big requirements up front” waterfall approach doesn’t give a team much flexibility to respond to these changes.

So how can we find a better way to satisfy stakeholders and customers, but still run a project that delivers software that works?

Principle #1: Our Highest Priority Is to Satisfy the Customer Through Early and Continuous Delivery of Valuable Software.

This first principle includes three distinct and important ideas: releasing software early, delivering value continuously, and satisfying the customer. To really understand the core of this principle, you need to know how those three things work together.

Project teams work in the real world, and in the real world things never go perfectly. Even a team that does a brilliant job of gathering and writing down requirements will miss things, because it’s impossible to completely and perfectly capture the requirements for any system. That’s not to say that teams shouldn’t try, and agile methodologies are based on some great practices for communicating and writing down requirements. But the fact is that until customers get their hands on working software, they have a hard time envisioning exactly how the software will work.

So if the customer can only give you real, informed feedback after seeing working software, then the best way to get that feedback is through early delivery: shipping the first working version of the software to the customer as early as possible. Even if you only deliver a single working feature that the customer can use, that’s still a win. It’s a win for the team, because the customer can now give informed feedback to help them move the project in the right direction. It’s also a win for the customers, because they can do something today that they couldn’t do before—with this working software. Because the software works, and because the customers can actually use it to do something they need, the team has delivered real value. It might only be a small amount of value, but it’s far better than delivering no value at all—especially if the alternative is an increasingly frustrated user who has to wait a long time before getting his or her hands on software from the team.

The downside to delivering early is that the software that’s first delivered to the customers is far from complete. This can be really hard for some users and stakeholders to deal with; while some users are used to the idea that they’ll see software early, others are much less so. Many people have a hard time when they are handed software that’s not perfect. In fact, in a lot of companies (especially larger companies where people have spent years working with software teams), the software team has to literally negotiate the terms under which they’ll provide software to their stakeholders. When there’s not a great partnership between the team and the people they build software for, if the team delivers incomplete software, the users and stakeholders may judge it very harshly and even panic if a feature they expect to see is missing.

The core agile values have an answer to this: customer collaboration over contract negotiation. A team that’s bound to a fixed specification with inflexible bureaucratic barriers to change doesn’t have the option to let the software evolve over time. A team working under those conditions has to kick off a whole new change management process, which requires a fresh set of contract negotiations with the customer. On the other hand, a team that truly collaborates with the customers has the option of making all the necessary changes along the way. That’s what continuous delivery means.

This is why agile methodologies are typically iterative. Agile teams plan the project’s iterations by choosing the features and requirements that will deliver the most value. The only way that the team can figure out what features deliver that value is to collaborate with the customer and incorporate the feedback from the previous iteration. This lets the team satisfy the customer in the short run by demonstrating value early, and in the long run by delivering the finished product that delivers as much value as possible.

Principle #2: Welcome Changing Requirements, Even Late In Development. Agile Processes Harness Change for the Customer’s Competitive Advantage.

Many successful agile practitioners have a lot of trouble with this principle when they first encounter it. It’s easy to talk in abstract terms about welcoming change. But in the heat of a project, when a team runs into a change that will require a lot of work, things can get emotionally charged—especially for a developer who knows that his boss will hold him to the old deadline, no matter how much work the change requires. Getting past this, especially in a culture where people are blamed for delays, can be difficult. But it can also be very rewarding, because welcoming changing requirements is one of the most powerful tools in the agile toolbox. 

Why are changes in a project emotionally charged? Understanding that is a key to this principle. Think back to the last time you were working on a project and found out that you needed to change what you were building. How did it feel? Until that moment, you thought the project was going well. You probably made a lot of decisions: how to structure your work, what you were building, and what you promised to deliver to your customers. Now someone outside of the project is telling you that some of that planning and work was wrong—that you were wrong.

It’s very difficult to accept someone telling you that you’ve been wrong, especially if you’ve been doing work for that person. Most software engineers are driven by pride of workmanship: we want to deliver products that we can stand behind, and that satisfy our users’ needs. A change in a project threatens that, because it’s questioning the path that you’ve taken and the assumptions that you’ve made.

Very often, someone is telling you to change course after explicitly setting you on that course. If that person told you to build one thing, and you went ahead and built half of it, it’s very frustrating for that same person to then come back to you and say, “Actually, now that I think about it, can we build something completely different?” It feels disrespectful of the effort that you put in. Now you have to go back and make changes to things you thought were done. It’s hard not to get defensive about that! And it gets even worse if you’re the one who’s held responsible for not reading your customer’s mind and, as a result, blowing your deadline.

Almost every professional developer has been through that situation at least once. In light of that, how can we bring ourselves to welcome changing requirements?

The first step toward welcoming changing requirements is to try to see things from the customer’s perspective. This is not always easy, but it’s enlightening. Do you think that the customer who originally sent you down the wrong path meant to? What do you think went through his head when he realized that he told you months ago to build the wrong software, and now you’ve wasted months of work because of what he said? Coming to you and requesting the change is, to him, admitting that he made a mistake—a mistake that will cost you a lot of work. That’s not easy to do. No wonder customers often wait a long time before coming to the team with a change! It’s embarrassing, and they know that they’re delivering bad news. Your deadline is going to be blown, but so is his. If he has needs, and if the company is spending money building software to meet those needs, then his needs aren’t getting met, and the project isn’t delivering value. And it’s all his fault, because he told you the wrong thing at the start of the project.

In other words, two people are being asked to do the impossible. You’re being asked to read the customer’s mind. He’s being asked to predict the future. When you look at it like that, it starts to become a lot easier to welcome change. On the other hand, if you want to resist project change and stubbornly stick to the same plan that you wrote at the beginning of the project, it’s easy—just make sure to hire only telepathic and clairvoyant team members.

So what does it mean to welcome change? It means:

• Nobody gets in “trouble” when there’s a change. We acknowledge—and our bosses acknowledge—that we’re human and fallible, and that it’s better for the company to allow us to make mistakes and correct them frequently, rather than expect us to do things perfectly the first time.

• We’re all in this together. Everyone on the team, including the customers you’re collaborating with, owns the requirements and the changes to those requirements. If those requirements are wrong, it’s as much your fault as the customer’s, so there’s no sense in placing blame for a change.

• We don’t sit on change until it’s too late. Yes, it’s embarrassing to have made a mistake. But because we all recognize that, we simply do our best to fix it as early as possible. That way, the damage is limited.

• We stop thinking of changes as mistakes. We did our best with the information we had at the time, and it only became clear that we were wrong because the decisions we made along the way gave us the perspective to recognize the changes that we have to make today.

• We learn from the changes. This is the most effective way for a team to grow and get better at building software together.

Principle #3: Deliver Working Software Frequently, from a Couple of Weeks to a Couple of Months, with a Preference to the Shorter Timescale.

You might be thinking that the idea of welcoming changing requirements is interesting, and could help your projects. But you might also be thinking that it sounds terrible. This is not an uncommon reaction to the idea. Many people who work on software teams—especially traditional project managers—are very uncomfortable with the idea of welcoming changes when they first encounter it. These project managers deal with changes every day, but the agile attitude toward change is very different than what they are used to. Agile practitioners refer to the traditional attitude toward project changes as command-and-control.

The term “command-and-control” is borrowed from the military. In our 2010 book, Beautiful Teams, we interviewed Northrop Grumman chief engineer Neil Siegel, who gave us a military definition of the term:

Andrew: I’m not familiar with military systems—what’s a command-and-control system?

Neil: An information system for military commanders. It allows them to communicate with each other, and allows them to maintain situational awareness: where everybody is, what their status is. It’s how a commander can understand what’s going on. Battlefields used to be small areas—a commander could stand on a hill with a pair of binoculars and could see what was going on. But by about 1900, the battlefields started getting so big that you couldn’t stand on the hill like Napoleon did. You had to start using technology to “see” the entire battlefield. And the systems that do that are called “command-and-control systems.”

Command-and-control project management is similar to military command-and-control:

• “Command” refers to the way the project manager assigns work to the team. The team may not report directly to her, but she has control over their assignments. She breaks down the work, builds a schedule, and assigns activities to the resources on her team.

• “Control” refers to the way she manages changes. Every project has changes along the way: work takes longer than anticipated, people take sick days or leave the project, hardware is not available or breaks, and all sorts of other unexpected things happen. Project managers constantly monitor for these changes, and then control the project by evaluating each change as it occurs, updating the plans to incorporate the change into the schedule and documentation, giving the team new assignments, and managing their stakeholders’ expectations so that nobody is blindsided.

The reason a traditional project manager is uncomfortable with welcoming change when she first encounters it is because she recognizes that those same problems will happen on agile projects too, and the team needs to be able to respond to them. Simply accepting changes and welcoming them seems like a guaranteed way to introduce chaos to the project. If agile teams don’t use command-and-control, how do they manage to keep up with all of those changes while still dealing with the normal day-to-day issues that project teams always face?

The key to welcoming changes without introducing chaos lies in delivering working software frequently. The team uses iteration to break the project into regular deadlines. During the iterations, the team delivers working software. At the end of each iteration, the team holds a demo to show the customer what they built, and a retrospective to look back and see what lessons can be learned. Then they start a planning session to figure out what they’ll build in the next iteration. The predictable schedule and the constant checkpoints help the team catch those changes early, and give everyone a blame-free environment where they can discuss each change and come up with a strategy to incorporate it into the project.

This is where agile starts to become very attractive to a traditional command-and-control project manager. A command-and-control project manager wants to control the deadline. Timeboxed iterations give the project manager that control. This also solves one of the project manager’s biggest challenges: dealing with changes that come in very late in the project. One of the most difficult parts of a traditional project manager’s job is monitoring for changes. The daily reviews and iteration retrospectives let the project manager enlist the whole team as eyes and ears to spot those changes much earlier, before they can cause more serious problems with the project.

The project manager’s role starts to shift away from command-and-control, where she gave the team a daily battle plan and constantly adjusted to keep them on track. Instead, now she’s working with the team to make sure that each person is constantly looking at the big picture and working toward the same goals. It’s easier to do that when the team is working on a short iteration that will deliver working software. This gives each person concrete goals, and a much better idea of what everyone is delivering—and a sense that each person is responsible not only for what he built, but also for what the entire team will deliver at the end of the iteration.

Better Project Delivery for the Ebook Reader Project

How can these principles help our troubled ebook reader project? Think back to the problems that the project team faced: their product flopped because it was missing important features that their competitors had (supporting an industry standard format for ebooks, and letting users mail documents to their device), and it had features that were no longer relevant to their market (an Internet storefront).

Let’s run the project again, but this time let’s have the project manager work with the stakeholders and the team to set one-month sprints. The project turned out very different this time:

• After the third sprint, one of the developers reported that the industry standard for a new format for ebooks had been approved. The team decided to implement a library that supports it in the fourth sprint, and incorporate it into the reader’s user interface in the fifth iteration.

• After 10 months of development, they created a working build that can be loaded into prototype machines and given to early beta users. The project manager talked to these users and discovers that they would really like a way to get Microsoft Word documents and newspaper articles onto their readers. The team dedicated part of the next sprint to building email integration into the reader, so users can mail articles to their devices.

• One year into the project, the stakeholders let the team know that the Internet storefront is actually not needed after all, now that the retailers all use the standard ebook format. Luckily, this feature was always low on the backlog and other sprints were dedicated to more important features, so very little work had been done on it.

Because the team had always delivered working software after each sprint, eliminating the features from the backlog meant that they could deliver early! The publishing partner was ready with books, because their senior managers all got early versions of the software and prototype hardware to play with. That kept them involved, and gave them incentive to make sure they had books ready to go as soon as the first version of the product was ready.

By using continuous releases, welcoming changes, and delivering working software after each iteration, the ebook reader project was able to deliver a successful product much earlier. Unlike the inefficient waterfall process, which sealed the team off from the customers once the requirements were approved, the agile team stayed engaged with their customers. That let them respond to changes and build a better product.

But things aren’t perfect on our ebook reader team—far from it. They’re using iterations to deliver working software, but they’re getting buried under documentation. Everyone is really happy that they’re not stuck building software that won’t sell. But every time the team finds a good change they need to make to the project, half the team gets stuck going back to the spec and updating it so that their plans stay up to date and they stay on course. It seems like they’re burning as much effort updating documentation as they are writing code.

People on the team have talked about ways to cut down the amount of work it takes to maintain all of these documents. They’ve had extensive discussions about the “right level of detail” for their documentation. But whenever they try to cut something out, someone always brings up a (valid) point that if they don’t write down that particular feature, requirement, design, or test case, then someone might misunderstand it. If it gets implemented incorrectly, they’ll get blamed for it. So it seems like every piece of documentation is needed, because otherwise the team will be on the hook for building the wrong software.

Is there any way they can reduce this burden without hurting the project? Is there even such a thing as a “right” level of documentation for a project?

Principle #4: The Most Efficient and Effective Method of Conveying Information To and Within a Development Team Is Face-To-Face Conversation.

Agile practitioners recognize that documentation is just another form of communication.⁴ When I write a document and hand it to you, the goal is not to write documentation. The goal is to make sure that the ideas in my head match the ideas in your head as closely as possible. In many cases, documentation is a good tool to accomplish that goal—but it’s not the only communication tool that we have.

Face-to-face communication is almost always a better tool for sharing ideas within a software team than documentation. Everybody knows that talking through a problem in person is the most effective way to understand a new idea. We’re all much more likely to retain ideas that were expressed in conversation than if we’d read them on a page or in a Microsoft Word document. That’s why agile communication practices focus most on individual people communicating with each other, and reserve documentation for those cases where complex information needs to be recalled in detail later.

Luckily, it’s not hard for teams that are used to comprehensive documentation to move to more effective face-to-face communication. That’s because most of those teams don’t actually try to hit the “ideal” of complete comprehensive documentation and traceability. Software engineers are notoriously practical. When they see how much work is involved in building up that comprehensive documentation, they end up having those face-to-face conversations anyway. In the end, that’s the only way they can effectively build software. By recognizing that comprehensive documentation is often inappropriate, they can stop feeling guilty for not accomplishing the impossible—building perfect, comprehensive documentation—because it would not, in fact, be particularly useful for their projects if they did!

Instead, the most efficient way for the team to communicate the important ideas needed for the project is to get everyone thinking the same way. That way, each person can handle every decision as it comes. When a group of people look at the world the same way, and communicate about both positive and negative ideas openly, they end up with a shared perspective. When a change comes along that requires rethinking, the team members don’t have to explain as much to each other.

The ultimate goal for team communication is to create a sense of community so that there’s a lot of implied knowledge, because it’s just not efficient to explain the same thing over and over again. Without that sense of community, people in different roles have to work harder to match each other’s perspectives. The closer the team gets to having a sense of community and a shared perspective, the more often each person will independently come to a similar answer when posed the same question. This gives them a much more stable ground for handling changes, because they can get past the conflicts and start working on the code—and won’t have to get sidetracked managing documentation.

Principle #5: Businesspeople and Developers Must Work Together Daily Throughout the Project.

Agile teams sometimes forget that the businesspeople they work with have day jobs. This leads to a natural disconnect between software teams and the businesspeople they build software for. 

To build software well, teams need to have a lot of face-to-face discussion with businesspeople, because they need to tap into their knowledge of the company’s problems that are going to be solved with software. The businesspeople got that knowledge by solving those same problems without software. So for most businesspeople who work with software teams, the software project is typically a small part of their job. For most of them, in an ideal world, the team would have very little contact with them. They’d have a meeting or two, figure out what the software needs to do, and then come back a short time later with perfect, working software.

The teams, on the other hand, want as much contact with the businesspeople as possible. Programmers need to learn about the business problem that needs to be solved. They do this by talking to businesspeople, watching them work, and looking at what they produce. A programmer who needs this information would love to have each businessperson’s full-time attention. The longer they have to wait to get their questions answered, the slower their projects progress. But the businesspeople don’t want to spend all of their time with the software team, because they have jobs to get back to. So who wins?

Agile teams have a way past this problem that lets both the businesspeople and the developers win together. They start with an understanding that the team is delivering valuable software to the company. The finished software is worth money to the company. If that value is more than the cost to build the software, it’s worth it for the company to invest in its development. A good project should be valuable enough that the businesspeople can see that it’s worth the effort they’ll need to put in over the course of the project.

When businesspeople and developers build software as a team, it’s most efficient in the long run to work together daily throughout the project. That’s because the alternative is to have the businesspeople wait until late in the project to look over the team’s work and give feedback, and it’s much more expensive to make changes at the end of the project. It takes a lot less of everyone’s time if they catch those changes as early as possible. Working daily with the team actually requires less of each businessperson’s time over the course of the project.

This is why teams that deliver working software frequently should prioritize the most valuable features first—so the businesspeople can get that value delivered early. It’s part of the bargain. It’s also why good agile teams consider the businesspeople to be an important part of the team, and equal members with the programmers. That’s a big distinction between an agile team and a traditional team. The traditional team treats the business user as a customer to be negotiated with; the agile team collaborates with the customer (typically the product owner) as someone with an equal say in the way the project is run. (That gets back to the core agile value of customer collaboration over contract negotiation!)

A good product owner can help reduce the amount of time the businesspeople spend with the team. They still need to meet daily, but the product owner can concentrate on understanding the value of the software and the business problem that needs to be solved. That way, the team can use their face-to-face time with the businesspeople to validate information that they’ve already learned from the product owner.

Principle #6: Build Projects Around Motivated Individuals. Give Them the Environment and Support They Need, and Trust Them to Get the Job Done.

Projects run best when everyone in the company recognizes that the team is building valuable software, and when everyone on the team—including the product owner—understands what makes the software valuable to the company. 

On the other hand, projects can break down in an environment where people don’t see the value in the software, or where they aren’t rewarded for building the software well. It’s actually not uncommon for companies to set up performance reviews and compensation systems that discourage people from building software in an effective, agile way, and this can work against the project. Some common “incentives” that work against agile teams include:

• Giving programmers poor performance reviews when code reviews routinely turn up bugs, and rewarding them for “clean” code reviews. (This just leads programmers to stop finding bugs in code reviews.)

• Rewarding testers for the number of bugs that they report. (This encourages nitpicking and poor reporting, and discourages them from partnering with the programmers, because it sets up an antagonistic relationship between them.)

• Basing business analysts’ performance reviews on the amount of documentation they produce (rather than the amount of knowledge they’ve shared with the team).

Ultimately, everyone’s performance should be based on what the team delivers, rather than the specific role that they played in it. Now, that’s not to say that a programmer who does a poor job or is disruptive shouldn’t be called out for it in a review with his boss. People should be reviewed based on their contribution (or lack thereof) toward the whole team’s goals. But they should definitely not be discouraged from looking beyond the strict confines of a specific role. A good environment for a team will reward a programmer who recognizes a part of the business problem that isn’t being addressed and fixes it, or a tester who sees a problem with the code or architecture and raises that with the team. An environment like this gives everyone on the team the support they need, and makes the project more successful.

Comprehensive documentation and traceability matrices are a particularly insidious source of problems with a team’s environment and support. Instead of encouraging an environment of trust, they encourage a “cover your ass” (CYA) attitude in which teams move toward a “contract negotiation” approach, rather than customer collaboration.

A tester with a CYA attitude will spend time making sure every requirement has a test, whether or not it does anything to actually help the quality of the software. Developers will CYA by adhering to the letter of the requirement, without bothering to think about whether or not it actually delivers value to the users—because if they’re working in a CYA environment, trying to build what the customer actually needs can get them dinged for not building the software to spec. Business analysts and product owners will CYA by spending their time making sure that the scope and requirements line up perfectly, and often find themselves tempted to omit pesky requirements that don’t align perfectly with the existing documentation, regardless of their value.

Software team members need to CYA most in an environment where change is seen as a bad thing. For a team that uses comprehensive documentation, it’s easy to see change as a bad thing, because it requires so much work to revisit the scope, update the specs, alter the design, repair the traceability matrices, etc. This leads to a divisive environment, because managers in that sort of company will naturally try to find a “culprit” who can be blamed for all that extra work caused by the change. People on the team increasingly turn to “defensive documentation” to protect themselves when that blame inevitably comes around to them. This forces everyone on the team to CYA, so they can point to the part of the documentation that they were sticking to, in order to avoid a poor performance review or punishment.

The opposite of CYA is trust. A company that develops only the minimal documentation needed for the project has an environment where the team is trusted to do the right thing when a change happens. An agile team with a “we’re all in this together” attitude, where if the project fails everyone shares the blame, doesn’t need to CYA. It’s easier for them to handle changes, because they don’t need to maintain all of that unnecessary documentation. Instead, they can tackle the actual problem with face-to-face communication, and only write down what’s necessary. And they can do this knowing that the company trusts that they’re doing the right thing—even if it causes the project to take longer.

Better Communication for the Ebook Reader Project

Our ebook reader project could definitely have benefited from better communication. Remember the days of intense meetings where the business analysts carefully pieced together an extensive set of comprehensive requirements? The requirements didn’t start out as a cynical CYA scheme. Everyone on the team genuinely thought that by arguing out every minute aspect of the software for days, they would cover all of their bases, and that would lead to the best possible product. Because they had spent all that time up front, they were able to stick to their guns and defend the original spec, even when it seemed like the end product might not be viable on the market. And if they’d been able to predict exactly what the market needed two years in advance, it would have worked out perfectly! Too bad it didn’t work out that way—but at least nobody lost their jobs, because they could all point to the spec that they implemented precisely.

What would have happened had they used better communication from the beginning? How would the product have changed if instead of writing comprehensive requirements, the team had written down the minimum amount of documentation they needed to get started?

They would have had to trust each other to make the right decisions along the way. This would have helped a lot when it came time to work on the ebook format, because they wouldn’t have been tied to an outdated format specified at the beginning of the project, and they could have adopted the newer format instead. Even better, by the time they started work on the Internet storefront, it might have been obvious that it wasn’t a good idea, and they could have simply abandoned it—which isn’t an option if it’s part of a specification that the team is on the hook for. Better communication would have let them keep the project up to date, and could have delivered a more valuable product.

Let’s imagine that this happened for the ereader team. Now our team is much happier with the documentation that they’re writing, because it’s much less of a burden. They thought they’d save lots of time by communicating better and cutting out unnecessary documents. But what happens if even though they’ve done this, the project still isn’t on track?

Somehow, that time savings never materialized. It seems like they’re putting in more nights and weekends than ever before, trying to include all of the features that the product owner had promised in each timeboxed iteration. It seems that the more agile they get, the more work they have to do, and the more nights and weekends they spend away from their families. This can’t be an improvement! Is there something they can do to fix this, before the team burns out for good?

Principle #7: Working Software Is the Primary Measure of Progress.

A good team works to make sure that everyone—team members, managers, stakeholders, and customers—really understands where the project is at any time. But how do you actually communicate the status of a project? It’s a much more difficult problem than it might seem at first.

A typical command-and-control project manager tries to keep the project on course and keeps everyone up to date with extensive schedules and status reports. But it’s hard to capture the real “essence” of a project in a status report. The report itself is an imperfect communication tool: often, three different people can read the same status report and get three totally different impressions of where the project is. And status reporting can be highly politically charged for a project manager. Almost every project manager occasionally finds himself or herself under pressure to leave something off of a status report that will make a manager or team lead look bad—and it’s always a piece of information that someone else needs in order to make a decision. So how do you report progress, if status reports aren’t good enough?

The answer lies in working software. The minute you actually see the software working in front of you, you “get” it. You can see what the software does, and what it doesn’t do. If a manager promised to deliver something that didn’t make it into the software, it might be embarrassing—but it’s impossible to not communicate that, because the software speaks for itself.

This is one reason why agile teams use iterative development. By delivering working software at the end of each iteration, and by doing a real product demonstration that shows everyone exactly what the team did, they keep everyone up to date on the progress of the software in a way that is almost impossible to misread.

Principle #8: Agile Processes Promote Sustainable Development. The Sponsors, Developers, and Users Should Be Able to Maintain a Constant Pace Indefinitely.

The ebook reader team isn’t the first project team to work insane hours in order to meet an unrealistic deadline. In fact, the hard-and-fast deadline is the primary tool in the command-and-control project manager’s toolbox. Whenever a deadline is approaching, everyone turns to nights and weekends as the first resort. An unrealistic deadline can be an underhanded tool to force more work out of a team by squeezing extra hours out of every week.

Except that in the long run, it doesn’t work. It’s well known that a team can crunch for a few weeks and get more work done, but after that their productivity typically falls off a cliff. This makes sense: people get tired and demotivated, and fatigue starts to set in. All of those errands and life events they pushed off have to force their way back in somehow. In fact, teams that work extreme amounts of overtime for an extended period actually deliver less work than teams that work normal, sane hours—and it’s usually of lower quality.

This is why agile teams believe in maintaining a sustainable pace. They should plan on delivering work that can actually be done in the time they reserve for it. Iterative development makes this more realistic, because it’s a lot easier to estimate how much software they can deliver in the next two, four, or six weeks than it is to estimate how much they can deliver in the next year and a half. By promising to deliver only what they can build, the team creates an environment where late nights and weekends are rare.⁵

Principle #9: Continuous Attention to Technical Excellence and Good Design Enhances Agility.

Poor estimates aren’t the only cause of late nights and weekends. Most developers recognize that sinking, pit-of-the-stomach feeling that comes with the realization that a seemingly simple bit of coding turned out to be a nightmare to design. There go the next three weekends, which will now be spent tracking down bugs and patching up code. 

It’s a lot faster in the long run to avoid a bug now than it is to fix it later. And it’s a lot easier to maintain code that’s well designed, because well-designed code is built in a way that’s easy to extend.

The last two decades or so have brought us a revolution in software design. Object-oriented design and analysis, design patterns, decoupled and service-oriented architecture, and other innovations have given developers patterns and tools to bring technical excellence to every project.

But this doesn’t mean that agile teams spend a lot of time creating large-scale designs at the start of every software project. Agile developers build up great coding habits that help them create well-designed code. They’re constantly on the lookout for design and code problems, and take the time to fix those problems as soon as they’re discovered. By taking a little extra time during the project to write solid code and fix problems today, they create a codebase that’s easy to maintain tomorrow.

A Better Working Environment for the Ebook Reader Project Team

Your ebook team—and their spouses—would definitely have appreciated a sustainable pace. But the project itself would also have come out better. The team was doomed to late nights and weekends from the very first day of the project, because they simply didn’t have the tools to create a realistic plan that would remain accurate a year and a half later.

What’s worse, because the team laid out software design and architecture that was built to support a very detailed specification at the beginning of the project, they ended up with very complex code that was difficult to extend. This caused many of the little changes along the way to require large patches that left spaghetti code throughout the codebase. Had the team followed an iterative approach and delivered working software along the way, they could have planned each iteration so that they could keep a sustainable pace. A simpler, more just-in-time approach to the architecture could have allowed a more flexible and extensible design. Had they used better design, architecture, and coding practices, they would have ended up with code that’s easier to maintain and extend.

Let’s imagine that our ereader team has applied these new principles, and now they’re a well-oiled software-producing machine. They’re iterating and delivering working software regularly, and they’re constantly adjusting to make sure they’re always building the most valuable software they can. They’re communicating well, and only documenting what they need to document. They’re using great design practices, and building a maintainable codebase. And they’re doing it all without having to work overtime. Our team has gone agile!

But the storm clouds are already gathering around the next project. A new project manager just sent out an invite to a giant meeting with everyone he can find. The meeting participants start to accept the invites, rooms get booked, and arguments about all of the requirements that need to be documented start flying around…and everyone on our newly minted agile team starts to get that “pit of the stomach” feeling.

They know what’s coming. The first of the many specs, plans, and Gantt charts are just starting to circulate. How can they make sure the next project doesn’t hit those same pitfalls they worked so hard to climb out of?

Principle #10: Simplicity—the Art of Maximizing the Amount of Work Not Done—Is Essential.

Adding code to an existing project often makes it more complex, especially when you then add even more code that depends on it. Dependencies between systems, objects, services, etc., make code more complex and difficult to change: the dependencies increase the likelihood of one change cascading out to another part of the system, which then requires changes to a third part of the system, creating a domino effect of increasing complexity for every change. Using iteration and building the minimal documentation at the start of the project helps your team avoid delivering unnecessary software.

However, a lot of developers feel a little uncomfortable when they first hear phrases like “use iterative development,” and “do the minimal amount of planning needed to start the project.” They feel like it’s too early to start the code for a project until they’ve made a lot of the design and architecture decisions and written them down—otherwise the team will write code today that they’ll need remove tomorrow when the design changes.

This is an understandable reaction, because for many projects outside the realm of software development, this wouldn’t make sense. It’s not uncommon for a programmer who’s new to agile to raise an objection like this: “If I’m having a contractor renovate my house, I want to see the complete blueprints first. I don’t want him to have a quick discussion with me, and then start knocking down my walls.”

This argument makes sense for home renovation. The reason is that when you’re making a change to a house, the most destructive thing you can do is pick up a sledgehammer and break down a wall. But software projects are different from houses or other physical objects. Deleting code is not particularly destructive, because you can usually just recover it from the version control system. The most destructive thing you can do to your project is to build new code, and then build more code that depends on it, and then still more code that depends on that, leading to that painfully familiar domino effect of cascading changes…and eventually leaving you with an unmaintainable mess of spaghetti code.

Maximizing the amount of work not done means avoiding this mess—and the best way to do this is by building systems that don’t have a lot of dependencies and unnecessary code. The most effective way to do this is to work with your customers and stakeholders to only build the most useful and valuable software. If a feature is not valuable, it’s often cheaper for the company in the long run to not build it at all, because the cost of maintaining the additional code is actually higher than the value it delivers to the company. When teams write code, they can keep their software designs simple by creating software designs based on small, self-contained units (like classes, modules, services, etc.) that do only one thing; this helps avoid the domino effect.⁶

Principle #11: The Best Architectures, Requirements, and Designs Emerge from Self-Organizing Teams.

Overly complex designs are common in teams that do too much up-front planning. This isn’t surprising when you think about it. Flip back to the picture of the waterfall process in Chapter 2. There’s an entire phase dedicated to requirements, and another phase dedicated to design and architecture. Doing the best job possible while you’re in the design and architecture phase must mean building the most awesome architecture the team can come up with. For a team that works this way, having few requirements and a simple design intuitively feels to them like they’re short-changing the project. Is it any wonder that they come back with a large requirements document and a complex design? Of course not, because that’s exactly what they’re explicitly being asked to do, by having entire phases of their process devoted to those things.

A self-organizing team, on the other hand, does not have an explicit requirements or design phase. When a team is self-organizing, it means that they work together to plan the project (instead of relying on a single person who “owns” the plan), and they continually come together as a team to revise that plan. A team that works like this typically breaks the project down into user stories or other small chunks, and starts working on the ones that deliver the most value to the company first. Only then do they start thinking about detailed requirements, design, and architecture.

This makes the traditional software architect’s job more difficult, but also more satisfying. Traditionally, a software architect sits in an office with a closed door and thinks abstractly about the problems that need to be solved. Obviously, many architects are not like this, but it’s definitely not atypical for an architect to be somewhat disconnected from the day-to-day work of the team.

On an agile team, everyone shares responsibility for the architecture. A senior software architect or designer still has an important role to play—but it is no longer one where he or she can work in isolation. In fact, when a team is building the software piece by piece, starting with the most valuable chunks, the architect’s job becomes more challenging (but often more interesting). Instead of creating one big design at the beginning of the project that covers all of the requirements, agile architects use incremental design, which involves techniques that allow them to design a system that is not just complete, but also easy for the team to modify as the project changes. 

Principle #12: At Regular Intervals, the Team Reflects on How to Become More Effective, Then Tunes and Adjusts Its Behavior Accordingly.

A team isn’t agile until they’re constantly improving the way they build software. Agile teams constantly inspect and adapt—they look at how their projects have run, and they use that knowledge to improve in the future. And they don’t just do this at the end of the project; when they meet every day, they look for ways to change, and they’ll change the work that they’re currently doing if it makes sense.⁷ You need to be comfortable being brutally honest with yourself and your teammates about what’s working and what isn’t. This is especially important for teams that are just beginning down the road of agile development. The only way to become more capable as a team is to constantly look back at what you’ve done so far, to assess how well you’re working as a team, and to come up with a plan for getting better.

This makes sense, and almost everyone agrees that it’s a good thing to do. But looking back and taking stock of what went right and wrong is one of those things that many teams genuinely intend to do, but rarely actually get around to doing. One reason is that it’s uncomfortable at first. It often involves looking at specific problems and mistakes, and very few people are comfortable publicly pointing out the mistakes that the other people on their team made. Over time, people on the teams get more and more comfortable talking about these things. Eventually, everyone starts to see this as more constructive than critical.

Another reason that teams don’t do this reflection is that they never set aside the time to do it—or even if they set aside the time, getting a head start on the next project seems more important than thinking about the one that’s done. Teams that start each project by reserving time both at the end of each iteration and at the end of the project to meet, review, assess, and come up with a plan for improvement are more likely to actually meet and talk about how they did. This helps them learn from their experiences and become more effective.