Practical Development Environments

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Projects have Environments in which People produce Products.

"Pithy, but painful," was a friend's wry reply when I asked his opinion of the above epigraph as a summary for this book. At a very high level, it really does describe the creation of software. The key word is environments, which in this book refers to all the tools used to produce software and how they are used, but not the details of how the software is written, or even what language the software is written in. For this book, an environment includes the software configuration management (SCM) tools, build tools, testing and bug tracking tools, and the release and maintenance processes. All these areas are covered in subsequent chapters of this book.

This chapter begins with an overview of how a software product is typically created. If you are involved in this process from day to day, you will probably be familiar with most of the steps. The amount of work in each step is different for different projects, groups of people, or companies, but all the steps appear in some guise. This chapter also briefly describes the difference between open and closed software.

The chapter then continues with some uncomfortable software development mistakes that most developers have stumbled across; avoiding these costly mistakes justifies reading the rest of this book.

shows the different activities that are generally involved in creating a software product, though it's only a loose description of what really happens when developing software. The activities probably don't occur in the same order in your experience, and each one is often repeated many times as a product is developed and maintained. You can probably also add some other arrows from experience with different projects that you've worked on. Some projects may not even have had all their activities connected, which is one of the things that a good development environment can help you with.

Figure 1-1: Typical activities involved in creating software products

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

shows the different activities that are generally involved in creating a software product, though it's only a loose description of what really happens when developing software. The activities probably don't occur in the same order in your experience, and each one is often repeated many times as a product is developed and maintained. You can probably also add some other arrows from experience with different projects that you've worked on. Some projects may not even have had all their activities connected, which is one of the things that a good development environment can help you with.

Figure 1-1: Typical activities involved in creating software products

Large or small, formal or more relaxed, open or closed (see , later in this chapter), all software projects have activities that resemble those shown in . A short description of each activity follows:

Product marketing: This goes by many names, but whatever it is called, it always involves creating the ideas for the product. A subtle blend of technical knowledge about what has been done and what might be possible is combined with some imaginative flair for what might actually be wanted by potential customers. This is different from general marketing, which is described below.
Gathering requirements: The process of collecting information to clearly communicate to everyone in the project exactly what the product is supposed to do. Some different ways that this information can be described include documents, prototypes, and examples of how the product would be used.
Writing functional specifications: Functional specifications describe how the product can be created to fulfill the requirements.
Implementation: The gritty edge, where the product is actually created. A surprisingly small amount of the total time to develop a product is usually spent here.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Traditionally, software development is closed. This means that whether or not users are charged for the actual product, the source files that are used to create the product are confidential. Sometimes you can purchase the source, but that's extra.

Open software development refers to a number of different ideas about software development. The best known one is that the source files for the product are openly and freely available, so customers can change the source files and then rebuild the product themselves, at least in theory. The source files may also be free, as in costing nothing. Many open source projects also allow anyone to redistribute the source files, subject to the requirement that if you make changes, you must make those changes publicly available. If you continue to distribute the changes you make to a product, then you may have produced a fork of the project, and the product will have become two different products. Forks are an accepted part of open software development.

Examples of some well-known open source software products are shown in .

Table 1-1: Major open software projects
Software	URL	Description
Apache	http://www.apache.org	Most of the world's web sites run on the Apache web server.
GCC	http://gcc.gnu.org	The most widely used compiler for C and other languages.
GNU/Linux	http://www.linux.org	The best-known open source Unix-based operating system.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Software projects can fail for many reasons, and one of them is a bad development environment. Here is an informal list of some frustrations and embarrassing mistakes commonly made in producing software. None of these have anything to do with failures caused by things like missing specifications, specifications changing over time (the dreaded "feature creep"), poor estimates of how long a project will take, or even whether the problem being solved is a really difficult one. Each of these problems is due to the tools or processes of the environment in which the software was produced:

We can't rebuild the same product that we shipped to the customer, so we'll ship her the latest version, just to be safe.
Building from scratch (also known as a clean build) each time is often the only way to get the product to build reliably. This takes so long that the developer goes off and does something else. Consequently, every developer thinks that his build tool is too slow.
Testers, technical writers, and managers can't build the latest version of the product. Even if it has already been built for them by someone else, it's unclear where to find the latest version.
Fixes for known bugs are not released, because it's too hard to properly test the required changes and they may break other parts of the product.
Everyone finds the bug tracking tool awkward to use.
The environment used by developers and the one in which the product is used by customers are different in some important ways, so the developers never actually see problems the same way that a customer does.
Getting a product ready to release takes so long that any late-arriving changes don't get tested or documented.
The wrong set of bits gets shipped to the customer. This one should make anyone involved in developing software wince, but it does happen.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

What makes something practical? The title of this book strongly suggests that its contents are about practical solutions to common problems in development environments. Here is what practical means for the tools used in development environments:

Available (whether open or closed source)
Appropriately priced (for open software, this often means no cost)
Usable—installation and configuration to match the local development process is possible, the tool doesn't crash regularly, there aren't too many bugs, and documentation is adequate
Can eventually produce software for the platforms used by the customer

On the other hand, impractical tools for development environments are those that are:

Dead—no longer sold or supported
Too expensive to even consider spending any time evaluating
Tortuous to get working or configured as needed, or impossible to maintain and upgrade

Abstract frameworks, conceptual models, and design patterns are all useful for categorizing solutions to problems encountered in development environments, but the aim of this book is to provide ideas that are more practical. The basic areas of projects that are summarized in the next chapter are ideas that are applicable to all software projects, whatever the product's purpose is and whoever the customers are.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

To make the ideas in this book seem more personal, try answering the following questions (honestly):

Name the biggest mistake that you, personally, made in each of your last three projects. Program design doesn't count; tools and processes do.
Name the biggest mistake you think someone else made in your last three projects.
Imagine that you could totally replace just one software tool that you use daily. Which one would it be, and why?
Which tool is so good that you would have to be seriously bribed to be persuaded to stop using it?
Which part of producing software just seems to take much longer than it should? Do you feel that you really understand what goes on in that part?

Your answers should give you some good ideas about which parts of this book will particularly interest you. It was questions like these, and a lack of written guidance on addressing the issues raised by their answers, that made me want to write this book in the first place.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This chapter briefly describes the different parts of a project and then introduces the main activities invoved in software development and the corresponding tools that make up a development environment. The activities are software configuration management (SCM), building software, testing software, tracking bugs, writing documentation, releasing products, and maintenance. This chapter ends with some personal recommendations of tools for three different types of development environments.

Whether you are starting a project from scratch or looking to improve an existing development environment, my opinion is that you should consider the different parts of the environment in the same order used in this chapter and in this book. That is, SCM is the most important part of an environment; next in importance are the build tools, then testing and bug tracking, and so on. This is because your choice of SCM tool is likely to have the largest impact on your environment. This is not to say that any of the parts are unimportant, just that improving how your SCM tool and build tool are used will probably improve your environment more than improving the bug tracking tool or release process. Similarly, if you are creating a new project, an SCM tool should be chosen before a bug tracking tool.

shows the different parts of a single project. This project has source files that become products, and those products have customers who use the products. The project has project members (developers, testers, technical writers, toolsmiths, managers, and product marketing staff), who are also customers for their own products. The development environment for this project is made up of both the tools and the local processes and policies for using those tools.

Figure 2-1: Different parts of a project

If there are many projects within an organization, then each project will have some areas that overlap and some that don't. For instance, two projects may have different policies about how to use the same bug tracking tool. The processes and tools are chosen to work with the source files but need to be able to operate even when the same source files are used by different projects with different policies.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

shows the different parts of a single project. This project has source files that become products, and those products have customers who use the products. The project has project members (developers, testers, technical writers, toolsmiths, managers, and product marketing staff), who are also customers for their own products. The development environment for this project is made up of both the tools and the local processes and policies for using those tools.

Figure 2-1: Different parts of a project

If there are many projects within an organization, then each project will have some areas that overlap and some that don't. For instance, two projects may have different policies about how to use the same bug tracking tool. The processes and tools are chosen to work with the source files but need to be able to operate even when the same source files are used by different projects with different policies.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

SCM, the subject of , is the ability to keep track of different versions of the source files that are used to create software products. Even when just one person is changing these source files, it's very useful to be able to see how a file evolved over time or even to undo some changes that in hindsight you regret making. As projects grow in size and complexity, effective SCM tools become vital. To put it another way, projects that don't use an SCM tool won't grow successfully.

SCM tools also provide a good way to share your work with other people in a controlled manner. Rather than just using a common location such as a directory to exchange files with other people, with an SCM tool you can make sure that interdependent files are changed together and you can control who is allowed to make changes. SCM tools also allow you to save messages about what changed in the source files, and also why the files were changed, which can be used to work out which releases a particular bug was fixed in.

Many SCM tools provide ways to support one or more existing releases of a product, while still allowing the team to develop the next release using different versions of the same source files. For instance, after a product is released, all the source files for that release can be marked (or even branched) to allow future bug fixes for that release, while the next release is developed independently.

describes all of the above in more detail and examines seven of the most commonly used or promising SCM tools: CVS, Subversion, Arch, Perforce, BitKeeper, ClearCase, and Visual SourceSafe.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Software products are built from their source files, also known as source code, which is often the collection of files stored in an SCM tool. A build tool uses the source files and follows some specified build rules to run other tools such as compilers to create the product from the source files. These build rules are usually specified in configuration files known as build files, which are part of the source files for a product.

Build tools have to be aware of which products can be built from a given set of source files and other files such as libraries. They also have to know which parts of the product depend on which other parts, so that if one source file is changed by a developer, then all the other affected parts will also be rebuilt. Build tools should be able to execute the correct commands for building the same product on different platforms, while hiding the idiosyncrasies of each platform from the product's developers as much as possible. Build tools are also used to generate executables that run on platforms other than the platform that the build tool is actually run on; this is known as cross-compiling.

describes what to look for in a build tool and examines some commonly used build tools such as make, Ant, Jam, and SCons in more detail.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Testing a product spans the range from individual developers writing unit tests that check small parts of an application, to system tests that use the whole application, to customers giving feedback about how they really want the product to work. focuses on testing environments for unit tests and system tests.

One of the cornerstones of the XP (extreme programming) methodology is the importance of extensive unit tests, written even before the functionality that they test has been written. No matter what your methodology or style of programming is, a healthy test suite is good for reassuring yourself that your latest changes haven't broken some other distant part of the product.

Of course, running all those different tests and interpreting their results quickly becomes tedious; if testing is not automated, it is often postponed and then finally abandoned. So it's important when developing and maintaining a product to have a test environment where tests can be added easily and run easily, and where the test results can be clearly understood.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Testing a product provides information about which parts of it are working and which are not. This information needs to be made available to developers, other testers, managers, the people that decide when a product is ready for release, and also to those who support a product. Bug tracking tools are commonly used to do this. Bugs are sometimes referred to as issues, because they are often requests for changes or some other category; some other terms, particularly defect and incident, can have legal implications and are best avoided if possible. The term bug is used colloquially throughout this book to refer to all these categories.

Bug tracking tools often store information about bugs in a database and then provide convenient GUIs and command-line interfaces (CLIs) for adding information about bugs to the database, changing the information recorded about bugs, and creating reports about different kinds of bugs. At a minimum, a recorded bug has a description of the bug (what happened, what should have happened) and an identifier that is unique for each different bug. Other information that is frequently recorded with each bug includes who found it, the steps to reproduce it, who is working on it now, which releases the bug exists in, and which releases it has been fixed in.

Many bug tracking systems define a number of states and allow each bug to be in just one state at a time. This is intended to help guide the workflow of the team when they are working on different bugs. For instance, a new bug may be in the Open state, then move to an Accepted state, then to a Fixed state, and then to a Closed state. This is part of applying a change management process (see ) to how you want bugs to be fixed.

Tracking bugs is a lot more than just not forgetting what still needs fixing in a product. For good and bad, it becomes a way to measure development and testing progress toward the next version. Observing the numbers of bugs in different states over time can play a part in deciding when a product is stable enough to ship. It is also a rather simplistic way to measure how busy individuals are—for instance, by the number of bugs they have assigned to them. As such, bug totals often take on a meaning far beyond a simple record of the problems in a project. Bug tracking tools can even become a way to avoid communicating directly with other project members, as described in .

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

An ideal product is so transparent that it needs (almost) no documentation. Software products that are as well-designed and well-implemented as this Platonic ideal are rare indeed, so documentation is an expected part of all products. Whether the documentation is a simple README file, a large manual that ships with the product, or interactive help available when the product is used, the contents still have to be written by somebody, usually a technical writer.

takes the viewpoint that the documentation is part of the product and that there are plenty of similarities between writing software and writing documentation. For instance, large documents are made up of smaller ones; different parts of a document depend on other parts of the document; and both source code and natural-language documents can benefit from tools, whether they are compiler warning flags or spellcheckers.

Another similarity between software and documentation is that both are transformed from one file format into other file formats as part of being released. A source file is compiled to an executable. A document is often written in one source file format (such as Microsoft Word or FrameMaker) and then converted to another release format (such as HTML or PDF, Adobe's Portable Document Format) for use by the customer. Some commonly used documentation formats are discussed in (HTML, PostScript, and PDF) and (TEX, Texinfo, troff, and POD).

A different aspect of documentation is when it is intended for use by a project itself. An example of this is when the APIs (application programming interfaces) of different parts of a product are documented to help other developers use them. There are a number of tools, such as Javadoc, to help with this, and these are also examined in .

also examines in more detail a variety of file formats and their related documentation environments, including raw text, FrameMaker, XML in DocBook and OpenOffice, and Microsoft Word.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Once a product has been developed, tested, and documented, it eventually becomes ready to release. You want to release the software according to some predetermined plan. Too often, however, products just escape into the hands of customers because no one considered issues such as release numbers and license keys before releasing the product. describes all of this.

With the increase in malicious software, assuring customers that the files they download are actually the same as the ones that you released is becoming an essential part of releasing software products. The use of digital signatures and checksums to help with this is discussed further in .

Since installing a product is often a customer's first experience of the software, and first impressions count, the installation process is important. also examines the most common packaging formats and installation tools for a variety of platforms, as well as some common irritations with installation tools and the installers that they produce.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Maintenance of a product after it has been released takes up a large part of a product's life span. describes some typical product maintenance activities and how the tools in a development environment can help you with them.

How to maintain a development environment is also discussed, including what kinds of things stop working as an environment ages and how to know when to throw tools and files away.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This section contains my personal recommendations for tools for development environments. The recommendations are intended for projects with less than 1 million lines of source code and under 200 people involved in developing, testing, documenting, and releasing the product. The annual budget for tools probably ranges from zero to $100,000. These choices are purely personal ones made from the tools available in 2005, with no undue influences from any individual companies or projects.

If you use a tool that you feel is much better than one of the tools I've recommended, feel free to send me email about it via bookquestions@oreilly.com. My own contact details are available at http://www.pobox.com/~doar.

IDE recommendations are also welcome, but rants about editors (the programs, not the people) are generally unproductive—use one that does the job for you, and learn it well.

If these recommendations are enough for you to make progress with a development environment, that's great! Reading the sections about each tool later in the book is still a good idea to get some more background, especially .

However, a development environment is more than just its tools. The discussions of the best practices and annoyances of each area in the chapters that follow will help you use each of these tools in a more productive manner.

This list of tools is for environments that can afford the effort of using tools that are still themselves being developed. Some reading of mailing lists and weblogs, and possibly some local development of the tools by a toolsmith, may be necessary.

SCM tool: Subversion () with FishEye (http://www.cenqua.com/fisheye)
Build tool: Ant () for projects using Java©; SCons () for most projects and other languages

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This chapter describes ideas that are part of a development environment but are not specific to any particular kind of tool. Examples of such ideas are the following:

Preconstructed environments, where all the tools for a development environment are provided "out of the box" for you
Integration of tools to make them work better together
Automation of the different parts of software creation
Naming schemes
Internationalization of products and tools for use in other countries

Preconstructed development environments (PDEs) are what I have chosen to call software development environments that are provided as a single set of software applications. (Whether or not they are practical development environments, as in the title of this book, is a separate issue for each PDE.) Another name sometimes used for them is collaborative development environments, but it seems to me that development environments are always collaborative, and what sets these environments apart is that they come ready-made, prefabricated—in short, preconstructed.

If you are setting up a development environment and one of these environments works for you, maybe you won't have to choose every tool to use in your environment. Still, the other chapters in this book contain much more than just which tool to choose, so read on.

One well-known example of a PDE is SourceForge (http://www.sourceforge.net), which uses the term "software development website" to describe itself. A typical PDE provides an SCM tool, a bug tracking tool, a project home page, and somewhere to download released files from. Some PDEs also provide online discussion forums, weblogs, and "farms" of different machines for developing ports of the project to multiple platforms and for running tests. Some even provide databases ready for use by the different projects. Still others have usage statistics for each project and places to post requests for help or jobs. The various parts of the PDE should all work together and, as much as possible, already be configured to make hosting your project quick and easy.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Preconstructed development environments (PDEs) are what I have chosen to call software development environments that are provided as a single set of software applications. (Whether or not they are practical development environments, as in the title of this book, is a separate issue for each PDE.) Another name sometimes used for them is collaborative development environments, but it seems to me that development environments are always collaborative, and what sets these environments apart is that they come ready-made, prefabricated—in short, preconstructed.

If you are setting up a development environment and one of these environments works for you, maybe you won't have to choose every tool to use in your environment. Still, the other chapters in this book contain much more than just which tool to choose, so read on.

One well-known example of a PDE is SourceForge (http://www.sourceforge.net), which uses the term "software development website" to describe itself. A typical PDE provides an SCM tool, a bug tracking tool, a project home page, and somewhere to download released files from. Some PDEs also provide online discussion forums, weblogs, and "farms" of different machines for developing ports of the project to multiple platforms and for running tests. Some even provide databases ready for use by the different projects. Still others have usage statistics for each project and places to post requests for help or jobs. The various parts of the PDE should all work together and, as much as possible, already be configured to make hosting your project quick and easy.

PDEs are rather like preconstructed houses. They are quick to set up and start living in, but they can be harder to change later on. One benefit is that once you have used a particular PDE for one project, it becomes easy to find your way around other projects that are using the same PDE. Once you have found the statistics for one SourceForge project, you know where to look for the statistics for every other SourceForge project.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Regardless of which tools are used in a development environment, integration of all the different kinds of tools is part of what creates an environment that is satisfying to work in. Manually copying information from one tool in order to do something with another tool is an inefficient use of time and is also prone to error. Tools that work well with each other are one mark of a good development environment.

There is another side to the idea of integration, though. Just as with most machines and hardware, the connections in software are often the most fragile part of a system. Upgrading tools can break integration schemes, and unexpected inputs from one tool can break another tool in unusual and hard-to-debug ways.

There is also the problem of how to handle one tool being unavailable for a period of time. For instance, if your SCM tool usually updates your bugs with information about the file changes related to each bug, what should the SCM tool do when the bug tracking tool is unavailable? Stopping work for the whole environment means that any one tool in an environment becomes a single point of failure. However, simply ignoring the bug tracking system means that the information there becomes "best effort" and cannot be relied on to be complete. A reliable distributed environment is a hard thing to achieve. The key question to consider when integrating tools is what the developers most want to know. Effort is often spent on uncommon scenarios just as an intellectual challenge, to see if something could be done.

Two of the simplest ways of integrating existing tools in development environments are through URLs and email. A web server can be a useful place for gathering project information from multiple tools.

Imagine that you want to make emails about changes to source files that are sent from the SCM tool more helpful than just a list of the names of the changed files. You could have the SCM tool insert the details of the changes in the email or send them as an attached file. But that's a lot of text that may hide other information. Adding URLs into the email that in turn point to a web server with the files' differences makes the changes easily available from most mail clients. Another advantage of URLs is that they are relatively easy to generate, since they are just short text strings. The mail doesn't even have to be sent as HTML for this to work with some mail clients.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

There is no separate chapter on automation in this book, because I believe that automation is vital in all aspects of a development environment. If you do something nontrivial twice, then at least document the steps that you followed. If you have to do it again, consider automating it. If a task has two or more steps and is manual, then someday someone will either perform the two steps out of order or forget one of them.

Automation by reducing the repeated parts of a task to a single command is one good idea. Improving tools and other parts of your environment will relieve the short-term points of pain, but automation will raise the whole level of productivity of your project one notch.

While it is easiest to automate an environment when it is first created, automation can be gradually introduced to an existing environment. First choose an automation environment (see the next section, ) and then begin by automating how source code is obtained from your SCM tool. Then automate running a build using that source code. Then work out how to run unit tests and capture their results automatically. Finally, make sure you can automatically update bugs with details of the build, create change logs and a list of bugs fixed in each build, tag the source files, and deploy the results of the build and tests to a suitable location for the rest of the project members to use. Once all of these steps have been automated, you won't have to do any of this manually. Each of these steps is discussed in the relevant chapters. The point is that you can add automation one piece at a time.

However, automation does have some consequences:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

An automation environment is a tool that performs a variety of automated tasks for you. An automated task is anything that has been modified so that you can run it, usually from a command line, without any manual interaction. An automated environment makes it easier to run each of these tasks in the correct order and at chosen intervals.

An automation environment is a key part of the XP concept of continuous integration (CI), which involves keeping a product integrated and working correctly all the time. For that reason, automation environments are sometimes referred to as "CI frameworks." If you find the phrase "automated environments" too unwieldy, then perhaps EFA for "environments for automation" might do.

Many different tasks are commonly controlled by automation environments. (Exactly what some of these tasks involve is explained in more detail in subsequent chapters.) Some of these tasks are:

Checking out the latest versions of source files
Calculating the appropriate build or release number
Tagging the source files
Building one or more products from the virgin source code (source code that has never been used for any kind of build; see )
Testing the products with both unit tests and system tests
Moving the generated release packages to a suitable location for other people's use
Marking certain bugs as having potential fixes available in this release
Creating change logs and release notes about what changed in this release
Notifying people when a release becomes available, and also notifying the responsible individuals when a build or test fails (see )
Publishing test reports and build logs to a web site (see )
Collecting project data and running static analyzers on the source code (see )

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

There are numerous places in a development environment where having a standard way of describing a build is useful. If you don't define a standard way, then the SCM tags for a build will be formatted one way, the build will appear in the bug tracking system in a different way, and the release names will be defined in yet a third way.

What information should be in a build label? My suggestions are:

Build type: Was this build for internal testing (QA) or release (REL)?
Version: The major, minor, and patch numbers; for example, 1_3_0. has more details about different versioning schemes.
Build number: A number that uniquely identifies each build. has more details about using build numbers.
Date: The year, month, and day when the build was started, in that order so that builds can be sorted on this field. If you plan to do more than one build per day, then the hour and minute from a standard time zone are useful too.
Special: If the build is otherwise significant, add an optional field at the end. An example is BETA for a beta release. Put this at the end of the label so that alphanumeric sorting of the label is not overly affected by it.

Each of these fields should be separated from the next with a character such as #, -, or _ so that the label can be parsed by shell scripts and other programs. If the # character is used with the above label format, then the label for internal build 129 of Version 1.3.0 of a product on July 9, 2005 would look like:

QA#1_3_0#129#2005_07_09

Another requirement for a build label is that it should not contain characters that are illegal in any of the contexts that the label will be used in—that is, SCM labels, file and directory names, field values in your bug tracking system, web pages, and documentation. Characters to avoid include

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This section contains ideas for choosing names for projects and names for computers. Planning all these things in advance may seem excessively controlling to some people, but in my experience, deciding on this information once, before things are done differently by each person, will help project members communicate clearly.

Project names are usually chosen by engineering groups, with one name for each significantly different version of the products that they are working on. There should be no need to change a project's name once it has been chosen. Product names, on the other hand, are the names that customers see, and these names are usually chosen to help a product sell or to become popular. Product names can change at the whim of a market research poll or a new VP of Sales.

Some general guidelines for choosing names for projects are:

Keep it short: Since project names may appear in filenames or source code, shorter project names are preferable; four to six characters is common. Longer names will only be abbreviated anyway, and usually in two different ways.
Use distinctive sounds: Project names should sound different from each other when spoken aloud by people whose native language is not the one used by the rest of the group. Even if everyone speaks English, having two projects named "ctest" and "seebest" is too close for comfort.
Use low-frequency letters: It's much easier to be confident that all references to a project name can be found if the name contains characters that are less common in the local language. This is a good argument for choosing project names that use unusual characters, such as the letters q and z for English.

A few years ago there was a project named IDS that apparently had a function named IDSConnect. Then the project was renamed DIS and all its functions were renamed accordingly, which led to their function for creating connections being renamed to

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Whether you are choosing a tool for a new project or looking for a tool to replace one already in use, there are a number of helpful places to start.

First, other members of the project are likely to have suggestions for different tools. Send out a request for information, making sure it has an obvious deadline for replies. Specify what the tool must do and what you would merely like it to do. If there is a tool in current use, be very clear about what you believe the problems are with it. Be sure to ask people about whether they have actually used their recommended tool, not just evaluated it. Make sure you ask them whether they have administered the tool since using and administering tools can be very different experiences. (ClearCase is the classic example of this phenomenon.)

Other places to search for more information include:

Open Directory Project: The Open Directory Project (http://dmoz.org) claims that it's the largest human-edited directory of the Web. You won't find every tool in each category, but you'll probably find most of them.
Newsgroups: Don't use Google or some other search engine to search only web sites. Search the newsgroups as well—for example, by using the Google Groups link. Company web sites' search forms may also turn up information that is not available from other search engines.
Discussion web sites: Slashdot (http://slashdot.org) and other opinionated forums are sure to reflect a diversity of thoughts about the tools. Some of these opinions may be based on experience, but many are surely not. Caveat lector, or "let the reader beware."

A number of web sites, magazines, books, and other resources for choosing tools are listed in .

When you have decided that a tool must be changed and you have some ideas about what to replace it with, there are a number of steps to follow:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Two other aspects of both environments and the products that are created with them are internationalization (i18n) and localization (L10n). i18n is designing and implementing a product so that it can be localized, and it is notoriously difficult to do so after a product has grown. L10n is the work of creating a version of the product for a particular locale. Among other changes, L10n can involve translations; changes in the sort order for strings; different keyboard shortcuts; changes to date, number, and currency formats; and changes to the layout of UIs.

i18n and L10n of products and their installers is widespread. English is just another language; it will be the first one localized for only if it is the common language of the project. However, fully localized software development environments still seem quite uncommon outside Japan and France. This would require that the operating system and all other tools (such as the file editors, compilers, build tools, and bug tracking systems) use the local language and formats.

A number of non-English-speaking developers choose their tools by what the tools can do; whether the tools support their particular language is a secondary consideration. The basic free translation services that are provided by Google and Babel Fish (http://world.altavista.com) can help with the occasional message from a tool that they find confusing. To be precise, you can translate the words in the message, but messages from some tools are notoriously hard to understand even for fluent speakers of the original language of the message.

If you expect your environment to be used by people who are not comfortable with English, then the development tools should be carefully chosen to support this need. Open source tools from projects with lots of contributors and tools from large companies usually support the largest number of localized versions. If a tool discussed in this book has been internationalized, then this is mentioned. If a number of localized versions of the tool are available for the current release, then this is also noted.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

AAA is a convenient acronym for authentication (proving who someone is), authorization (deciding who is permitted to do what), and accounting (keeping a record of what happened). AAA is important for all of the tools in this book, but perhaps SCM tools and bug tracking systems depend on it the most. Effective AAA depends on a good understanding of security issues and often uses cryptographic techniques and tools to enforce the chosen policies.

For some environments, strong AAA is a crucial requirement. In the same way that laboratory notebooks can be used as evidence, the output from some SCM tools has become evidence in cases such as the SCO debacle and other corporate patent wranglings. Accounting data for the U.S. Sarbanes-Oxley Act may be related to the information stored by an environment's tools. The FAA (Federal Aviation Authority) requires storage of the software used in its systems for at least 25 years.

Even the smallest companies and projects want to be confident that no one has added a "back door" to their product, either to get around paying for licenses or for other, more malicious purposes, such as spreading computer viruses and other malware. To avoid this, changes to a project should be made only by the people who have been both authenticated and authorized to make them. Finally, the changes themselves need to be auditable.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|

Table of Contents