Optimizing Cloud Native Java, 2nd Edition

Chapter 2. Performance Testing Methodology

Performance testing is undertaken for a variety of reasons. In this chapter, we will introduce the different types of performance test that a team may wish to execute and discuss some best practices for each subtype of testing.

Later in the chapter, we will discuss statistics—and some very important human factors—that are often neglected when considering performance problems.

Types of Performance Tests

Performance tests are frequently conducted for the wrong reasons, or conducted badly. The reasons for this vary widely but are often rooted in a failure to understand the nature of performance analysis and a belief that “doing something is better than doing nothing.” As we will see several times throughout the book, this belief is often a dangerous half-truth at best.

One of the more common mistakes is to speak generally of “performance testing” without engaging with the specifics. In fact, there are many different types of large-scale performance tests that can be conducted on a system.

Note

Good performance tests are quantitative. They ask questions that produce a numeric answer that can be handled as an experimental output and subjected to statistical analysis.

The types of performance tests we will discuss in this book usually have independent (but somewhat overlapping) goals. It is, therefore, important to understand the quantitative questions you are trying to answer before deciding what type of testing should be carried out.