Evidence for an Information Mismatch
We compared the results from the developer and reporter surveys to find out whether they agree on what is important in bug reports.
Figure 24-2. Results from the survey among reporters (215 consistent responses by Apache, Eclipse, and Mozilla reporters)
First, we compared the information developers use to resolve bugs (question D1) and the information reporters provide (R1). In Figure 24-3, items in the left column are sorted decreasingly by the percentage of developers who have used them, and items in the right column are sorted decreasingly by the percentage of reporters who have provided them. Lines connect the same items across columns and indicate the agreement (or disagreement) between developers and reporters on that particular item. Figure 24-3 shows that the results match only for the top three items and the last one. In between there are many disagreements, and the most notable ones are for stack traces, test cases, code examples, product, and operating system. Overall, the Spearman correlation[33] between what developers use and what reporters provide was 0.321, far from ideal.
Figure 24-3. Used by developers versus provided by reporters
Next, we checked whether reporters provide the information that is most important for developers. In
Get Making Software now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
