Injecting faults to achieve comprehensive testing

Project: To achieve an extremely high level of software quality, the U.S. Federal Aviation Administration (FAA) requires the testing of thousands of modules created by different development teams.

Problem: To fully test an application, it must be tested under all possible conditions, including error conditions. But simulating errors like a disk offline or a disk broken can be difficult. The typical approach is to modify the source code temporarily before testing. However, this is so labor-intensive it is most often only done once. For this project, the FAA wanted a higher level of quality assurance, and asked the contractor to find a way to test many different error conditions.

Solution: The contractor chose Aprobe technology, since it can "spoof" a system to believe that any error has already occurred. Sometimes this involved using Aprobe to stub out the execution of specified methods and to return an error code instead. In other cases, the contractor used Aprobe to throw an exception. No application changes were necessary for these error tests. The fault injection process was so straightforward that it was added to the regression testing process and run on every build.

Remarks: Aprobe allowed the contractor to achieve comprehensive error testing. The quality of the final code was demonstrably improved.