Traditional testing approaches focus on evaluating if our production code is doing what it's supposed to do. However, we could make mistakes in the testing code introducing incomplete or useless assertions that create a false sense of confidence. This is where mutation testing comes in; it assesses the quality of the tests themselves, finding corner cases that are hard to realize. Our teams have used Pitest for a while now, and we recommend its use in Java projects to measure the health of the test suite. In short, mutation testing introduces changes in the production code and executes the same tests a second time; if the tests are still green it means that the tests are not good and need to improve. When you’re using programming languages other than Java Stryker is a good choice in this space.
Pitest is a test coverage analysis tool for Java that uses a mutation-testing technique. Traditional test coverage analysis tends to measure the number of lines that are executed by your tests. It is therefore only able to identify code that is definitely not tested. Mutation testing, on the other hand, tries to test the quality of those lines that are executed by your test code and yet might contain general errors. Several problems can be spotted this way, helping the team to measure and grow a healthy test suite. Most of such tools tend to be slow and difficult to use, but Pitest has proven to have better performance, is easy to set up, and is actively supported.
