This week, we have the honor of hosting a presentation from Gunel Jahangirova, a PhD student in a joint program between Fondazione Bruno Kessler and University College London.
Gunel will give a talk about the representative tests based on their execution. You can find more information about the seminar below.
We will have the seminar at 5pm in MPEB 1.02.
Title:
“Behavioral Execution Comparison: Are Tests Representative of Field Behavior?” by Qianqian Wang, Yuriy Brun, Alessandro Orso
Abstract:
Software testing is the most widely used approach for assessing and improving software quality, but it is inherently incomplete and may not be representative of how the software is used in the field. This paper addresses the questions of to what extent tests represent how real users use software, and how to measure behavioral differences between test and field executions. The authors study four real-world systems, one used by end-users and three used by other (client) software, and compare test suites written by the systems’ developers to field executions using four models of behavior: statement coverage, method coverage, mutation score, and a temporal-invariant-based model they have developed. We find that developer-written test suites fail to accurately represent field executions: the tests, on average, miss 6.2% of the statements and 7.7% of the methods exercised in the field; the behavior exercised only in the field kills an extra 8.6% of the mutants; finally, the tests miss 52.6% of the behavioral invariants that occur in the field. In addition, augmenting the in-house test suites with automatically-generated tests by a tool targeting high code coverage only marginally improves the tests’ behavioral representativeness. These differences between field and test executions—and in particular the finer-grained and more sophisticated ones that we measured using our invariant-based model—can provide insight for developers and suggest a better method for measuring test suite quality.
URL:
https://people.cs.umass.edu/~brun/pubs/pubs/Wang17icst.pdf
Bio:
PhD student in a joint program between Fondazione Bruno Kessler and University College London.