EvoSuite at the SBST 2016 Tool Competition

EvoSuite is a search-based tool that automatically generates unit tests for Java code. This paper summarizes the results and experiences of EvoSuite's participation at the fourth unit testing competition at SBST 2016, where Evo-Suite achieved the highest overall score

Unit Test Generation During Software Development: EvoSuite Plugins for Maven, IntelliJ and Jenkins

Different techniques to automatically generate unit tests for object oriented classes have been proposed, but how to integrate these tools into the daily activities of software development is a little investigated question. In this paper, we report on our experience in supporting industrial partners in introducing the EVOSUITE automated JUnit test generation tool in their software development processes. The first step consisted of providing a plugin to the Apache Maven build infrastructure. The move from a research-oriented point-and-click tool to an automated step of the build process has implications on how developers interact with the tool and generated tests, and therefore, we produced a plugin for the popular IntelliJ Integrated Development Environment (IDE). As build automation is a core component of Continuous Integration (CI), we provide a further plugin to the Jenkins CI system, which allows developers to monitor the results of EVOSUITE and integrate generated tests in their source tree. In this paper, we discuss the resulting architecture of the plugins, and the challenges arising when building such plugins. Although the plugins described are targeted for the EVOSUITE tool, they can be adapted and their architecture can be reused for other test generation tools as well

Private API Access and Functional Mocking in Automated Unit Test Generation

Not all object oriented code is easily testable: Dependency objects might be difficult or even impossible to instantiate, and object-oriented encapsulation makes testing potentially simple code difficult if it cannot easily be accessed. When this happens, then developers can resort to mock objects that simulate the complex dependencies, or circumvent object-oriented encapsulation and access private APIs directly through the use of, for example, Java reflection. Can automated unit test generation benefit from these techniques as well? In this paper we investigate this question by extending the EvoSuite unit test generation tool with the ability to directly access private APIs and to create mock objects using the popular Mockito framework. However, care needs to be taken that this does not impact the usefulness of the generated tests: For example, a test accessing a private field could later fail if that field is renamed, even if that renaming is part of a semantics-preserving refactoring. Such a failure would not be revealing a true regression bug, but is a false positive, which wastes the developer's time for investigating and fixing the test. Our experiments on the SF110 and Defects4J benchmarks confirm the anticipated improvements in terms of code coverage and bug finding, but also confirm the existence of false positives. However, by ensuring the test generator only uses mocking and reflection if there is no other way to reach some part of the code, their number remains small

Unit Test Generation During Software Development: EvoSuite Plugins for Maven, IntelliJ and Jenkins

Different techniques to automatically generate unit tests for object oriented classes have been proposed, but how to integrate these tools into the daily activities of software development is a little investigated question. In this paper, we report on our experience in supporting industrial partners in introducing the EVOSUITE automated JUnit test generation tool in their software development processes. The first step consisted of providing a plugin to the Apache Maven build infrastructure. The move from a research-oriented point-and-click tool to an automated step of the build process has implications on how developers interact with the tool and generated tests, and therefore, we produced a plugin for the popular IntelliJ Integrated Development Environment (IDE). As build automation is a core component of Continuous Integration (CI), we provide a further plugin to the Jenkins CI system, which allows developers to monitor the results of EVOSUITE and integrate generated tests in their source tree. In this paper, we discuss the resulting architecture of the plugins, and the challenges arising when building such plugins. Although the plugins described are targeted for the EVOSUITE tool, they can be adapted and their architecture can be reused for other test generation tools as well

Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study

Work on automated test generation has produced several tools capable of generating test data which achieves high structural coverage over a program. In the absence of a specification, developers are expected to manually construct or verify the test oracle for each test input. Nevertheless, it is assumed that these generated tests ease the task of testing for the developer, as testing is reduced to checking the results of tests. While this assumption has persisted for decades, there has been no conclusive evidence to date confirming it. However, the limited adoption in industry indicates this assumption may not be correct, and calls into question the practical value of test generation tools. To investigate this issue, we performed two controlled experiments comparing a total of 97 subjects split between writing tests manually and writing tests with the aid of an automated unit test generation tool, EvoSuite. We found that, on one hand, tool support leads to clear improvements in commonly applied quality metrics such as code coverage (up to 300% increase). However, on the other hand, there was no measurable improvement in the number of bugs actually found by developers. Our results not only cast some doubt on how the research community evaluates test generation tools, but also point to improvements and future work necessary before automated test generation tools will be widely adopted by practitioners

Oral health-related quality of life in partially edentulous patients before and after implant therapy: a 2-year longitudinal study

The aim of this study was to measure the Oral Health-Related Quality of Life (OHRQoL) before and after a prosthodontic implant therapy so to determine the physical and psychological impact of implant-supported fixed partial dentures (IFPD) rehabilitation among edentulous patients. Methods. 50 partially edentulous patients aged 40-70 years, treated with IFPD, completed the OHRQoL questionnaire before the implant surgery (Time 0) and 2 years after their whole implant-prosthetic rehabilitation (Time 1). The questionnaire was proposed in a short version of Oral Health Impact Profile (OHIP-14, range 0-56) and analyzed through the ‘additive method’. We evaluated statistical mean, standard deviation, median, variance and mode of all OHIP-14 domains and the statistical significance about oral changes at Time 0 and Time 1 using the Chi-square test (p-values 0.05). Patients with I and IV Kennedy’s class edentulism showed better improvement (p < 0.05). Preoperative and post-treatment assessments of OHRQoL exhibited significant differences. The IFPD treatment had a positive effect on the OHRQoL, which improved better in patients with I and IV Kennedy’s edentulous class

A detailed investigation of the effectiveness of whole test suite generation

© 2016 The Author(s)A common application of search-based software testing is to generate test cases for all goals defined by a coverage criterion (e.g., lines, branches, mutants). Rather than generating one test case at a time for each of these goals individually, whole test suite generation optimizes entire test suites towards satisfying all goals at the same time. There is evidence that the overall coverage achieved with this approach is superior to that of targeting individual coverage goals. Nevertheless, there remains some uncertainty on (a) whether the results generalize beyond branch coverage, (b) whether the whole test suite approach might be inferior to a more focused search for some particular coverage goals, and (c) whether generating whole test suites could be optimized by only targeting coverage goals not already covered. In this paper, we perform an in-depth analysis to study these questions. An empirical study on 100 Java classes using three different coverage criteria reveals that indeed there are some testing goals that are only covered by the traditional approach, although their number is only very small in comparison with those which are exclusively covered by the whole test suite approach. We find that keeping an archive of already covered goals along with the tests covering them and focusing the search on uncovered goals overcomes this small drawback on larger classes, leading to an improved overall effectiveness of whole test suite generation

Anti-Listeria activity of lactic acid bacteria in two traditional Sicilian cheeses

Listeria monocytogenes is a pathogen frequently found in dairy products, and its growth is difficult to control. Bacteriocin-like inhibitory substances (BLIS), produced by lactic acid bacteria (LAB), having proven in vitro anti-Listeria activity, could provide an innovative approach to control L. monocytogenes; however, this application needs to be evaluated in vivo. In this study, twenty LAB strains isolated from different Sicilian dairy environments were tested for control of growth of L. monocytogenes in three different experimental trials. First, raw and UHT milk were inoculated with LAB strains alone, and LAB strains mixed with L. monocytogenes. Second, mini-cheeses containing LAB and/or L. monocytogenes were produced. Third, two traditional Sicilian cheeses inoculated with a multi-strain LAB mixture combined with L. monocytogenes were produced. The addition of BLIS produced by LAB to milk and in mini-cheese production was unable to inhibit the growth of L. monocytogenes. However, an anti-Listeria effect was observed in the Pecorino Siciliano cheeses, where, after 15 days of ripening, the cheeses with added LAB had fewer L. monocytogenes compared to the control cheeses with no added LAB, while in the Vastedda della valle del Bel\uecce cheeses, the multi-strain LAB mixture completely prevented the growth of L. monocytogenes

Characterisation of the microflora contaminating the wooden vats used for traditional sicilian cheese production

Traditional Sicilian cheese productions are carried out employing traditional wooden vats, called tina. Many studies have highlighted the beneficial role of wooden dairy equipment by contributing to enriching the milk microflora and improving the acidification processes. The present work was undertaken to evaluate the safety of the wooden vats used to coagulate milk. To this purpose, the different microbial populations hosted onto the internal surfaces of the vats used to produce two different stretched cheeses, namely Caciocavallo Palermitano and Vastedda della valle del Bel\uecce DOP, were investigated for the presence of spoilage and pathogenic microorganisms as well as for bacteria with inhibitory effect in vitro against pathogenic microorganisms. A wide biodiversity of protechnological lactic acid bacteria (LAB), in terms of species, was revealed. Several LAB inhibited the growth of Listeria monocytogenes ATCC 7644. The wooden vats analysed resulted safe for three main findings: absence of the main pathogenic species, presence of high levels of LAB, anti-Listeria activity of many LAB