Regression testing goals -View of practitioners and researchers

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2010

Aim: Regression testing practices in industry have to be better understood, both for the industry itself and for the research community. Method : We conducted a qualitative industry survey by i) running a focus group meeting with 15 industry participants and ii) validating the outcome in an on line questionnaire with 32 respondents. Results: Regression testing needs and practices vary greatly between and within organizations and at different stages of a project. The importance and challenges of automation is clear from the survey. Conclusions: Most of the findings are general testing issues and are not specific to regression testing. Challenges and good practices relate to test automation and testability issues.

Empirical Software Engineering

Regression testing is a means to assure that a change in the software, or its execution environment, does not introduce new defects. It involves the expensive undertaking of rerunning test cases. Several techniques have been proposed to reduce the number of test cases to execute in regression testing, however, there is no research on how to assess industrial relevance and applicability of such techniques. We conducted a systematic literature review with the following two goals: firstly, to enable researchers to design and present regression testing research with a focus on industrial relevance and applicability and secondly, to facilitate the industrial adoption of such research by addressing the attributes of concern from the practitioners' perspective. Using a reference-based search approach, we identified 1068 papers on regression testing. We then reduced the scope to only include papers with explicit discussions about relevance and applicability (i.e. mainly studies involving industrial stakeholders). Uniquely in this literature review, practitioners were consulted at several steps to increase the likelihood of achieving our aim of identifying factors important for relevance and applicability. We have summarised the results of these consultations and an analysis of the literature in three taxonomies, which capture aspects of industrial-relevance regarding the regression testing techniques. Based on these taxonomies, we mapped 38 papers reporting the evaluation of 26 regression testing techniques in industrial settings.

2011

Regression testing is the retesting of a software to check its reliability against the new functionality that is implemented or changes are made to the software. Regression testing plays a significant role to assess the quality of a product that is changed frequently in functionality as expected by the end user of the software. There has been a number of studies on various regression testing techniques as mentioned by Yoo and Harman in their survey but a very few are dedicated to the evaluating regression testing techniques. In this study various methods or schemes are suggested to measure the uniqueness of a test case. The uniqueness measure of a test case is a tool that is utilized to make decision on the effectiveness of various regression test techniques. Finally, building blocks for the construction of a framework are provided in the form of various schemes classified by their level of complexity involved. Concepts and methods that are utilized are already proven by academia an...

International Journal of Advanced Research in Computer Science, 2018

The prevailing scenario of ever increasing dependency of human beings on software applications has built pressure on software organizations to produce quality software. The quality of software is determined by many factors. One of the vital factors in deciding the software quality is optimized usage of testing tools and techniques employed during regression testing. Extremely high complexity of regression testing makes it necessary to utilize optimized way of running a selected and minimized test suite. Still, various challenges like redundancy, repetition ratio, Recurrence ratio or missing functionalities during regression testing are common. This paper discusses about the problems and challenges encountered during regression testing.

International Journal of Software Engineering and Knowledge Engineering, 2016

Software regression testing techniques verify previous functionalities each time software modifications occur or new characteristics are added. With the aim of gaining a better understanding of this subject, in this work we present a survey of software regression testing techniques applied in the last 15 years; taking into account its application domain, kind of metrics they use, its application strategies and the phase of the software development process where they are applied. From an outcome of 460 papers, a set of 25 papers describing the use of 31 software testing regression techniques were identified. Results of this survey suggest that at the moment of apply a regression testing technique, metrics like cost and fault detection efficiency are the most relevant. Most of the techniques were assessed with instrumented programs (experimental cases) under academic settings. Conversely, we observe a minimum set of software regression techniques applied in industrial settings, mainly...

Journal of Software Engineering & Intelligent Systems, 2017

Software testing facilitates with different methodologies. Among them regression testing has specific and valuable worth. Regression testing is merely a repetitive testing process, ensuring that the existing activities of the software are not affected by the changes being made in it. This testing basically quantifies the quality control aiming at approval of meeting the particular requirements after the code has been modified and making it sure that the presented code is not been pretentious. This paper is about analysis of regression testing and how it enhances and defines quality improvement in the particular software.

Regression testing is a costly but important maintenance activity which performed to revalidate modified software to make sure that changes did not adversely affect software behavior. Several studies have been carried out regarding regression testing domain in the literature. However, these studies need to be classified, summarized and ordered in a systematic manner to spot on the current state of the art of regression testing field. Therefore, this paper will carry out a systematic literature review (SLR) protocol to illustrate the process of preparation and undertaking a systematic review later. In this protocol, a manual search has been conducted to obtain related studies from four various sources where those studies were classified under journal, conference proceeding, and book chapter categories within specific criteria. Consequently, the initial search resulted in 1261 unfiltered studies, which then decreased according to certain predefined criteria up to 246 selected relevant studies. As a result, this protocol will behave as a guideline towards conducting SLR in the near future.

Communications of the ACM, 1998

In recent years, regression testing has been receiving increasing attention. Several selective regression testing strategies have been introduced. One question that demands an answer is the cost benefits of the selective regression testing strategies compared to the traditional retest-all strategy. This paper first presents a test cost model and then identifies the conditions under which the selective strategy is more economical than the retest-all strategy.

Information and Software Technology

Context. A majority of the regression testing techniques proposed by the research have not been adopted in industry. To increase adoption rates, we need to better understand the practitioners' perspectives on regression testing. Objective. This study aims at exploring the regression testing state of practice in the large-scale embedded software development. The study has two objectives, 1) to highlight the potential challenges in practice, and 2) to identify the industry-relevant research areas regarding regression testing. Method. We conducted a qualitative study in two large-scale embedded software development companies, where we carried out semi-structured interviews with representatives from five software testing teams. We did conduct the detailed review of the process documentation of the companies to complement/validate the findings of the interviews. Results. Mostly, the practitioners run regression testing with a selected scope, the selection of scope depends upon the size, complexity, and location of the change. Test cases are prioritized on the basis of risk and critical functionality. The practitioners rely on their knowledge and experience for the decision making regarding selection and prioritization of test cases. The companies are using both automated and manual regression testing, and mainly they rely on inhouse developed tools for test automation. The challenges identified in the companies are: time to test, information management, test suite maintenance, lack of communication, test selection/prioritization, lack of assessment, etc. The proposed improvements are in line with the identified challenges. Regression testing goals identified in this study are customer satisfaction, critical defect detection, confidence, effectiveness, efficiency, and controlled slip through of faults. Conclusions. Considering the current state of practice and identified challenges we conclude that there is a need to reconsider the regression test strategy in the Preprint submitted to Information and Software Technology November 29, 2018 companies. Researchers need to analyze the industry perspective while proposing new regression testing techniques. The industry-academia collaboration projects would be a good platform in this regard.