JDiff: A Differencing Technique and Tool for Object-Oriented Programs
T. Apiwattanapong, A. Orso, and M.J. Harrold
During software evolution, information about changes between different versions of a program is useful for a number of software engineering tasks. For example, configuration-management systems can use change information to assess possible conflicts among updates from different users. For another example, in regression testing, knowledge about which parts of a program are unchanged can help in identifying test cases that need not be rerun. For many of these tasks, a purely syntactic differencing may not provide enough information for the task to be performed effectively. This problem is especially relevant in the case of object-oriented software, for which a syntactic change can have subtle and unforeseen effects. In this paper, we present a technique for comparing object-oriented programs that identifies both differences and correspondences between two versions of a program. The technique is based on a representation that handles object-oriented features and thus, can capture the behavior of object-oriented programs. We also present JDIFF, a tool that implements the technique for Java programs. Finally, we present the results of four empirical studies, performed on many versions of two medium-sized subjects, that show the efficiency and effectiveness of the technique when used on real programs.
MaTRIX: Maintenance-oriented Testing Requirement Identifier and Examiner
T. Apiwattanapong, R. Santelices, P.K. Chittimalli, A. Orso, and M.J. Harrold
This paper presents a new test-suite augmentation technique for use in regression testing of software. Our technique combines dependence analysis and symbolic evaluation, and uses change information between old and new versions of a program to (1) identify parts of the program affected by the changes, (2) compute the conditions under which the effects of the changes are propagated to such parts, and (3) create a set of testing requirements based on the computed information. Testers can use these requirements to assess the effectiveness of the regression testing performed so far and to guide the selection of new test cases. The paper also presents MaTRIX, a tool that implements our technique, and its integration into a regression-testing environment. Finally, the paper presents two empirical studies that demonstrate both the effectiveness of our technique and the shortcomings of previous techniques in assessing the adequacy of a test suite in exercising program changes.
Efficient and Precise Dynamic Impact Analysis Using Execute-After Sequences
T. Apiwattanapong, A. Orso, and M.J. Harrold
As software evolves, impact analysis estimates the potential effects of changes, before or after they are made, by identifying which parts of the software may be affected by such changes. Traditional impact-analysis techniques are based on static analysis and tend to identify most of the software as affected by the changes, due to their conservative assumptions. More recently, researchers have begun to investigate dynamic impact-analysis techniques, which rely on dynamic, rather than static, information about software behavior. Existing dynamic impact-analysis techniques are either very expensive-in terms of execution overhead or amount of dynamic information collected-or imprecise. In this paper, we present a new technique for dynamic impact analysis that is almost as efficient as the most efficient existing technique and is as precise as the most precise existing technique. The technique is based on a novel algorithm that collects (and analyzes) only the essential dynamic information required for the analysis. We discuss our technique, prove its correctness, and present a set of empirical studies in which we compare our new technique with two existing techniques, in terms of performance and precision.
A Differencing Algorithm for Object-oriented Programs
T. Apiwattanapong, A. Orso, and M.J. Harrold
During software evolution, information about changes between different versions of a program is useful for a number of software engineering tasks. For example, in regression testing, knowing which parts of a program are unchanged can help identifying test cases that need not be rerun. For many of these tasks, a purely syntactic differencing may not provide enough information for the task to be performed effectively. This problem is especially relevant in the case of object-oriented software, for which a syntactic change can have subtle and unforeseen effects. In this paper, we present a technique for comparing object-oriented programs that identifies both differences and correspondences between two versions of a program. The technique is based on a representation that handles object-oriented features and, thus, can capture the behavior of object-oriented programs. We also present JDiff, a tool that implements the technique for Java programs, and empirical results that show the efficiency and effectiveness of the technique on a real program.
An Empirical Comparison of Dynamic Impact Analysis Algorithms
A. Orso, T. Apiwattanapong, J. Law, G. Rothermel, and M.J. Harrold
Impact analysis - determining the potential effects of changes on a software system - plays an important role in software engineering tasks such as maintenance, regression testing, and debugging. In previous work, two new dynamic impact analysis techniques, CoverageImpact and PathImpact, were presented. These techniques perform impact analysis based on data gathered about program behavior relative to specific inputs, such as inputs gathered from field data, operational profile data, or test-suite executions. Due to various characteristics of the algorithms they employ, CoverageImpact and PathImpact are expected to differ in terms of cost and precision; however, there have been no studies to date examining the extent to which such differences may emerge in practice. Since cost-precision tradeoffs may play an important role in technique selection and further research, we wished to examine these tradeoffs. We therefore designed and performed an empirical study, comparing the execution and space costs of the techniques, as well as the precisions of the impact analysis results that they report. This paper presents the results of this study.
Leveraging Field Data for Impact Analysis and Regression Testing
A. Orso, T. Apiwattanapong, and M.J. Harrold
Software products are often released with missing functionality, errors, or incompatibilities that may result in failures, inferior performances, or, more generally, user dissatisfaction. In previous work, we presented the Gamma approach, which facilitates remote analysis and measurement of deployed software and allows for gathering program-execution data from the field. In this paper, we investigate the use of the Gamma approach to support and improve two fundamental tasks performed by software engineers during maintenance: impact analysis and regression testing. We present a new approach that leverages field data to perform these two tasks. We also present a set of empirical studies that we performed to assess the usefulness of the approach. The studies were performed on a real subject and on a real user population. The results of the studies show that the use of field data is effective and, for the cases considered, can considerably affect the results of dynamic analyses. Moreover, the empirical studies show that the approach is also efficient: the kind of field data that we consider requires very limited space and little instrumentation to be collected.
Improving Impact Analysis and Regression Testing Using FieldData
A. Orso, T. Apiwattanapong, and M.J. Harrold
Software products are often released with missing functionality, errors, or incompatibilities that may result in failures, inferior performances, or, more generally, user dissatisfaction. In previous work, we presented the Gamma approach, which enables analyses that (1) rely on actual eld data instead of synthetic in-house data and (2) leverage the vast and heterogeneous resources of an entire user community. In this paper, we investigate the use of the Gamma approach to support and improve two fundamental tasks performed by software engineers during maintenance: impact analysis and regression testing. We propose a new approach that leverages eld data to perform these two tasks. We also discuss ongoing empirical studies, performed on a real subject and on a real user population, to assess the feasibility of the approach.
Selective Path Profiling
T. Apiwattanapong and M.J. Harrold
Recording dynamic information for only a subset of program entities can reduce monitoring overhead and can facilitate efficient monitoring of deployed software. Program entities, such as statements, can be monitored using probes that track the execution of those entities. Monitoring more complicated entities, such as paths or definition-use associations, requires more sophisticated techniques that track not only the execution of the desired entities but also the execution of other entities with which they interact. This paper presents an approach for monitoring subsets of one such program entity-acyclic paths in procedures. Our selective path profiling algorithm computes values for probes that guarantee that the sum of the assigned value along each acyclic path (path sum) in the subset is unique; acyclic paths not in the subset may or may not have unique path sums. The paper also presents the results of studies that compare the number of probes required for subsets of various sizes with the number of probes required for profiling all paths, computed using Ball and Larus' path profiling algorithm. Our results indicate that the algorithm performs well on many procedures by requiring only a small percentage of probes for monitoring the subset.