Feedback topics in modern code review: Automatic identification and impact on changes

Abstract

Recent empirical studies show that the practice of peer-code-review improves software quality. Therein, the quality is examined from the external perspective of reducing the defects/failures, i.e., bugs, in reviewed software. There is a very little to no investigation on the impact of peer-codereview on improving the internal quality of software, i.e., what exactly is affected in code, due to this process. To this end, we conducted an empirical study on the human-To-human discourse about the code changes, which are recorded in modern code review tools in the form of review comments. Our objective of this study was to investigate the topics which are typically addressed via the textual comments. Although, there is an existing taxonomy of topics, there is no automatic approach to categorize code reviews. We present a machine-learning-based approach to automatically classify reviewer-To-reviewer and reviewer-Todeveloper comments on proposed code changes. We applied this approach on 468 code review comments of four open source systems, namely eclipse, mylyn, android and openstack. The results show that Evolvability categories are dominating topics. In an attempt to verify these observations, we analyzed the code changes that developers performed on receiving these comments. We identified several refactorings that are congruent with the topics of review comments. Refactorings are mechanisms to improve the internal structure of software. Therefore, our work provides initial empirical evidence on the effectiveness of peercode-review on improving internal software quality.