by Afsoon Afzal, Manish Motwani, Kathryn T. Stolee, Yuriy Brun, Claire Le Goues
Abstract:
Automated program repair holds the potential to significantly reduce software maintenance effort and cost. However, recent studies have shown that it often produces low-quality patches that repair some but break other functionality. We hypothesize that producing patches by replacing likely faulty regions of code with semantically-similar code fragments, and doing so at a higher level of granularity than prior approaches can better capture abstraction and the intended specification, and can improve repair quality. We create SOSRepair, an automated program repair technique that uses semantic code search to replace candidate buggy code regions with behaviorally-similar (but not identical) code written by humans. SOSRepair is the first such technique to scale to real-world defects in real-world systems. On a subset of the ManyBugs benchmark of such defects, SOSRepair produces patches for 23 (35%) of the 65 defects, including 3, 5, and 8 defects for which previous state-of-the-art techniques Angelix, Prophet, and GenProg do not, respectively. On these 23 defects, SOSRepair produces more patches (8, 35%) that pass all independent tests than the prior techniques. We demonstrate a relationship between patch granularity and the ability to produce patches that pass all independent tests. We then show that fault localization precision is a key factor in SOSRepair's success. Manually improving fault localization allows SOSRepair to patch 24 (37%) defects, of which 16 (67%) pass all independent tests. We conclude that (1) higher-granularity, semantic-based patches can improve patch quality, (2) semantic search is promising for producing high-quality real-world defect repairs, (3) research in fault localization can significantly improve the quality of program repair techniques, and (4) semi-automated approaches in which developers suggest fix locations may produce high-quality patches.
Citation:
Afsoon Afzal, Manish Motwani, Kathryn T. Stolee, Yuriy Brun, and Claire Le Goues, SOSRepair: Expressive Semantic Search for Real-World Program Repair, IEEE Transactions on Software Engineering (TSE), vol. 47, no. 10, October 2021, pp. 2162–2181.
Bibtex:
@article{Afzal21tse,
author = {Afsoon Afzal and Manish Motwani and Kathryn T. Stolee and Yuriy Brun and Claire {Le Goues}},
title = {\href{http://people.cs.umass.edu/brun/pubs/pubs/Afzal21tse.pdf}{SOSRepair: Expressive Semantic Search for Real-World Program Repair}},
journal = {IEEE Transactions on Software Engineering (TSE)},
venue = {TSE},
issn = {0098-5589},
year = {2021},
volume = {47},
number = {10},
month = {October},
pages = {2162--2181},
doi = {10.1109/TSE.2019.2944914},
note = {\href{https://doi.org/10.1109/TSE.2019.2944914}{DOI:
10.1109/TSE.2019.2944914}},
abstract = {Automated program repair holds the potential to significantly
reduce software maintenance effort and cost. However, recent studies have
shown that it often produces low-quality patches that repair some but break
other functionality. We hypothesize that producing patches by replacing
likely faulty regions of code with semantically-similar code fragments, and
doing so at a higher level of granularity than prior approaches can better
capture abstraction and the intended specification, and can improve repair
quality. We create SOSRepair, an automated program repair technique that
uses semantic code search to replace candidate buggy code regions with
behaviorally-similar (but not identical) code written by humans. SOSRepair
is the first such technique to scale to real-world defects in real-world
systems. On a subset of the ManyBugs benchmark of such defects, SOSRepair
produces patches for 23 (35%) of the 65 defects, including 3, 5, and 8
defects for which previous state-of-the-art techniques Angelix, Prophet,
and GenProg do not, respectively. On these 23 defects, SOSRepair produces
more patches (8, 35%) that pass all independent tests than the prior
techniques. We demonstrate a relationship between patch granularity and the
ability to produce patches that pass all independent tests. We then show
that fault localization precision is a key factor in SOSRepair's success.
Manually improving fault localization allows SOSRepair to patch 24 (37%)
defects, of which 16 (67%) pass all independent tests. We conclude that (1)
higher-granularity, semantic-based patches can improve patch quality, (2)
semantic search is promising for producing high-quality real-world defect
repairs, (3) research in fault localization can significantly improve the
quality of program repair techniques, and (4) semi-automated approaches in
which developers suggest fix locations may produce high-quality patches.},
fundedBy = {NSF CCF-1453474, NSF CCF-1563797, NSF CCF-1564162, NSF CCF-1645136, NSF CCF-1763423},
}