SOSRepair: Expressive Semantic Search for Real-World Program Repair
by Afsoon Afzal, Manish Motwani, Kathryn T. Stolee, Yuriy Brun, Claire Le Goues
Abstract:
Automated program repair holds the potential to significantly reduce software maintenance effort and cost. However, recent studies have shown that it often produces low-quality patches that repair some but break other functionality. We hypothesize that producing patches by replacing likely faulty regions of code with semantically-similar code fragments, and doing so at a higher level of granularity than prior approaches can better capture abstraction and the intended specification, and can improve repair quality. We create SOSRepair, an automated program repair technique that uses semantic code search to replace candidate buggy code regions with behaviorally-similar (but not identical) code written by humans. SOSRepair is the first such technique to scale to real-world defects in real-world systems. On a subset of the ManyBugs benchmark of such defects, SOSRepair produces patches for 23 (35%) of the 65 defects, including 3, 5, and 8 defects for which previous state-of-the-art techniques Angelix, Prophet, and GenProg do not, respectively. On these 23 defects, SOSRepair produces more patches (8, 35%) that pass all independent tests than the prior techniques. We demonstrate a relationship between patch granularity and the ability to produce patches that pass all independent tests. We then show that fault localization precision is a key factor in SOSRepair's success. Manually improving fault localization allows SOSRepair to patch 24 (37%) defects, of which 16 (67%) pass all independent tests. We conclude that (1) higher-granularity, semantic-based patches can improve patch quality, (2) semantic search is promising for producing high-quality real-world defect repairs, (3) research in fault localization can significantly improve the quality of program repair techniques, and (4) semi-automated approaches in which developers suggest fix locations may produce high-quality patches.
Citation:
Afsoon Afzal, Manish Motwani, Kathryn T. Stolee, Yuriy Brun, and Claire Le Goues, SOSRepair: Expressive Semantic Search for Real-World Program Repair, IEEE Transactions on Software Engineering (TSE), 2020.
Bibtex:
@article{Afzal20tse,
  author = {Afsoon Afzal and Manish Motwani and Kathryn T. Stolee and Yuriy Brun and Claire {Le Goues}},
  title = {\href{http://people.cs.umass.edu/brun/pubs/pubs/Afzal20tse.pdf}{SOSRepair: Expressive Semantic Search for Real-World Program Repair}},
  journal = {IEEE Transactions on Software Engineering (TSE)},
  venue = {TSE},
  issn = {0098-5589},
  year = {2020},
  doi = {10.1109/TSE.2019.2944914},
  note = {\href{https://doi.org/10.1109/TSE.2019.2944914}{DOI:
  10.1109/TSE.2019.2944914}},

  abstract = {Automated program repair holds the potential to significantly
  reduce software maintenance effort and cost. However, recent studies have
  shown that it often produces low-quality patches that repair some but break
  other functionality. We hypothesize that producing patches by replacing
  likely faulty regions of code with semantically-similar code fragments, and
  doing so at a higher level of granularity than prior approaches can better
  capture abstraction and the intended specification, and can improve repair
  quality. We create SOSRepair, an automated program repair technique that
  uses semantic code search to replace candidate buggy code regions with
  behaviorally-similar (but not identical) code written by humans. SOSRepair
  is the first such technique to scale to real-world defects in real-world
  systems. On a subset of the ManyBugs benchmark of such defects, SOSRepair
  produces patches for 23 (35%) of the 65 defects, including 3, 5, and 8
  defects for which previous state-of-the-art techniques Angelix, Prophet,
  and GenProg do not, respectively. On these 23 defects, SOSRepair produces
  more patches (8, 35%) that pass all independent tests than the prior
  techniques. We demonstrate a relationship between patch granularity and the
  ability to produce patches that pass all independent tests. We then show
  that fault localization precision is a key factor in SOSRepair's success.
  Manually improving fault localization allows SOSRepair to patch 24 (37%)
  defects, of which 16 (67%) pass all independent tests. We conclude that (1)
  higher-granularity, semantic-based patches can improve patch quality, (2)
  semantic search is promising for producing high-quality real-world defect
  repairs, (3) research in fault localization can significantly improve the
  quality of program repair techniques, and (4) semi-automated approaches in
  which developers suggest fix locations may produce high-quality patches.},
}