Publicly released software implementations of network protocols often have latent specification violations, or bugs. We present Ape, a technique that automatically explores program behavior to identify potential specification violations. Ape overcomes the challenge of exploring the very large space of behavior by dynamically inferring precise models of behavior, stimulating unobserved behavior likely to lead to violations, and refining the behavior models with the new, stimulated behavior. Ape can (1) discover new specification violations, (2) verify that violations are removed, (3) identify related violations in other versions and implementations of the protocols, and (4) generate tests. Ape works on binaries and requires a lightweight description of the protocol's network messages and a violation characteristic. We use Ape to rediscover the known heartbleed bug in OpenSSL, and discover one unknown bug and two unexpected uses of three popular BitTorrent clients. Manual inspection of Ape-produced artifacts reveals four additional, previously unknown specification violations in OpenSSL and μTorrent.
@inproceedings{Walls15issre, author = {Robert J. Walls and Yuriy Brun and Marc Liberatore and Brian Neil Levine}, title = {\href{http://people.cs.umass.edu/brun/pubs/pubs/Walls15issre.pdf}{Discovering Specification Violations in Networked Software Systems}}, booktitle = {Proceedings of the 26th IEEE International Symposium on Software Reliability Engineering (ISSRE)}, venue = {ISSRE}, address = {Gaithersburg, MD, USA}, month = {November}, date = {2--5}, year = {2015}, pages = {496--506}, doi = {10.1109/ISSRE.2015.7381842}, note = {\href{https://doi.org/10.1109/ISSRE.2015.7381842}{DOI: 10.1109/ISSRE.2015.7381842}}, accept = {$\frac{33}{151} \approx 22\%$}, abstract = {<p>Publicly released software implementations of network protocols often have latent specification violations, or bugs. We present Ape, a technique that automatically explores program behavior to identify potential specification violations. Ape overcomes the challenge of exploring the very large space of behavior by dynamically inferring precise models of behavior, stimulating unobserved behavior likely to lead to violations, and refining the behavior models with the new, stimulated behavior. Ape can (1) discover new specification violations, (2) verify that violations are removed, (3) identify related violations in other versions and implementations of the protocols, and (4) generate tests. Ape works on binaries and requires a lightweight description of the protocol's network messages and a violation characteristic. We use Ape to rediscover the known heartbleed bug in OpenSSL, and discover one unknown bug and two unexpected uses of three popular BitTorrent clients. Manual inspection of Ape-produced artifacts reveals four additional, previously unknown specification violations in OpenSSL and μTorrent.</p>}, fundedBy = {NSF CCF-1446683, NSF CCF-1446932, NSF CCF-1446966, NSF CCF-1453474}, }