Discovering Specification Violations in Networked Software Systems
by Robert J. Walls, Yuriy Brun, Marc Liberatore, Brian Neil Levine
Abstract:

Publicly released software implementations of network protocols often have latent specification violations, or bugs. We present Ape, a technique that automatically explores program behavior to identify potential specification violations. Ape overcomes the challenge of exploring the very large space of behavior by dynamically inferring precise models of behavior, stimulating unobserved behavior likely to lead to violations, and refining the behavior models with the new, stimulated behavior. Ape can (1) discover new specification violations, (2) verify that violations are removed, (3) identify related violations in other versions and implementations of the protocols, and (4) generate tests. Ape works on binaries and requires a lightweight description of the protocol's network messages and a violation characteristic. We use Ape to rediscover the known heartbleed bug in OpenSSL, and discover one unknown bug and two unexpected uses of three popular BitTorrent clients. Manual inspection of Ape-produced artifacts reveals four additional, previously unknown specification violations in OpenSSL and μTorrent.

Citation:
Robert J. Walls, Yuriy Brun, Marc Liberatore, and Brian Neil Levine, Discovering Specification Violations in Networked Software Systems, in Proceedings of the 26th IEEE International Symposium on Software Reliability Engineering (ISSRE), 2015, pp. 496–506.
Bibtex:
@inproceedings{Walls15issre,
  author = {Robert J. Walls and Yuriy Brun and Marc Liberatore and Brian Neil Levine},
  title = {\href{http://people.cs.umass.edu/brun/pubs/pubs/Walls15issre.pdf}{Discovering 
	Specification Violations in Networked Software Systems}},
  booktitle = {Proceedings of the 26th IEEE International Symposium on
  Software Reliability Engineering (ISSRE)},
	venue = {ISSRE},

  address = {Gaithersburg, MD, USA},
  month = {November},
  date = {2--5},
  year = {2015},
  pages = {496--506},
  doi = {10.1109/ISSRE.2015.7381842},  
  note = {\href{https://doi.org/10.1109/ISSRE.2015.7381842}{DOI: 10.1109/ISSRE.2015.7381842}},
  accept = {$\frac{33}{151} \approx 22\%$},

  abstract = {<p>Publicly released software implementations of network
  protocols often have latent specification violations, or bugs. We present
  Ape, a technique that automatically explores program behavior to identify
  potential specification violations. Ape overcomes the challenge of
  exploring the very large space of behavior by dynamically inferring precise
  models of behavior, stimulating unobserved behavior likely to lead to
  violations, and refining the behavior models with the new, stimulated
  behavior. Ape can (1) discover new specification violations, (2) verify
  that violations are removed, (3) identify related violations in other
  versions and implementations of the protocols, and (4) generate tests. Ape
  works on binaries and requires a lightweight description of the protocol's
  network messages and a violation characteristic. We use Ape to rediscover
  the known heartbleed bug in OpenSSL, and discover one unknown bug and two
  unexpected uses of three popular BitTorrent clients. Manual inspection of
  Ape-produced artifacts reveals four additional, previously unknown
  specification violations in OpenSSL and μTorrent.</p>},

  fundedBy = {NSF CCF-1446683, NSF CCF-1446932, NSF CCF-1446966, NSF CCF-1453474},
}