Synoptic: Summarizing system logs with refinement
by Sigurd Schneider, Ivan Beschastnikh, Slava Chernyak, Michael D. Ernst, Yuriy Brun
Abstract:

Distributed systems are often difficult to debug and understand. A typical way of gaining insight into system behavior is by inspecting its execution logs. Manual inspection of logs is arduous, and few tools can analyze an arbitrary system log out of the box. To support this task we developed Synoptic. Synoptic outputs a concise graph representation of logged events that captures important temporal event invariants mined from the log.

We applied Synoptic to synthetic and real distributed system logs, and found that it augmented a distributed system designer's understanding of system behavior with reasonable overhead for an offline analysis tool.

Synoptic makes no assumptions about the system, and requires no system modifications. In contrast to prior approaches, Synoptic uses a combination of refinement and coarsening instead of just coarsening to explore the space of representations. Additionally, it infers temporal event invariants to capture distributed system semantics that are often present in system logs. These invariants drive the exploration process and are preserved in the final representation.

Citation:
Sigurd Schneider, Ivan Beschastnikh, Slava Chernyak, Michael D. Ernst, and Yuriy Brun, Synoptic: Summarizing system logs with refinement, in Proceedings of the Workshop on Managing Systems via Log Analysis and Machine Learning Techniques (SLAML), 2010.
Bibtex:
@inproceedings{Schneider10,
  author = {Sigurd Schneider and Ivan Beschastnikh and Slava Chernyak and
  Michael D. Ernst and Yuriy Brun},
  title =
  {\href{http://people.cs.umass.edu/brun/pubs/pubs/Schneider10.pdf}{Synoptic:
  Summarizing system logs with refinement}},
  booktitle = {Proceedings of the Workshop on Managing Systems via Log
  Analysis and Machine Learning Techniques (SLAML)},
  venue = {SLAML},
  month = {October},
  date = {2--3},
  year = {2010},
  doi = {10.1145/1928991.1928995},
  address = {Vancouver, Canada},
  accept = {$\frac{9}{19} \approx 47\%$},

  note = {\href{http://doi.acm.org/10.1145/1928991.1928995}{DOI:
  10.1145/1928991.1928995}},

  abstract = {<p>Distributed systems are often difficult to debug and understand.
  A typical way of gaining insight into system behavior is by inspecting its
  execution logs. Manual inspection of logs is arduous, and few tools can
  analyze an arbitrary system log out of the box. To support this task we
  developed \emph{Synoptic}. Synoptic outputs a concise graph representation
  of logged events that captures important temporal event invariants mined
  from the log.</p>

  <p>We applied Synoptic to synthetic and real distributed system logs, and found
  that it augmented a distributed system designer's understanding of system
  behavior with reasonable overhead for an offline analysis tool.</p>

  <p>Synoptic makes no assumptions about the system, and requires no system
  modifications. In contrast to prior approaches, Synoptic uses a combination
  of refinement and coarsening instead of just coarsening to explore the space
  of representations. Additionally, it infers temporal event invariants to
  capture distributed system semantics that are often present in system logs.
  These invariants drive the exploration process and are preserved in the
  final representation.</p>},

  fundedBy = {Fulbright fellowship, 
	NSF CNS-0937060 to the CRA for the CIFellows Project, IBM John Backus Award},
}