Distributed systems are often difficult to debug and understand. A typical way of gaining insight into system behavior is by inspecting its execution logs. Manual inspection of logs is arduous, and few tools can analyze an arbitrary system log out of the box. To support this task we developed Synoptic. Synoptic outputs a concise graph representation of logged events that captures important temporal event invariants mined from the log.
We applied Synoptic to synthetic and real distributed system logs, and found that it augmented a distributed system designer's understanding of system behavior with reasonable overhead for an offline analysis tool.
Synoptic makes no assumptions about the system, and requires no system modifications. In contrast to prior approaches, Synoptic uses a combination of refinement and coarsening instead of just coarsening to explore the space of representations. Additionally, it infers temporal event invariants to capture distributed system semantics that are often present in system logs. These invariants drive the exploration process and are preserved in the final representation.
@inproceedings{Schneider10, author = {Sigurd Schneider and Ivan Beschastnikh and Slava Chernyak and Michael D. Ernst and Yuriy Brun}, title = {\href{http://people.cs.umass.edu/brun/pubs/pubs/Schneider10.pdf}{Synoptic: Summarizing system logs with refinement}}, booktitle = {Proceedings of the Workshop on Managing Systems via Log Analysis and Machine Learning Techniques (SLAML)}, venue = {SLAML}, month = {October}, date = {2--3}, year = {2010}, doi = {10.1145/1928991.1928995}, address = {Vancouver, Canada}, accept = {$\frac{9}{19} \approx 47\%$}, note = {\href{http://doi.acm.org/10.1145/1928991.1928995}{DOI: 10.1145/1928991.1928995}}, abstract = {<p>Distributed systems are often difficult to debug and understand. A typical way of gaining insight into system behavior is by inspecting its execution logs. Manual inspection of logs is arduous, and few tools can analyze an arbitrary system log out of the box. To support this task we developed \emph{Synoptic}. Synoptic outputs a concise graph representation of logged events that captures important temporal event invariants mined from the log.</p> <p>We applied Synoptic to synthetic and real distributed system logs, and found that it augmented a distributed system designer's understanding of system behavior with reasonable overhead for an offline analysis tool.</p> <p>Synoptic makes no assumptions about the system, and requires no system modifications. In contrast to prior approaches, Synoptic uses a combination of refinement and coarsening instead of just coarsening to explore the space of representations. Additionally, it infers temporal event invariants to capture distributed system semantics that are often present in system logs. These invariants drive the exploration process and are preserved in the final representation.</p>}, fundedBy = {Fulbright fellowship, NSF CNS-0937060 to the CRA for the CIFellows Project, IBM John Backus Award}, }