A Framework for System Event Classification and Prediction by Means of Machine Learning

During operation, software systems produce large amounts of log events, comprising notifications of different severity from various hardware and software components. These data include important information that helps to diagnose problems in the system, e.g., post-mortem root cause analysis. Manual processing of system logs after a problem occurred is a common practice. However, it is time-consuming and error-prone. Moreover, this way, problems are diagnosed after they occurred—even though the data may already include symptoms of upcoming problems. To address these challenges, we developed the SCAPE approach for automatic system event classification and prediction, employing machine learning techniques. This paper introduces SCAPE, including a brief description of the proof-of-concept implementation. SCAPE is part of our Hora framework for online failure prediction in component-based software systems. The experimental evaluation, using a publicly available supercomputer event log, demonstrates SCAPE’s high classification accuracy and first results on applying the prediction to a real world data set.