Performance Evaluation of a Legacy Real-Time System: An Improved RAST Approach

A challenging aspect in optimizing legacy distributed systems with strict real-time requirements is how to evaluate the performance of the system running in a production environment without disrupting its regular operation. The challenge is even greater when the System Under Evaluation (SUE) runs within a resource-sharing environment and, thus, is affected by the resource usage of other software running in the same environment. Current performance evaluation methods dealing with this challenge rely on data collected by Application Performance Monitoring (APM) tools that are not always available in existing systems and hard to establish when the system is already in production. In this paper, we improve the initial, proof-of-concept implementation of our RAST (RegressionAnalysis,Simulation, and loadTesting) approach to evaluate the response time of a distributed system using the available system’s request logs. In particular, we greatly improve the prediction model based on machine learning. Our use case is a commercial alarm system in productive use, developed and maintained by the GSelectronic company in Germany. We experimentally demonstrate that our improvements significantly enhance RAST’s capability to adequately predict the system performance and verify the strict requirements on the response time. We make our model and software freely available in order to enable reproducing our experiments.