Combining profiling and monitoring to analyze test coverage and identify performance problems

The use of profilers is a common approach for locating bottlenecks in software performance. Existing profilers typically generalize memory consumption and CPU usage. This work is dedicated to profiling-based identification of performance problems for specific moments of program execution. By combining conventional profiling with monitoring of user actions (e.g. mouse and keyboard inputs), a more fine-grained analysis of program behavior is possible. The calculation of coverage levels for GUI tests will also be available. The current state of this work describes a proposed solution. Realization of a prototype implementing the approach is currently ongoing.


INTRODUCTION
When improving the performance of a program, it is necessary to initially locate its bottlenecks.The use of profilers is a common practice and a helpful approach.These mostly extensive analysis tools monitor various system properties, such as memory or CPU usage, as well as information from the current program, such as method calls.In the subsequent evaluation phase, the previously collected measured values are combined to give the user a statistical picture.Part of this overview are statistics about data types and method calls, but also diagrams illustrating the use of resources over time.However, this deliberately generalized illustration also leads to a loss of information, since the state information of the program at a particular time can not be considered further.Nevertheless, this information would be helpful.Thus, in some cases it would be good to know, for example, which method calls or which user actions at a given time have led to the increase in resource consumption.In particular, with regard to automatic test execution, when the program execution is not observed by the user, this information would be useful in order to easily identify error sources.
This work should therefore be dedicated to the question of how to identify, based on profiling mechanisms, performance problems as precisely as possible in a specific scenario.

GOALS
The aim of this work is to implement a profiling tool for recording the actual program flow, especially the interplay of actions and reactions in order to later be able to identify parts of the program with potential performance issues as precisely as possible.Actions considered here are all events that provoke the invocation of methods, so as user input, but also the method calls itself.Reactions typically would be the called methods.The use of resources (memory usage and processor load) is of particular interest.Figure 1 illustrates the results of recording in a graphical representation.The graph shows the CPU usage as a function of time.The markings Button1 and Button2 are intended to easily identify user input at a certain time.This approach helps to easily identify performance issues from the user's view.It also allows to analyze the program flow from top-down without initially taking the code base into consideration, which might lead to confusion because of too many additional information.Furthermore, the combined reporting of user activities and method calls allows a detailed monitoring of the code coverage during test execution.Figure 2 shows the result of such coverage analysis.The coverage rate for method calls can be determined separately for each user action.The total coverage (e.g. of elements in the graphical user interface (GUI)) can also be reported (Figure 3).
Looking at test environments, this would allow to assess the effectiveness of each test step within a test case, or the effectiveness of the test suite as a whole.

WORK BREAKDOWN
To complete the above-mentioned objectives, several logging mechanisms has to be implemented.These include the tracking of system characteristics such as memory usage and CPU utilization, the program flow (method calls) and the logging of user interactions.In the second step, the data needs further processing and synchronization.Part of this step is the measurement of the GUI coverage as well as the code coverage.Finally a user interface must be implemented for the graphic representation of the results.Future tasks can involve further data analyzes like the comparison of different measurements or the identification of memory leaks.

RELATED WORK
For conventional profiling, there is broad body of tools available [3].The combination with user interaction at the GUI is out of focus.There is some existing work on combining profiling and monitoring [5].Jovic et al. describe an approach for the automation of performance testing of interactive java applications [4].
While conventional capture and replay (C&R) testing also focuses on monitoring user actions [7,2], the intention is different.In C&R the user actions are recorded for later replay only.There is no analysis on coverage levels of GUI elements.Since C&R targets GUI testing only, the underlying code remains hidden from the testing process.While this is in accordance to black-box testing procedures, it might not be very beneficial for performance testing, esp.for identifying and analyzing bottlenecks.
While there are approaches for monitoring GUI coverage [8] and also structured test design for GUI testing [6], the per- For approaches targeting the GUI performance [1], coverage is out of focus.

CONCLUSIONS
In this work, a combination of conventional profiling with monitoring of user actions (e.g.mouse and keyboard inputs) is proposed.This allows for a more fine-grained analysis of program behavior and also for a calculation of coverage levels (e.g. in GUI tests).The approach is currently a proposal, realization of a prototype implementing the approach is currently ongoing.