Ant Colony Optimization Based Model Checking Extended by Smell-like Pheromone

Model Checking is a technique for automatically checking the model representing software or hardware about whether they satisfy the corresponding specifications. Traditionally, the model checking uses deterministic algorithms, but the deterministic algorithms have a fatal problem. They are consuming too many computer resources. In order to mitigate the problem, an approach based on the Ant Colony Optimization (ACO) was proposed. Instead of performing exhaustive checks on the entire model, the ACO based approach statistically checks a part of the model through movements of ants (ant-like software agents). Thus the ACO based approach not only suppresses resource consumption, but also guides the ants to reach the goals efficiently. The ACO based approach is known to generate shorter counter examples too. This article presents an improvement of the ACO based approach. We employ a technique that further suppresses futile movements of ants while suppressing the resource consumption by introducing a smell-like pheromone. While ACO detects the semishortest path to food by putting pheromones on the trails of ants, the smell-like pheromone diffuses differently from the traditional pheromone. In our approach, the smell-like pheromone diffuses from food, and guides ants to the food. Thus our approach not only makes the ants reach the goals farther and more efficiently but also generates much shorter counter examples than those of the traditional approaches. In order to demonstrate the effectiveness of our approach, we have implemented our approach on a practical model checker, and conducted numerical experiments. The experimental results show that our approach is effective for improving execution efficiency and the length of counter examples. Received on 17 January, 2016; accepted on 11 April, 2016; published on 21 April, 2016


Introduction
Model Checking is a technique that checks whether the design of a software or hardware described as a state transition model satisfies the property specified by the user, which is called a specification [1].Tools that automatically perform the model checking are called model checkers.General model checkers use a deterministic algorithm for the checking, which often consumes too much machine resources because they exhaustively check their search spaces.In order to mitigate this problem, techniques based on Ant Colony Optimization (ACO) [2] have been proposed [3][4][5][6].They are nondeterministic algorithms.ACO is a swarm intelligence-based method inspired by the behaviors of ants' collecting food, and a multi-agent system that exploits artificial stigmergy (artificial pheromone) for the solution of combinatorial optimization problems.In other words, ACO is a kind of statistic algorithms that is categorized into meta-heuristics [7].Since ACO only searches a part of entire search space based on the property in which pheromone attracts ants, the ACO based model checking can suppress the use of the computational resources.
One of the major features of the model checker is to present an error trace to its users (called a T. Kumazawa et al. counter example), which helps the users understand why the model violates the property [8].Thus, the shorter the presented counter example is, the more comprehensible it is for the users.As an optimization method, ACO based model checking enables model checkers to generate shorter counter examples.
We propose a new ACO-based model checking that consumes less computational resources than the traditional ACO approaches do.Most model checking techniques deal with properties roughly categorized into safety and liveness [9,10].Safety properties state that undesirable things never hold, while liveness properties assert that the desirable things finally hold.Our approach handles the safety and focuses on the fact that finding the violation of the safety is reduced to the reachability problem on directed graphs.In other words, the model checking for the safety just searches final (i.e.error) states starting with a specific start state.For safety checking, a path from the start state to a final state is a counter example.
In the ACO based model checking, ants move in the directed graph to look for a final state.Ants probabilistically choose the move direction according to the amounts of pheromone trails on the graph deposited by preceding ants.This behavior of ants helps ants to find relatively short paths to the final states, because the paths that have much pheromone tend be selected by many ants.Thus pheromone trails represent the quality of paths to the final state in terms of path length.In order to further assist ants to search final states, we introduce another special pheromone that attracts ants to the final states.The attraction of the pheromone efficiently guides ants, and localizes the search space, so that our approach can achieve the less consumption of the resources.We can summarize the contributions of our approach to the traditional ACO based approaches as follows: 1. Suppressing memory consumption, 2. Decreasing the time for completing checking, and

Shortening obtained counter examples.
The structure of the balance of this article is organized as follows.The next section presents preliminaries of our approach.In the third section, we give a brief explanation of a traditional ACO based approach for safety.After that, in the fourth section, we introduce the smell-like pheromone and the details of our ACO approach that takes advantages of the smelllike pheromone.In the fifth section, we demonstrate the feasibility of our approach, and conclude our discussion in the sixth section.

Model Checking
In this section, we give an overview of model checking, and explain the problems that we address.

Overview
Formal verification is one of the techniques that verify properties of software or hardware.The formal verification techniques can be categorized into two kinds in terms of their approaches.One is logical reasoning approach that represents properties of software as a mathematical theorem such as Hoare logic or predicate calculus.The logical reasoning verifies software using a tool such as theorem provers.It is, however, difficult for the provers to verify software in completely automatic manner, and therefore, they require some interactions with the users.
The other one is Model Checking, which is proposed by Clarke et al [1,11] and known as one of the most successful research topics in Software Engineering.The model checking was initially applied for the verification of communication protocols [12] and hardware circuits.Model checking is currently used for verifying software without any support from users.Model checking consists of the following steps: 1) representing the model of a system as a state transition system such as an automaton, 2) describing a specification representing required property with temporal logic such as Linear Temporal Logic (LTL) [13] or Computation Tree Logic (CTL) [11], and 3) checking whether the state transition system satisfies the specification.The final step is automatically performed by a model checker.For example, SPIN [10,14] is one of the most popular model checkers.It checks whether a model described in Promela satisfies the specification described in LTL.Promela is the description language specifically designed for SPIN.At this time, SPIN converts the model and the negation of the specification into Büchi automata, then builds their intersection to exhaustively search for its accepting paths on it.If some accepting paths are found, it means the model does not hold the specification, and the corresponding paths are shown as the counter examples.Otherwise the fact of satisfaction is notified.
In the case of safety checking, the Büchi automaton of the given specification has the property that every path from its start state to its accepting state is accepting.Thus, its accepting states are considered to be final error states.Owing to the property, the accepting states of the intersection are also final error states, to which paths show counter examples.In other words, in order to find counter examples, it is sufficient to search for the accepting states on the intersection.In model checking, the automata tend to be very large.Actually, the size of an automaton increases exponentially as the size of the corresponding model becomes large.The exponential increase is called state explosion, which may cause exhaustion of resources and may lead to system failure.In order to mitigate the state explosion, techniques such as Partial Order Reduction that decreases the number of states [15], and Bitstate Hashing that reduces the memory area occupied by each state [10] have been proposed.Also, Symbolic Model Checking is known that it does not consume too much memory [16].They are, however, are symptomatic treatments, and therefore, do not essentially solve the problem.
The other problem is related to the length of counter examples.Because a counter example is presented to the user as a diagnosis of specification violation, it is highly desirable to make it comprehensible for the human user.As mentioned above, a counter example is represented as the path over the directed graph.It means that the counter example is more understandable if it is small enough in lengthwise.That is to say, we need to aim at the model checking methods that generate as short counter examples as possible.
Traditionally, the search techniques adopted by most model checkers are based on Depth First Search (DFS), such as Nested Depth First Search [10].However, DFS may find long counter examples because there is no guarantee of the shortness of paths that DFS searches.
Recently, parallel algorithms have been studied as promising techniques to improve the performance of model checkers.Holzmann et al. parallelizes Breadth First Search (BFS) to verify safety properties and implements the proposed algorithms on SPIN [17,18].Their technique can be used for the detection of the shortest counter examples, because BFS is guaranteed to find shortest counter examples unlike DFS.They also extend the proposed algorithm to handle the subset of liveness properties.
Apart from the improvements for traditional deterministic approaches, some heuristics algorithms have been proposed.They are effective in cases where counter examples are required, although it is difficult for the algorithms to prove that the given model holds the specification because heuristics cannot give exact solutions but quasi-optimal solutions.Edelkamp et al. developed HSF-SPIN [12,19], which is an extension of SPIN with heuristic search algorithms such as A*, Weighted A*, Iterative A* or Best First Search.The HSF-SPIN gives shorter counter examples and consumes less memory than the deterministic approaches.Groce and Visser proposed some heuristics for model checking of Java programs, based on the specifications to be checked, the structures of the programs, and users' definition [20].Their evaluation was conducted with DFS, Best First Search, A* and Beam Search.
Recent researches have shown that various metaheuristic approaches provide more favorable results than traditional deterministic algorithms do in terms of efficiency.Alba et al. showed the applications of Genetic Algorithm (GA) for detection of deadlock, and unnecessary states and transitions [21].Godefroid et al. proposed a GA based model checking technique [22].The GA based approaches were the first applications of meta-heuristics for model checking.Alba et al. later showed the effectiveness of Ant Colony Optimization (ACO) for the model checking [3].We describe the details of the ACO approaches in the next section.Their approach is extended to the checking of safety with Partial Order Reduction [5] and that of liveness [6].As another work, Francesca et al. proposed a deadlock detection technique using ACO [23].They utilize the heuristics based on the structures of models to tell ants estimated directions to food.In the paper [24], Ferreira et al. presented a novel technique based on Particle Swarm Optimization (PSO) that finds safety violations.Chicano et al. compared several deterministic and nondeterministic software model checking algorithms using Java PathFinder [25].Their target algorithms include DFS, BFS, A*, GA, ACO, PSO, Simulated Annealing, Random Search, and Beam Search.They reported that meta-heuristic techniques effectively found safety violations and short counter examples.Note that the work was the first one that applies Simulated Annealing to software model checking.
In addition to those already mentioned, there are some heuristic or meta-heuristic extensions.[29].Poulding and Feldt extended a variation of Monte-Carlo Tree Search algorithm for heuristic model checking [30].
It is worth noting that meta-heuristic approaches draw a lot of interest from the viewpoint of Search-Based Software Engineering (SBSE) [31].SBSE refers to a research area of Software Engineering that uses meta-heuristic optimization methodologies.For example, Shousha et al. used GA for the detection of concurrency problems (i.e., starvation and deadlocks) in design models described in Unified Model Language (UML) [32].Their method does not adopt model

ACO Based Approach
In this section, we give the basics of ACO, and then, describe ACOhg that is a variant of ACO proposed by Alba et al. [3][4][5] for model checking with large state spaces.

ACO
ACO is one of the meta-heuristic search algorithms inspired by the behaviors of ants that discover paths from their nest to food.It is known that ACO is effective for some optimization problems such as routing, assigning and scheduling problems.When ACO is applied to model checking problems, a problem is a model expressed in as a directed graph, where the nest and food are the start and error states respectively.The paths between these nodes represent candidate solutions of the model checking problem, i.e. counter examples.
In ACO, agents corresponding to ants cooperatively search the shortest paths through indirect communications using pheromone.The optimal paths found by ants correspond to the optimal solution in optimization problems.The effect of the pheromones is represented as weight on edges, which are assigned to the paths that ants have visited.The pheromones play a role as guides that induce ants to select shorter paths to error states.
The basic model of ACO was given by Ant System (AS) proposed by Dorigo et al. [33].In most cases, however, AS was not as powerful as other heuristic methods, and needs extensions for practical use.One of the most powerful extensions is Max-Min Ant System (MMAS) [34].MMAS, which is a kind of iterative algorithms, not only updates pheromones at each iteration step based on the best solution over the previous iterations, but also gives the upper and lower limits to pheromone value.The extension prevents MMAS going into local minimums.
Figure 1 shows how a typical ACO metaheuristic algorithm works.Basically, ACO is the repetition of ConstructAntSolutions step for searching paths, and UpdatePheromones step for updating the distribution of pheromone.Also, terminationCondition is satisfied when some solutions are found or the number of the repetition exceeds a fixed number.In the following, we discuss each step in more detail.
Initialization step initializes pheromones on edges for preparation of searching by ants.While regarding the start state and final states as the nest and food respectively, pheromones with random strength are randomly located between these states.
ConstructAntSolutions step makes each ant probabilistically transit states starting from the start state to find candidate solutions.At this time, a transition to the next state is selected using the following equation: In the equation, N i is the set of states to which an ant can move from state i; j is the destination state of one step movement of the ant; τ ij is the value of pheromone on the edge (i, j); and η is heuristic value representing the number of transitions to a final state, which is set to a smaller value than actual one.The heuristic value is estimated based on the locations of final states, or the length and property of a specification that are described in LTL.Also, α and β are empirically determined values in order to adjust the effects of the heuristic value and pheromone value, respectively.When ants have moved on edges in the fixed number of times, or some candidates of solutions are found, UpdatePheromones step follows.
UpdatePheromones step updates the pheromone value on each edge.The pheromone strengths on selected edges are increased according to the appropriateness of the edges.At the same time, the pheromones on the other edges are updated as follows: In the equation, x i is set to a value between 0 and 1.This is the value representing the degree of evaporation of pheromones.Through the update process, the priorities of previously selected edges are decreased step by step until some ants select the edges again.

ACOhg
It is difficult for traditional ACOs to handle state transition systems for model checking that have thousands of nodes.Because there can be billions of edges in such systems, they require a number of megabytes in a memory to record pheromone values.Especially, the size of a model of concurrent system is known to be huge.For example, the size of the model of Dining Philosophers with n philosophers is 3 n , which The simple solutions such as prohibiting revisits to the same states are not effective, because some ants may run into brick walls, or may wander from state to state for a long time even if they finally reach the final states.Thus, the behaviors of the traditional ACO in model checking may result in the state explosion without finding any candidates of solutions.In addition, as a more fatal problem, the traditional ACO has to initially set pheromone values to all the transitions.This may also consume too much memory.
In order to mitigate the problems of the traditional ACO, Alba et al. proposed ACO for huge graphs (ACOhg), which can perform searching with less memory than the traditional ones [3,4].The basic idea of ACOhg is to give the upper limit λ ant to the number of move steps of ants at one stage.This search manner suppresses the time and memory consumption, but limiting of move steps may prevent ants from reaching the final states.That is, λ ant has to be decided so as to find the final states.ACOhg gives two kinds of techniques for determining λ ant ; they are expansion technique and missionary technique.
In the expansion technique, once the system cannot find a final state in the search for the current λ ant , λ ant is increased by the value given as a parameter, and then the system searches the wider area defined by the new λ ant again as shown in Figure 2. The process starts with small λ ant and repeats until it finds some final states.The expansion technique increases λ ant just enough, so that the memory consumption can be suppressed.Also, it is easy to implement because it is a simple extension of the traditional ACO.On the other hand, its behavior becomes closer to the traditional ACO's as λ ant increases.
The missionary technique is similar to the expansion technique, but searches are performed from not the start state but some states on the edge of the previously searched area, i.e. ignoring old pheromone as shown in Figure 3.The new start state on the edge is selected from the states that ants reach at the previous stage.This search manner enables ACO to gradually extend the search area without changing λ ant , so that it only requires constant time and memory consumption at each stage.
Both approaches decide the strength of pheromones to be assigned based on a fitness function.The fitness function returns the degree of penalty for the trail of each ant, which becomes very large in the case where the trail includes some cycles, or no final state.Based on the fitness function f , a best with the lowest penalty f (a best ) is determined, and then, the pheromone of its trace is strengthened as follows: ACOhg, which is based on MMAS, also uses the fitness function for calculating the limit values τ max and τ min of the pheromone value for each trail as follows: In the equation, ρ and a are constants that control the range of the pheromone values.Also, the missionary approach uses the fitness function to decide the next start states at each stage.

Extended ACOhg
We extend ACOhg by introducing a new kind of pheromones in order to suppress futile movement of ants.We call the extended ACOhg EACOhg.This section presents the outline of EACOhg, and then describes the details of the new pheromone guiding ants to the final states.

Outline of EACOhg
ACOhg can suppress the analysis time and memory consumption, but it searches from the start state in any directions like the traditional ACO.If there is information about the direction, the search area can be localized.We extend ACOhg so as to use the direction information to find the final states.In the real world, the direction information corresponds to smell diffused from food.Indeed, ants can recognize not only for k = 1 to colsize do 5: a k ← a k + node;  pheromones, but also the smell, which is important information to reach unfound food in the early stage.
In our model, we regard the smell as another kind of pheromones.We call this goal pheromone.We made the goal pheromone stronger than the normal pheromone.Once the goal pheromones are put on transition edges, the edges are selected in preference to the edges with the normal pheromone as shown in Figure 4.The search manner localizes the search area further, contributing to finding the final states more quickly and generating shorter counter examples, although memory consumption may increase a little to hold the goal pheromones.
Figure 5 shows the pseudo code of EACOhg, where the kth ant is represented by a k , and a path where a k traversed is represented by |a k |.Also, a k j and a k * represent the jth node and the last node on the path Ant Colony Optimization Based Model Checking Extended by Smell-like Pheromone respectively.T (a k * ) represents the set of nodes to which a k can transit from a k * .The EACOhg starts with randomly assigned pheromone value in [0 − 1.0] to all the edges as shown in line 1.After that, the search step, which consists of movement of ants and update of pheromones, is repeated until the number of repetitions exceeds upper limit msteps or some ants reach the final states F in the loop body in lines 3-19.
Each stage performs a movement within λ ant and the movement causes pheromone update.The destination node to which to move is probabilistically selected based on the strength of the pheromone value τ through function select_successor in line 7.This node is appended to a k .At this time, the pheromone on the selected edge is enhanced through function local_pheromone_update in line 9. Thus, the best path based on the fitness function f in the stage is always held in a best .
Once the operation in the current stage is completed, pheromone values are globally updated for the effect of evaporation and enhancing pheromones on a best through the functions pheromone_evaporation and pheromone_update respectively in lines 15 and 16.Furthermore, in our algorithm, the goal pheromone is diffused through the function scatter_goal_pheromone.Finally, in order to search a wider area in the case where no ant reaches the final states.λ ant is increased in line 18.

Goal Pheromone
The existence of the goal pheromone shows that there are some paths from the current state to the final states.Therefore, it is enhanced in order to attract ants more strongly through the following properties: 1. Attracting ants more strongly than normal pheromone, 2. Non-volatility, and 3. Defusing from the final states to their peripheries.
Note that the above properties induce ants to select relatively short paths to the final states, which leads to short counter examples.The normal pheromones behave along with MMAS and hence, have the value less than or equal to τ max .The goal pheromone is set to the value more than τ max in order to attract ants more strongly than normal pheromones.Moreover, while the normal pheromone evaporates, the goal pheromone does not evaporate and has a constant value.It is derived from the fact that the smell is always supplied by the source, i.e. food.Thus, once an ant finds some goal pheromones, it moves to the edges with them with high probability.The behavior of an ant is implemented by select_successor.The goal pheromone is assigned on in-edges of the current states by scatter_goal_pheromone in order to make them defuse.At that time, the pheromone is scattered on only some edges that are randomly selected.The scatter manner contributes to suppressing time and memory consumption.

Experimental Results
In order to demonstrate the effectiveness of our approach, we have implemented our extended ACO, EACOhg, on a practical model checker called LTSA (Labeled Transition System Analyzer) [9,35].LTSA is one of the model checkers that can be customized easily, and can handle models that are suitable for our purpose.LTSA supports the checking of Fluent Linear Temporal Logic [35], which is a kind of LTL specialized for event-based systems described as Labeled Transition Systems.For example, we may give a model with several final states as an input of LTSA as shown in Figure 6(a), while LTSA generates models with a single final state, which is an error state corresponding to the final states for checking safety, as shown in Figure 6 (b).The property of single final state of the models makes handling of models easy.In Figure 6 (b), the red state labeled 0 is the initial state, and the state  We used values as shown in Table 1 for the coefficients and constants that appear in equations and algorithms.Most of these values can be basically the same values as ACOhg because EACOhg is a simple extension of ACOhg.However, the values of P p and P c have to be empirically decided.
We decided the best settings for P p and P c through preliminary experiments.In the preliminary experiments, we generated two models with small and middle sizes based on dinning philosophers problem, checking them for dead lock safety while changing P p and P c from 10 to 1000.Figures 7 and 8 show execution time for each experiment, where the small and middle models have 213 and 7,773 states respectively.Note that the figures are semi-logarithmic plots.As shown in the figures, the execution time decreases at first as they increase, regardless of the size of models.Once P p and P c As shown in Figure 9, our approach is more efficient than ACOhg.It shows that goal pheromones guide ants to the final states, suppressing the chances going out their ways.Furthermore, Figure 10 shows that our approach generates much shorter counter examples than ACOhg.The fact also substantiates the efficiency of our approach by the shorter trails of ants.
On the other hand, our approach consumes more memory than ACOhg as shown in Figure 11.The increase of the memory consumption fell within 10% at most, and was 5% in average, though.Considering that our approach have achieved 23% more efficient execution and 50% shorter counter examples in their averages, we can say that the increase is negligible.

Conclusions and Future Directions
We have proposed a novel ACO based model checking, i.e.EACOhg that further suppresses futile movements of ants while suppressing the resource consumption by introducing smell-like pheromone.The smell-like pheromone diffuses from goals and guides ants to the goals.Thus our approach not only makes the ants reach the goals further more efficiently but also generates much shorter counter examples than the existing best ACO based approach, ACOhg.
In order to show the effectiveness of our approach, we implemented the EACOhg on a practical model checker, and conducted experiments.The results of the experiments show that our approach is effective for improving execution efficiency and the length of counter examples.Future work involves the enhancement of EACOhg and its evaluation based on more realistic case studies.
We have observed that our approach does not achieve effectiveness in the very large model (model C) as shown in Figures 9 and 10.The reason may be that the effect of the smell-like pheromone is compromised in a large model.In order to overcome this problem, we plan to add directionality in diffusing the smell-like pheromone.
We are pursuing further improvements of EACOhg to make it more practical and complete.We envisage that the improvements should be achieved by the combination of EACOhg with heuristics and the optimization methods of classical model checking.Some researchers showed that appropriate heuristics can improve the efficiency of model checking in the wide range of problem domains [12,19,20,23].Most heuristics are problem-specific.They depend on the specifications to be checked and the structures of the target system models.Therefore we have to carefully investigate which heuristic methods contribute to the performance improvements of EACOhg.
Traditional model checking researches have proposed various kinds of optimization techniques as noted in Section 2. Although most of the methods aim at optimizing deterministic search, EACOhg can be applicable to some of them such as Partial Order Reduction (POR) [5].In addition, we plan to extend EACOhg to the checking of liveness, because EACOhg only targets on the safety checking and does not support the check the properties such as Response and Existence [36].In the case of liveness checking, counter examples may have infinite length and contains cycles.Thus, we have to investigate the effects of goal pheromones on finding paths with cycles.One approach is proposed by Chicano and Alba [6].As the next phase, we plan to apply our method to safety checking with POR and liveness checking so as to demonstrate the applicability of our approach.
Finally and the most importantly, we have to apply EACOhg to practical models of software systems or communication protocols to evaluate its effectiveness empirically.For example, the recent work by Chicano et al. [25] evaluates several meta-heuristic model checking techniques using communication protocols.Similarly, Shousha et al. [32] uses models of a bank fund transfer problem and a cruise control problem.EACOhg is not built on the specific programming or modeling languages, hence it is possible to adapt to the real world systems.A candidate data set for our empirical research is the Software-artifact Infrastructure Repository [37], which is used for evaluation by Poulding and Feldt [30].

Figure 7 .
Figure 7. Preliminary execution time on small model

Figure 8 .Figure 9 .
Figure 8. Preliminary execution time on middle model

Figure 10 .Figure 11 .
Figure 10.The length of counter examples Staunton et al. proposed the heuristic search algorithms for model checking based on Estimation of Distribution Algorithm [26, 27].Yousefian et al. proposed a safety checking technique for graph transformation systems based on GA to deal with the state explosion problem.[28].Rafe et al. proposed a hybrid algorithm combining PSO with Gravitational Search Algorithm in order to find deadlocks in graph transformation systems

Table 1 .
Settings of coefficients