A federation of simulations based on cellular automata in cyber-physical systems

In cyber-physical system (CPS), cooperation between a variety of computational and physical elements usually poses di ﬃ culties to current modelling and simulation tools. Although much research has proposed to address those challenges, most solutions do not completely cover uncertain interactions in CPS. In this paper, we present a new approach to federate simulations for CPS. A federation is a combination of, and coordination between simulations upon a standard of communication. In addition, a mixed simulation is deﬁned as several parallel simulations federated in a common time progress. Such simulations run on the models of physical systems, which are built based on cellular automata theory. The experimental results are performed on a federation of three simulations of forest ﬁre spread, river pollution di ﬀ usion and wireless sensor network. The obtained results can be utilized to observe and predict the behaviours of physical systems in their interactions.


Introduction
In recent years, more and more research focus on the cyber-physical systems (CPSs) [1], which are defined as integrations of computation, networking, and the physical systems.Taking advantages of wireless sensor network (WSN) [12], the sensing ability of CPSs is widely considered over the last years.This ability allows CPSs to be able to sense the physical world and transmit sensed data to base station for performing studies, analysis, and decision making.Simulating the sensing ability before real implementations highly reduces cost and effort of the development of CPSs.However, due to uncertain interactions of complex physical systems, simulating such type of system is much more complicated compared to the traditional computing systems.One of the critical challenges is involving of interoperability in the models.Recently, several approaches have been suggested to confront with that issue [5] [6] [20].But, they do not so far consider on federating physical systems instead of tightly combining of existing tools and languages.Furthermore, those solutions are targeting embedded systems.Natural systems and phenomena have not still been involved and examined in literature.Thus, modelling and federating such systems and phenomena are taken into account in the context.For the rest of this paper, physical system and natural system are interchangeably used.At present, cellular automata (CA) [2][3] model has emerged as a very promising technique for solving complex physical systems [4].It has been used to address complex problems in many fields of science, engineering, computer science, and economy.In particular, parallel cellular automata models are effectively applied in fluid dynamics, molecular dynamics, biology, genetic, chemistry, road traffic flow, image processing, and environment modelling.Hence, we propose to use CA to facilitate modelling complex and large natural systems.A CA typically consists of two main components: • Cellular space: This presents a lattice of cells.For each cell, we define a neighborhood that locally determines the evolution of the cell.All cells have

EAI European Alliance
for Innovation the same size in the lattice.Each cell can be in a certain state.
• Transition rule: This rule acts upon a cell and its neighborhood.The cell's state changes from one discrete time step to another.The CA evolves in time and space as the rule is subsequently applied to all the cells in parallel.
In this article, we present a new approach for federating simulations.This solution enables several CA-based simulations working together as a distributed simulation system, so-called mixed simulation.To achieve it, we at first detail a method to model physical systems in accordance with the CA [2][3] model.Then, the parallelism is employed to accelerate large simulations.A federation of several simulations is conducted based on the high-level architecture (HLA) [19] standard.The remainder of this paper is organized as follows.
In Section 2, we introduce related work.Section 3 describes the process of modelling physical systems under the terms of CA.Section 4 presents the definition of the mixed simulation.And then, a federation of mixed simulations and related discussions are presented in Section 5. Section 6 gives some experiments of running a federation.The last section concludes the paper.

Related work
In the last decade, several studies focus on handling challenges of interoperability of various components in CPSs [1].A few commonly interesting trends are briefly described as follows.
First of all, an approach about coordinating data communication and time synchronization between simulation frameworks is considered.Some practical works following that track are presented in [5][23].These solutions allow to federate several simulations, however, the issues related to synchronization time are not taken into account.
Likewise, [22] proposes a framework for exchanging data and time synchronization by integrating two available tools.Its aim is to facilitate design and evaluation of networked control and cyber-physical system (NCCPS) [22] in CPSs.
To sum up, no work considers on using of the CA model for representing natural phenomena in the context of CPSs.Modelling such systems as well as their interactions in space and time are not regarded in most cases.

Modelling physical systems based on cellular automata
A physical system is viewed as a portion of the universe and has interactions with other ones outside it.For instance, water flows in a river may impact on riverbanks or even a road system located near that river.As mentioned earlier, since complex behaviours and large scale of such systems, modelling this type of system confronts with many issues.Cellular automata (CA) [2][3] is considered as a good alternative supposed to deal with those issues [4].a) Cell system: At first, we propose a definition of cell system constructed based on the CA model.Such system consists of a collection of cells.Each cell holds its local state (interested parameters), which may be values of pollution density, insect population, humidity, or wind speed.A cell has connections and directions to adjacent cells (neighbouring cells).Simply, directions can be organized as pairs of number, shown in Table 1.
Table 1.An organization of directions in a cell system.
In this study, we suggest using two common types of two dimensional neighborhoods Von neumann 1 (4 neighbours: E, W, S, N) and Moore 1 (8 neighbours: E, W, S, N, NW, NE, SW, SE) [2], as presented in Figure 1.Table 2 formally shows an example of a cell system, which is generated from geographic data except for its local state being "pollution density".Notably, the number of neighbours of each cell is determined by the chosen CA and they are typically equal for all cells.But, in this case, cells' neighbourhood can differ from cell to cell since their original positions in the physical system.For example, Figure 2 visualises a river cell system in which cells are close to the riverbanks have less neighbour than other cells.b) The process of modelling physical systems: This is a set of ordered activities to achieve physical simulations from geographic data.Figure 3 simply depicts the process under the terms of the cell system definition.Geographic data are initially processed to generate cell systems, which are associated with definitions of states and transition rules to make up complete models.Physical simulations are obtained by just running the models.

Eliminating useless calculations according to cell system model
Raster data are widely used as inputs in many natural phenomenon models, however, this often leads to useless time because of computations on the uninterested regions.In other words, they are regions outside the target systems, as depicted in Figure 4. Actually, several works have done to deal with that issue and one representative is presented in [26], in which cells are not belonging to any target areas are marked a label "NoData" in the preprocessing phase.During the execution of the models, those cells are omitted.It effectively works in most cases, however, time cost for data checking has not yet eliminated completely.Cell system model helps to avoid meaningless processing by default.For example, in Figure 4, a cell system, which is extracted from geographic data, represents the river system.During executions, computations only occur on the river cell system.This approach is apparently more productive than the previous solutions.In the next sections, we are going to present the models of two physical systems: forest fire spread model and river pollution diffusion model.In addition to those models, we suppose that a WSN [12] is deployed for monitoring fire in the forest, and its model is thus presented as well.Since the generations of cell systems are automatic by facilitating of an open source tool, the next considerations are definitions of local states and transition rules.

River pollution diffusion model
River pollution diffusion model is built based on the method presented in the previous section.In the context of pollution, it is possible to think of various potential situations such as chemical, oil, or contaminant.The diffusion of those generally depends on density.Thus, pollution density is chosen as cell state for this model.
• state: The cell state holds a value of the pollution density.
• transition rules: Updating pollution density at cell i at time t+1, termed as S t+1 i .Generally, to make an evolution of the system, pollution density of each cell is subtracted by a half.This portion is equally delivered to its neighbours.A reversing transfer also takes place at the same time.

Forest fire spread model
The fire spread model is defined for simulating the fire spread in the forest.In reality, temperature and humidity are common parameters to be monitored to decide whether there exist fire or not in the forest.However, for the sake of simplicity, the cell state is abstractly represented by one of the four values: tree, fire, ash, and empty.
• transition rule: Updating the new state of cell i at time t+1, termed as S t+1 i .if (S t i is TREE and at least one of its neighbor is FIRE)

Wireless sensor network model
To represent a sensing component of CPS, a wireless sensor network (WSN) [12] is added to the context.It consists of a set of wireless sensor nodes that are scattered on target regions.Its featuring is to collecting environmental data and sending those data to centers (base stations) for processing.In WSN, connections between nodes are ensured via their waves.The maximum distance waves of one node can cover is determined by its communication range.Hence, nodes lie within communication range one node can connect to it.We assume that a collection of sensors is deployed in the forest.Each sensor node is represented by one cell in the context of the cell system.Then, the cellular space of the model is formed by links among those cells.Figure 6   • state: The cell state holds values of sensing data, which is collected from the forest (fire or normal).
• transition rule: Every step, all cells perform two main tasks, sensing and communicating.
To prepare communications with its neighbours, cells store sensing data and their identity into packets.The simplicity of the CA models brings about several advantages in modelling and simulation, especially complex physical systems.However, since huge sizes and complex behaviours of such system, modelling and simulating them pose many challenges to existing simulators.And this becomes more serious in the case of a system including several simulations cooperating together.
In recent years, simulations based on the parallelism A federation of simulations based on cellular automata in cyber-physical systems mechanism has been regarded in academic as well as industrial works.Several new architectures have been proposed to make possible designing and development of high performance environment based on the CA theory.Significant instances of these environments are CAM [28], CAMEL [29], StarLogo [30], and NEMO [31].These environments allow the exploitation of the inherent parallelism of the cellular automata model for the efficient simulation of complex systems that can be modeled by a very large number of cells with local interaction only.To carry it out, we propose to use the graphic processing unit (GPU) [7][8] in hopes of accelerating very timeconsuming simulations.The GPU provides hundreds of threads running in parallel.In this case, a cellular automaton is instanced as a SIMD (Single Instruction, Multiple Data) program.In fact, CA implemented as a number of processes mapped on threads that execute the same code on different data simultaneously.According to this approach, the transition function of a single cell of the system must be specified.As a result, the computations on cells are thus to be executed at the same time.The details of the implementations of parallel simulations in accordance with the Cuda [10][9] model are formally presented in Figure 7.

Mixed simulations
A mixed simulation is defined as a collection of parallel simulations organized as a distributed system.In such systems, many parallel simulations are concurrently run on different hosts connected by a network infrastructure.However, many important aspects of this type of system have to be taken into account the context.For such systems, the major consideration is how to synchronize their activities among hosts having their own time, which may be different.Obviously, it is impossible to achieve a global time for all hosts.Therefore, a new mechanism is required for synchronizing time between simulations in the context.To do that, we propose a central component for the proposed architecture.It is not only responsible for connecting parallel simulations, but also for coordinating their activities in time, as depicted in Figure 8.In other word, this component plays a role as a coordinator in the system.Along with time synchronization, data exchanging among simulations is one of main focuses of this study.Due to interacting via a network, it is necessary to find out a way to effectively transfer data among hosts, but still ensure loose coupling and scalability characteristic of the system.This will clearly be described in the next section.By achieving interoperability among distributed simulations, mixed simulations are expected to be able to imitate not only behaviours of real systems, but also interactions between them.For instance, considering the interactions between the river and the forest system, as shown in Figure 9. Ashes produced by fire in the forest can pollute the river at the frontier between them.Otherwise, evaporation will also affects fired spread in the forest.Thus, those physical interactions will be put into models in this type of simulation.

Federation of simulations
In this section, we will answer the question: "How can several simulations work together?".As mentioned earlier, a mixed simulation consists of several parallel simulations concurrently running on different networked hosts.Time synchronization is required to make data exchanged between simulations worthwhile.Towards that goal, an architecture is designed in accordance with the high-level architecture (HLA) [13][14] [19].In which, the entire system is viewed as a federation containing several federates linked via the central component run-time infrastructure (RTI) [19].federates are .The The HLA is formally defined by three components.
1.A set of rules describes the responsibilities of federates and their relationship with RTI.An example is that all exchange of data among federates should occur via the RTI during a federation execution [13].
2. An interface specification provides services for managing federates and interactions.For example, it indicates how a federate join or leave a federation [14].
3. An Object Model Template [15] defines how information is communicated between federates, and how the federates and federation have to be documented (using Federation Object Model FOM) [15].FOM defines the shared objects, attributes, and interactions for a whole federation.
As shown in Figure 10, we present the federation of three main federates representing the three parallel simulations being forest fire spread, river pollution diffusion, and WSN.An observer federate is also designed to visualise the federation.To prepare for the next explanations, we assume that there exist interactions between the three main simulations.Firstly, as fires of the forest approach to the river, ashes produced by the fires will pollute the river.Secondly, the evolution of the physical environment results in the changing of WSN's behaviour, particularly emitting signals at sensor nodes as soon as fire is detected in the forest.

Exchanging data
In the proposed model, exchanging data between federates is performed via the central component.However, sending and receiving a large amount of data among networked systems are not very feasible.Thus, a publish-subscribe mechanism is employed to serve that goal.This mechanism enables subscribers and publishers to determine what data they will send or receive without knowledge about each other.This obviously provides the network scalability for the system.To achieve it, federates have to pre-declare what data they publish and subscribe to the central component before execution of the federation.As briefly presented in Table 3, a FOM file contains such declarations of sharing data that are encapsulated as object classes of attributes.It is noted that along with sharing status data, positions of objects are also considered as well.Technically, two exchanges exist in the entire system.One is internal transferring between GPUs and CPUs after new states simultaneously computed on the federates' GPU, and the one externally occurs as data are exchanged among federates.Therefore, the states of publishers at time t may be involved in the evolution of other subscribers at time t+1, as depicted in Figure 11.This mechanism comes up with two properties, constrained and regulating.The former ensures the federates to be able to send updates.Meanwhile, the latter allows the federates to receive updates from the central component.Therefore, both of them are often enabled for all federates, and only the constrained property is assigned to the observer federate since it is designed without any sending.Table 4 shows time policies proposed for the proposed federation.In order to synchronize time, each federate associates its logical time with sending data, so-called time-stamp.Thanks to this information, the central component is capable to calculate the next time step for the federation as well as to coordinate federates as a synchronous system.Time stepped federates will calculate values based on a point in time and process all data being sent up to the next point in time (current time + time step).Thus, to advance logical time for the time-stepped simulation, each federate has to send its request to the central component.Then, all receive data, which have been sent from federates, with the time-stamp less than or equal to the time requested will be released from the central component.After those data have been received by federates, a time grant is returned to the requesting federate.And then, the federate is able to advance its logical time.The time advancement of a Federate F is posed after sending UpdateAttributeValues(UAV) service, this phase has three steps: Step 1: X sends a request using TimeAd-vanceRequest (TAR) service.
Step 2: X can receive reflectAttributeValue (RAV) callbacks.Then, X may update its system with received data, for example.
Step 3: X waits for the granted time t2, TimeAd-vancedGrant(TAG).At the TAG(t2) reception, the local time of the federate will be advanced to t2.At the beginning of a federation execution, a synchronizing point is basically required.Figure 13 illustrates how to initialize the synchronizing point for all federates.First of all, the river federate sends a synchronizing request to the RTI.The RTI will then responses to it and send an announce to the forest federate to achieve a synchronization point.Next, services will be used by both federates to confirm the synchronized point achieved.Therefore, for the proposed federation, the time synchronization of the four federates can be obtained via the following steps.
Step 1: All federates connect to the federation initialized earlier.
Step 2: The federates are put into a common time progress by requesting synchronization point services, Figure 13.
Step 3: The federates update the new states, and then send the updates to the central component if they are publishers.
Step 4: Each federate sends an advance time request to the central component.Step 5: For each federate, it needs to wait until all data on the central component with the time stamps are less than or equal to its requested time are received by subscribers, then it is able to advance its logical time.
Obviously, the proposed solution enables to create an environment in which several parallel simulations synchronously working together.Although it costs a little of time due to data transportation, it is significantly advantageous in the case of several complex and large systems.

Data used
Data used in this study was taken from OpenStreetMap [16].In which, we considered on a small area located in the Mekong Delta of Vietnam as a study region 9.Those data were used to generate cell systems of the river, forest, and WSN.The CA patterns were 8, 4, and 4 neighborhood, respectively.Since our current focus is the federation of mixed simulation, input data were randomly created for simulations.

FESI tool
We have developed the FESI (FEderation of SImulation) tool by using C/C++ language.It enables to develop parallel simulations and federate them as distributed simulations.Cell systems generated by PickCell open source software [11] are used as inputs.The framework CERTI [17] [18] was used in this project as the role of RTI.Meanwhile, the X Window System [24] was employed to provide a GUI (Graphical User Interface) environment for displaying simulation results and interacting with users.The computations of the parallel simulations run on the NVIDIA card GeForce GTX 680 1.15GHz with 1536 CUDA Cores (8 Multiprocessors x 192 CUDA Cores/MP).This board was connected to the Intel CPU 3.4GHz (8 CPUs x 4 cores/CPU).

Scenario 1 -The parallel simulation of pollution diffusion in the river
We run the river model for pollution diffusion presented in Section 3.2.Initially, two points were randomly polluted in the river.After 4 steps, the diffusion of pollution is shown in Figure 14.The darker regions implied the larger density of pollution, and vice versa.

Scenario 2 -Accelerating simulations based on GPU
We still used the pollution diffusion model in this case, its aim is to measure how faster model running on GPU compared to running on CPU.Besides, we used five target rivers with different sizes.Their cell systems generated from PickCell with the different number of cells ranging from 1,220 to 83,661 cells.Figure 15 presents the overwhelmingly fast GPU compared to the CPU in most cases.The gap increasingly becomes significant according to the increase of the number of cells.As the size of cell system is 83,661, the GPU was approximately 22 times faster than the CPU.The number of cells influences the performance for both the CPU and the GPU.On the CPU, the upward trend is very noticeable.The great increase starts from the size of 10,703 to 83,661 at a rate of 0.26(seconds)/1,000 cells.It is projected that the trend anticipation will be maintained with bigger sizes.Whereas, the rising on the GPU is not dramatic.It gradually rises between 1,220 and 83,661 at a rate of 0.01(seconds)/1,000 cells.

Scenario 3 -The interactions between the forest and the river federate
Regarding simulate the interactions of physical systems in CPSs.We launched a federation of a mixed simulation with three federates: river, forest, and observer.In which, the two first ones run on the GPUs.In this case, the river federate created and joined the federation on the RTI-CERTI.It waited for the forest federates to enter.The river federate sent a request to others to achieve a synchronization point in the federation.And then the synchronization point was achieved.Figure 16 showed that the ashes (brown points) formed by the fire (red points) polluted the river from the step 4 as they spread close to the river.This also shows that models based on the CA can work together in the common time progress.The models were presented in Section 4.2.The sensors nodes were represented by black points in the forest.They appeared with the sensing ranges (small circles) and communication ranges (large circles).As the previous case, the four federates first need to achieve a synchronous point.At each step, these federates exchange data together via the RTI. Figure 17 presents the results captured from the observer federate.In this scenarios, due to no fire close to the river, until step 4, there were no new polluted points created in the river.Meanwhile, since a sensor recognizes that the fire appeared within its sensing range (smaller circle), it will change its color, sensing and communication range (larger circle) to red color.

Conclusion
In context of modelling and simulating for cyberphysical systems, we have described a new approach on the federation for simulations.The models of physical systems are based on cellular automata.In this method, they must have at least two components: cell system and transition rules.The FEMIS tool has developed in order to simulate those models in parallel and perform the federations of those simulations.The parallel computations on the GPU aim to reduce the simulating time for large and complex models.The experimental results were obtained by federating the three parallel simulations for forest fire spread, river pollution diffusion and wireless sensor network.By using federated simulations, the behaviour of physical systems with their interactions could be observed in simulation progress.

Figure 1 .
Figure 1.Two commonly used CA neighbourhoods: The Von Neumann neighbourhood (left) consists of the central cell itself plus 4 adjacent cells, and Moore neighbourhood (right) in which there are 8 adjacent cells.

Figure 2 .
Figure 2. A cell system of a river system with Von Neumann 1.

Figure 3 .
Figure 3.A process of developing physical simulations based on cell systems.

Figure 4 .
Figure 4.The river is considered as a target system.

Figure 5 .
Figure 5. Depicting wireless sensor network.Large dotted circles imply communication ranges of nodes.
shows an example in which there are three sensor nodes deployed on a field (left figure), n1, n2, and n3.The corresponding cell system is given in the right figure.Cell states of the model are represented by sensing data holding one of two values (fire or normal).

Figure 6 .
Figure 6.WSN model based on cell system.

Figure 7 .
Figure 7.An implementation model of parallel simulations based on Cuda programming.

Figure 8 .
Figure 8.A general architecture of mixed simulations.

Figure 11 .
Figure 11.The technical view of exchanging data in the federation of two federates.

5. 2 .
Time management Time management is one of the important components of the recommended architecture.It controls the advancement of each federate in simulation time according to its time.This component allows federates in one federation to be able to synchronize their 6 EAI Endorsed Transactions on Context-aware Systems and Applications 11 2015 -02 2016 | Volume 3 | Issue 7 | e3 EAI European Alliance for Innovation Hang Van Tran et al. activities based on local logical time together to ensure the causal relationships.It is understandable that two types of time are considered in the context, one is local time maintained by federates, the other is global time controlled by the central component.

Figure 13 .
Figure 13.Federate synchronization for the river and the forest federate.

Figure 14 .
Figure 14.The parallel simulation of diffusing pollution in a river.This is initialized with two polluted points (dark points).

Figure 15 .
Figure 15.Accelerating simulations based on cell systems by using GPU.

Figure 16 .
Figure 16.The screen shoot was taken from the observer federate.It shows the data exchange between the the river and forest federate in the federation.Two regions marked the small red circles represent the new polluted points created by the ashes, which are formed from the forest fire after 4 steps.

Figure 17 .
Figure 17.The interoperability of four federates under the context of the mixed simulation: river, forest, WSN, and observer.The sensors changed to red color since the fire was detected close to them.

Table 2 .
An example about a cell system of 601 cells (Von Neumann 1).

Table 3 .
Describing publishers and subscribers of shared data between the four federates: forest, river, WSN, and observer.

Table 4 .
Time management of the four federates.