An Application of SMC to continuous validation of heterogeneous systems

This paper considers the rigorous design of Systems of Systems (SoS), i.e. systems composed of a series of heterogeneous components whose number evolves with time. Such components coalize to accomplish functions that they could not achieve alone. Examples of SoS includes (among many others) smart cities or airport management system. Dynamical evolution of SoS makes it impossible to design an appropriate solution beforehand. Consequently, existing approaches build on an iterative process that takes its evolution into account. A key challenge in this process is the ability to reason and analyze a given view of the SoS, i.e. verifying a series of goals on a ﬁxed number of SoS constituents, and use the results to eventually predict its evolution. To address this challenge, we rely on a scalable formal veriﬁcation technique known as Statistical Model Checking (SMC). SMC quantiﬁes how close the current view is from achieving a given mission. We integrate SMC with existing industrial practice, by addressing both methodological and technological issues. Our contribution is: (1) a methodology for validation of SoS formal requirements; (2) a formal speciﬁcation language able to express complex SoS requirements; (3) adoption of current industry standards for simulation and heterogeneous systems integration ; (4) a robust SMC tool-chain integrated with system design tools used in practice. We illustrate the application of our SMC tool-chain and the obtained results on a case study.


Introduction
Context and challenges A System of Systems (SoS) is a large-scale, geographically distributed set of independently managed, heterogeneous Constituent Systems (CS). Those entities are collaborating as a whole to accomplish functions/goals that could not be achieved otherwise by any of them, if considered alone [1]. Those goals can either be global to the system, or local to some of its constituents. Constituent Systems are loosely coupled and pursue their own objectives, while collaborating for achieving SoS-level objectives. An SoS adapts itself to its environment through (1) an evolution of the functions provided by its Constituent Systems and (2) an evolution of its architecture. A typical example of an SoS is the Air Traffic Management System of an airport, which coordinates incoming and outgoing aircraft as well as ground-based vehicles and controllers. Such an SoS may evolve in order to accommodate a larger number of passengers, difficult climatic conditions or a modification in the laws regarding security.
Misbehaviors of the functions provided by an SoS can be dangerous and costly. Therefore, it is important to identify a set of analyses and tools to verify that the SoS implementation meets functional and non functional requirements and correctly adapts to changes of the environment during any phase of the design and operation life-cycle. The manager of a SoS should be able to run these analyses in order to take decisions regarding the evolution of the SoS. The main characteristics of an SoS have been long debated since [1]. Nevertheless, one of the key characteristics of an SoS is dynamical evolution, that is, the fact that Constituent Systems may evolve, leave, fail, or be replaced.
In fact, dynamical evolution of SoS makes it impossible to design an appropriate solution beforehand. Consequently, the design flow of a SoS is reiterated multiple times during the operation of the SoS, in order to adapt to its evolutions. As a summary, SoS rigorous design methodologies thus introduce two major challenges. The first one is to check whether a given view of the system (i.e. a fixed number of components) is able to achieve a mission. The second challenge is to exploit the solution to the first challenge in an engineering methodology that permits dynamical evolution of the system in case new missions are proposed, in case the environment has changed, or in case the current view cannot achieve the mission. This design approach has been followed by the DANSE project which developed a new SoS design and operation methodology [2].
Contributions This paper focuses on Challenge one described above, that is to reason on a given view of the system. A corner stone to solve the challenge is to develop a usable language for expressing formal requirements independent from the number and identity of CSs and thus from the architectural choices.
In this paper, we propose Goal Contract Requirement Language (GCSL). This is a very expressive patternbased language to specify requirements on an SoS and its CSs. The language uses an extension of OCL to express constraints independent of the view such as: "the total fire area (over a set of districts) is smaller than 1 percent of the total area of the districts". By combining those constraints with temporal patterns, we express timing requirements on the behaviors of the view. For instance, the GCSL formula "always [SoS.itsDistricts.fireArea→sum()< 0.01 * SoS.itsDistricts.area→sum()]" checks that the above constraint is verified at any point of the simulation. A first version of GCSL was introduced in [3]. In this paper we propose additional patterns for expressing properties about the amount of time during which a given predicate remains satisfied. We show that GCSL is powerful enough to capture most of SoS goals and emergent behaviors proposed by our industry partners. The reader shall observe that the language can methodologically be extended to capture more requirements on demand.
The second main difficulty is to detect emergent behaviors and verify the absence of undesired ones. This requires a suitable verification technique. A first solution would be to use formal techniques such as model checking. Unfortunately, both the complexity and the heterogeneous nature of the constituent systems prevent this solution. To solve this problem, we rely on Statistical Model Checking (SMC) [4]. SMC works by monitoring executions of the system, and then use an algorithm from the statistics in order to assess the overall correctness. Contrarily to classical Validation techniques, SMC quantifies how close the view is from achieving a given mission. This information shall later be exploited in the reconfiguration process.
The objective of this paper is not to improve existing statistical algorithms, but to expand the SMC verifier PLASMA [5] which proposes several algorithms such as Monte Carlo or hypothesis testing. In order to implement an SMC algorithm for SoS, one has to propose a simulation approach for heterogeneous components as well as a monitoring approach for GCSL requirements. While a GCSL monitor is easily obtained by translating GCSL to Bounded Temporal Logic (BLTL) as in [6], simulating SoS requires that one defines a formal representation for SoS constituents. In this work, we exploit the Functional Mock-up Interface (FMI) [7] standard as a unified representation for heterogeneous systems. In recent years, the development of DESYRE, a simulation engine based on the System-C standard, provided us joint simulation for FMI/FMU. One of the main contributions of the paper is a full integrated tool-chain between IBM Rhapsody, the statistical model checker PLASMA [5] and DESYRE. This tool-chain is, to the best of our knowledge, the first one offering a full SMC-based approach for the verification of complex heterogeneous systems.
The third main difficulty of our work is to make sure that the technology will be accepted by practitioners, both in terms of usability and seamless integration with existing industrial practice and tools. Our first contribution provides a solution for expressing requirements. However, we need a language to specify the system itself. In this paper, we propose to support wide-spread industry standards for SoS. This is done by exploiting UPDM [8] for SoS architecture design and the FMI standard for constituent systems integration. We build on IBM Rhapsody which is a tool used to describe UPDM views, that we enrich with probability distributions. The latter captures uncertainty on the environment so that the verification accounts for predictions on environment's evolution. We then integrate the Constituent Systems behavioral models with the UPDM architecture through the FMI standard. By exploiting our SMC tool-chain, we thus provide full integration of our SMC-based tool for FMI/FMU within Rhapsody.
Our approach is illustrated on a fire emergency response SoS, at the scale of a city. The constituent systems are the districts of the city, firefighting cars, firemen, fire stations and the central command center. We show how our tools are used to model the architecture and the constituents systems of the SoS. The architecture indicates how the Constituent Systems cooperate to provide an emergent behavior, namely the extinction of the fires. Using our tool-chain, we evaluate the probability of this emergent behavior.
Structure of the paper The paper is organized as follows. In Section 4 we provide a summary of the Statistical Model Checking approach adopted in our tool-chain. Following this, in Section 5 we introduce the GCSL formal language for specifying formal requirements on SoS and CSs behavior. Finally, we discuss in Section 6 the SMC tool-chain and its integration with existing industrial tools for System design. This tool-chain is demonstrated on an industrial case study in Section 7, where we show its application to a Fire Emergency Response system designed in DANSE [9], modeling a complex SoS that manages fire emergencies in a large city.

SoS Modelling
A SoS model consists of a model of the SoS architecture and a set of models formalizing each Constituent System's behavior. We chose the UPDM language to describe both the SoS architecture and Constituent System's behavior, even though the latter can be described through other modelling languages, as explained in the following.
UPDM, or Unified Profile for DoDAF 1 and MoDAF 2 [8], is a modeling language standardized by the Object Management Group (OMG) in 2012. UPDM 2.0 defines a UML 2 and optional SysML profile to model architectures, support analysis, specification, design and verification for a broad range of complex systems, including SoS architectures and service oriented architectures. Through the profile extension mechanism inherited from UML, UPDM allows adding, extending and customizing standard 1 [10]. When a stereotype is applied to a model element, the value of the tags can be set for that model element. The remaining of this Section describes how we extended UPDM to accomplish architectural and behavioural representations supporting statistical model checking analysis.

Modelling Constituent Systems' Behaviour
To perform SMC, the designer needs to specify a stochastic behaviour for the constituent system. Typical examples when stochastic behaviours are relevant in a model include formalizing variability of external inputs and waiting time before events' occurrence, unpredictability of components' failure, and so on. To this end, the designer can use functions provided by the tool used for modelling. However, when using UPDM the designer can rely on a DANSE extension to the UPDM profile developed specifically for statistical model checking. We have defined the «Stochastic» profile with a set of adhoc stereotypes to support stochastic variables in SV-10b viewpoint (state-charts), the UPDM view we use to model CS behaviours. Our stochastic profile defines Uniform, Normal and CustomDistribution distributions stereotypes on the Integer and Real domains; custom distribution variables are based on user-defined probability distribution functions (further details can be found in [3]); each of the two "Uniform" stereotypes (for Integer and Real values respectively) has two tags to specify the min and max bounds of the range interval, whereas each of the two Normal stereotypes has two tags to specify the desired mean and standard deviation parameters. UPDM is not the only language we support to model Constituent Systems' behaviour. The choice on what modelling language best fits the CS behaviour specification depends on the physical phenomenon that the CS has to do with. In our approach we leave total freedom to the CS designer to use any modelling language, provided that the modelling tool implementing that language is able to export the CS model according to the FMI 1.0 standard [7], which is discussed later in this paper.

Modeling the architecture
The architecture of the SoS specifies how instances of constituent systems are interconnected. A SoS does not have a fixed number of CS instances over its execution because CSs are independently managed and may decide to leave or join a given SoS on their own. Therefore, rather than defining exactly how many CS are present in the SoS, we first specify an abstract architecture that defines the kind of CSs that can join the system and how they can interconnect with other CSs. For simulation, we fix a concrete architecture that is a possible instance of the abstract architecture. Our need for describing the architecture is satisfied by the UPDM 2.0 System Viewpoint 1 (SV-1).
The SoS Abstract Architecture. The abstract architecture is specified using the SV-1 Block Definition Diagram (BDD), a representation of the SoS hierarchy defining: Creating the UPDM SoS abstract architecture model requires for each CS, including those not specified UPDM (see Section 2.1), a corresponding UPDM representation as «System» blocks so that they can be later instantiated as parts of the overall SoS in the SoS concrete architecture. In particular, for those CSs not specified UPDM the interface, the parameters and the observable state variables specified in the UPDM «System» block must be exactly the same as the ones specified in the CS modelling tool. This correspondence is fundamental to guarantee that the simulator can match the UPDM representation of each CS with its associated executable file, thus allowing correct simulation.
The SoS Concrete Architecture. In order to define the concrete architecture, we rely on the SV-1 Internal Block Diagram (IBD), an integration view of the SoS defining: 1. a set of instances of CS types specified in the SoS abstract architecture, 2. the role of each CS instance in the SoS, including instance-specific parameter assignments, and 3. the connections between ports of CS instances, according to the abstract architecture.
The instance of a «System» block specified in the SoS abstract architecture (UPDM SV-1 BDD) is a «ResourceRole» block in the SoS concrete architecture (UPDM SV-1 IBD). The Value tag of UPDM Attributes in both diagrams can be defined to set (BDD) or override (IBD) the initialization value of the corresponding variable in the CS executable file. To assure that the simulator will use the initialization values specified in either architectural view the user must ensure that the name of the attribute matches the name of the associated variable in the CS model. Besides the «Stochastic» profile described in Section 2.1, we defined two other profiles to provide information for joint simulation and to express requirements. With these three profiles we can specify all the information needed to accomplish statistical model checking analysis over the specific SoS architecture and CSs behavioural models.
Providing Information for Joint Simulation. To provide information for joint simulation our «Simulation» profile defines the following stereotypes: «FMI», «Traceable» and «FMIIgnore». To support SMC analysis the simulation engine needs to know -besides other important information-the name and the location on the filesystem or on a remote server of each CS's executable file (Functional Mockup Unit) expressing its behaviour. Such information is provided by the FileName and URI tags of the «FMI» stereotype. The «FMI» stereotype has been made applicable to both UPDM «System» blocks and «ResourceRole» blocks. When applied to a «ResourceRole» block the information stored in the «FMI» stereotype attributes overrides the one contained in the corresponding «System» block.
SMC analysis requires observation of all (and only) those SoS model variables constrained by GCSL contracts. To allow the simulation tool (see Section 3) properly generating variable observers to track values only for the needed variables we have defined the «Traceable» stereotype and made it applicable to any attribute of «ResourceRole» elements representing a CS and «System» element representing the SoS.
Sometimes, the SoS UPDM model contains information not needed for SMC analysis. Filtering out such extra information to leave only the one essential to the SMC analysis stage is enabled by our «FMIIgnore» stereotype, applicable to model elements' ports and attributes. When this stereotype is applied to make a port "invisible" to the host simulator (DESYRE) also the interconnection between the two end ports needs to become "invisible". Consistently, the corresponding CS behavioural model will neither have to have such ports, nor its behavior will have to generate any data flowing through those ports.
Expressing Requirements. To pursue our SMC analysis goals on SoS we have defined a number of stereotypes collected by the «GoalsAndContracts» profile. One of the stereotypes of the «GoalsAndContracts» profile associates GCSL contracts to elements of the abstract architecture, possibly including the root element. To express contracts on model elements throughout our SoS UPDM SV-1 diagrams we rely on the UPDM "Constraint" model element inherited from UML [8].
To link a contract to CS or SoS «System» elements in a UPDM BDD, or to specific CS instances, stereotyped as «ResourceRole» in a UPDM IBD, we use the "constrainedElement" association provided by the "Constraint" model element. We have further extended the features of "Constraint" by defining the «gcsl» stereotype. The «gcsl» stereotype defines tags to specify (1) the GCSL assumption in textual form, (2) the GCSL guarantee in textual form, (3) the probability threshold with which the GCSL requirement has to be satisfied, and (4) the GCSL requirement identifier as an alphanumeric string.

Integrating Heterogeneous CSs' Behaviours
Constituent Systems are systems that evolve according to their own nature and mission, independently from the SoS. The nature of the CS implies that its behaviour be modelled using the most appropriate modelling tool for the specific application, which typically leads to a set of proprietary format of model representation and interfaces. Formalizing different Constituent Systems' behavior requires supporting a variety of models of computation, including continuous-time dynamics, discrete-time dynamics with variable or fixed integration step intervals and event-based dynamics. In many cases the size of such integration time-step spans from milliseconds to years within the same SoS model. For this reasons we refer to a SoS model as being a composition of heterogeneous Constituent Systems' models. To guarantee correct integration and simulation of heterogeneous CSs models to support statistical model checking we need to translate all these differences to a common format. Our approach requires CS behavioural models to comply with the Functional Mock-up Interface (FMI) [7].
A number of approaches are possible for integrating Heterogeneous CSs' Behaviours. Co-simulation relies on distinct simulation tools to provide a global simulation of the overall system. The integration can be performed using simulators' API, a common proprietary interface, such as the Simulink S-function, or a standard interface, for instance FMI for cosimulation. The approach proposed in this paper is based on "joint-simulation": a single simulator integrates the models of system components (i.e. CS models in the context of this work), each consisting of differential, algebraic and discrete equations describing components' continuous and discrete dynamics. To maximize tool interoperability the choice for a common interface to components' equations has fallen on the one defined by the Functional Mock-up Interface (FMI) standard [7].
FMI is a tool-independent standard whose development was initiated within the ITEA2 project MOD-ELISAR with the objective of easing the exchange of simulation models between different entities despite the usage of a variety of different tools. FMI standard was developed as a collaborative effort involving simulation tool vendors and research institutes. FMI standard maintenance and development (FMI 2.0 has been published in July 2014) is today performed by the Modelica Associations and it is currently supported by tens of academic and industrial tools.
FMI defines the interface of an executable, called Functional Mock-up Unit (FMU) and supports both model exchange and co-simulation of dynamic models. An FMU can either be self-integrating (i.e. can include its own numerical solver), or require an external simulator to perform its numerical integration. The former is the case of "FMI for co-simulation", the latter is called "FMI for model exchange". FMUs used for joint-simulation have to comply to the "FMI for model exchange" interface, version 1.0. This version of the interface allows handling "hybrid ODEs" [11] i.e. ordinary differential equations in state space form with events. Systems described by hybrid ODEs combine a piecewise continuous state (discontinuities occur at event instants) with a time-discrete state that changes only in correspondence of event instants.
Several modeling and simulation tools like Modelica, JModelica, Dymola, Rhapsody and Simulink/StateFlow support exporting models to FMI 1.0. Such tools can be seamlessly integrated with this core tool-chain thanks to our choice to adopt that standard. Joint simulation capabilities are provided by DESYRE [3], a simulation framework based on the SystemC standard and its discrete-event simulation kernel. Inputs to DESYRE are the SoS architecture exported from Rhapsody and the Functional Mock-up Units (FMUs) associated to CS types. Joint simulation of several FMUs, that are units complying with the FMI standard, is implemented by a Master Algorithm (MA), with two alternatives. In co-simulation, each FMU embeds its own ODE solver and computes autonomously the evolution of its continuous-time variables. In model exchange, the MA is in charge of computing evolution of continuous-time variable. In general the implementation of a Master Algorithm (MA) is not a trivial task having to guarantee: (1) correctness of the composition according to the model(s) of computation (MoC) of both the host environment and the constituent FMUs, (2) termination of the integration step and (3) determinism of the composition. Challenges related to the implementation of Master Algorithms for model composition, have been extensively addressed in the literature. In [12] the authors define the operational and denotational semantics of the (hierarchical) composition of Synchronous Reactive (SR), Discrete Event (DE), and Continuous if (csEvt ∅); then 6: evtQueue.addEvt(cs.getID(), csEvt); 7: end if 8: if (((evtQueue.getClosestEvtTime()−simT ime) > maxIntStep) and isCT ) then 9: evtQueue.addEvt(cs.getID(), maxIntStep); 10: end if 11: end for 12: while (simT ime ≤ simEndT ime and not(simStopEvt)) do 13: simT ime = getSimT ime(); 14: while (not(isSoSFixP tReached())) do 15: for all cs ∈ csList do 16: cs.updateDiscrState(simT ime); 17: end for 18: end while 19: for all cs ∈ csList do 20: cs.updateContState(simT ime); 21: end for 22: evtQueue.updateEvts(); 23: simT ime = evtQueue.getClosestEvtTime(); 24: waitNextActivationEvt(); 25: end while Time (CT) models. Termination and determinacy properties of MA for co-simulation are addressed in [13].

DESYRE Master Algorithm
Within the context of the DANSE project a specific FMI Master Algorithm has been developed in DE-SYRE to address the unique needs of Systems of Systems simulation and SMC. The main focus is on simulation efficiency due to SoS model complexity and large observation (i.e. simulation) time span (up to several years) and to support large number of runs (tens or hundreds of thousands) as required by SMC analysis. The MA builds on a set of assumptions that are typically satisfied by the CS models used within the DANSE context. The choice of a MA for model exchange rather than co-simulation provides us with full control of the overall integration algorithm. The MA assumes that none of the FMUs contains direct feed-through i.e. FMU output does not depend on the value of its inputs at the current simulation time, removing the need for a causality analysis during the fixed point computation at each step. The solver used for the continuous state of continuous-time FMUs is the simple forward Euler algorithm rather than higher order Runge-Kutta (RK) methods. While this method can be numerically unstable and its global error is linear with the integration step, it does not require the computation of state (and inputs) at intermediate steps, which makes the integration method compositional and does not require any "roll-back" mechanism due to the validity of the selected step size. In the current implementation the step size maxIntStep is specified by the user for the entire model but can be easily extended to support a different step size for each FMU. Besides affecting the rounding error and the stability of the algorithm the step size determines also the capability of detecting state events (i.e. zero-crossing) occurring during an integration step as well as their location on the time axis. In fact, state events are checked for only at the end of each integration cycle and, if detected, are given a time stamp equal to current simulation time with an error that in the worst case equals the discretization step maxIntStep. All these limitations were proven acceptable in all the use cases exercised within the DANSE project.
Lines 1 to 11 in Algorithm 1 represent the initialization phase, while lines 12 to 25 describe the SoS system simulation loop. The algorithm determines the time synchronization instants for the different FMUs composing the SoS model. Time synchronization points represent those time instants in which (1) the different FMUs are executed, (2) the generated outputs are propagated among their interfaces (line 16) and (3)

Joint-Simulation Traces
The execution of a SoS model results in a trace, that is also exploited by our SMC analysis. The formal definition of trace is provided by Definition 1.

Definition 1.
Given a set of variables V and their domain D, a state σ is a valuation of the variables, that is σ ∈ D V . A trace τ is a sequence of states and timestamps (σ 0 , t 0 ), · · · , (σ k , t k ), where ∀i t i ∈ R + ∧ t i < t i+1 .
In our setting, a trace contains an element for each time step of the simulation. At each step, the value of all observed variables are recorded. The observed variables V are the exported variable to which the «traceable» stereotype has been applied.

Background on Statistical Model Checking
Analyzing Systems of Systems requires a careful choice of the verification technique to use. A first solution would be to use model checking. However, this formal approach often requires an input written in a dedicated language, which conflicts with industry acceptance. Even if a complete model were made in a suitable language, analysis would not be feasible because of the very large size of SoSs and its heterogeneous nature. Therefore, we rely on Statistical Model Checking (SMC), which is a trade off between testing and model checking. SMC works by simulating the system and verifying properties on the simulations. An algorithm from the statistic area exploits those results to estimate the probability for the system to satisfy a given requirement.
The quantitative results provided by SMC is richer than a Boolean one. Indeed, if the system does not satisfy the requirement, a Boolean tool returns "not satisfied" without any evaluation of the probability of a correct behavior.
In order to apply SMC, one has to assume that the behavior of the system is governed by a stochastic semantic, that is the choice of the next state in an execution depends on a probability distribution. This hypothesis shall not be seen as a drawback. Indeed, most of SoS do make stochastic assumptions on their external environment or on their hardware. In case no distribution is known, one relies on the uniform distribution which has the maximal entropy.
In the rest of this section, we first present the BLTL logic used to express properties on system's executions. Then, we present some statistical algorithms used by the SMC engine.

BLTL Linear Temporal Logic
In this paper, we focus on requirements of SoS that can be verified on bounded executions. This assumption is used to guarantee that the SMC algorithm will terminate. The bounded hypothesis shall not be viewed as a problem. Indeed, like it is the case in the testing world, it is sufficient to consider that the system has a finite live time.
We present here BLTL, a variant of LTL [14] where each temporal operator is bounded. Properties of SoS will be expressed in GCSL, that instantiates to BLTL (see Section 5). The core of BLTL is defined by the following grammar, where the time subscript t is interpreted as an offset from the time instant where the sub-formula is evaluated: Here, AP is a set of atomic predicates defined in Section 5.2. In our case, an atomic predicate depends on the past states. The temporal modalities F (the "eventually") and G (the "always") can be derived from the "until" U as F t ϕ = true U ≤t ϕ and G t ϕ = ¬F t ¬ϕ, respectively. The semantics of BLTL is defined with respect to finite traces τ. We denote by τ, i | = ϕ the fact that a trace τ = (σ 0 , t 0 ), · · · , (σ , t ) satisfies the BLTL formula ϕ at point i of execution. The meaning of τ, i | = ϕ is defined recursively: Typically, a monitor, such as in [15], is used to decide whether a given trace satisfies a given property.

Statistical Model Checking
Given a stochastic system M and a property ϕ, SMC is a simulation-based analysis technique [4,16] that answers two questions: (1) Qualitative : whether the probability p for M to satisfy ϕ is greater or equal to a certain threshold ϑ or not; (2) Quantitative : what is the probability p for M to satisfy ϕ. In both cases, producing a trace τ and checking whether it satisfies ϕ is modeled as a Bernoulli random variable B i of parameter p. Such a variable is 0 (τ | = ϕ) or 1 (τ | = ϕ), with P r[B i = 1] = p and P r[B i = 0] = 1 − p. We want to evaluate p.
Qualitative Approach The main approaches [16,17] proposed to answer the qualitative question are based on Hypothesis Testing. In order to determine whether p ≥ ϑ, we follow a test-based approach, which does not guarantee a correct result but controls the probability of an error. We consider two hypothesis: H : p ≥ ϑ and K : p < ϑ. The test is parameterized by two bounds, α and β. The probability of accepting K (resp. H) when H (resp. K) holds is bounded by α (resp. β). Such algorithms sequentially execute simulations until either H or K can be returned with a confidence α or β, which is dynamically detected. Other sequential hypothesis testing approaches exists, which are based on Bayesian approach [18].
Quantitative Approach In [19,20] Peyronnet et al. propose an estimation procedure to compute the probability p for M to satisfy ϕ. Given a precision , Peyronnet's procedure, which we call PESTIM, computes an estimate p of p with confidence 1 − δ, for which we have: P r(|p − p|≤ ) ≥ 1 − δ. This procedure is based on the Chernoff-Hoeffding bound [21], which provides the minimum number of simulations required to ensure the desired confidence level. The quantitative approach is used when there is no known approximation of the probability to evaluate, i.e. to obtain a first approximation. This method is useful when the goal of the analysis is to have a idea on how well the model behaves. On the contrary, the qualitative approach determines whether the probability is above a given threshold, with a high confidence and in a minimal number of simulations.

Timed OCL Constraints for SoS Requirements
The challenge in promoting the use of formal specification languages in an industrial setting is essentially to provide a good balance between expressiveness and usability. The SoS setting introduces further challenges for the need of specifying properties of the state of complex systems and architectures.
Finally, for scalability reasons, monolithic verification approaches should be avoided in favour of compositional ones, such as those based on the theory of contracts [22]. In this section we present the requirements language, called GCSL (Goal and Contract Requirement Language). The full GCSL language specification, as well as the details of translation of the GCSL patterns into BLTL, can be retrieved in [23]. In this section we focus on providing a brief recap of the language and illustrate why it is appropriate to define SoS and CSs requirements. We also discuss an explicit contribution of this paper to GCSL.

A Survey of GCSL
A GCSL contract is a pair of Assume/Guarantee assertions denoting requirements on SoS and CSs inputs and outputs, respectively. Contracts allow us to decompose requirements and perform local or global verification on need. Assertions are built upon GCSL natural-language patterns, some of which are shown in Figure 1. GCSL patterns are inspired by and extend the Contract Specification Language (CSL) patterns [24], developed in the SPEEDS European project. These natural-language based requirements have their formal semantics defined by translation into corresponding BLTL formulas, enabling the application of SMC. To simplify the specification of properties of complex systems and architectures, GCSL integrates the Object Constraint Language (OCL) [25], a formal language used to describe static properties of UML models. OCL is an important means to improve the expressiveness and usability of GCSL patterns. Using OCL we can describe properties about types of CS in the SoS architecture, while being independent of their actual number of instances and, thus, defining requirements that are adaptable to the natural evolution of the SoS, without the need of rewriting them.
OCL allows also to define algebraic constraints over Constituent Systems attributes. In summary, GCSL Patterns are able to express real-time constraints over complex portions of the entire SoS state.
To show the expressiveness of GCSL, consider the following simple example (based on Pattern 12 from Fig. 1 This architecture-abstract requirement says that if any CS of type CriticalComponent has one of its TempSensor measuring at time t a temperature that is higher than the threshold set by the specific CriticalComponent (the threshold may be different for distinct components) then one of the CoolingFan connected to that component should be switched on within the [t+1 min,t+5 min] time frame. This property does not depend on a concrete architecture or on the number of the mentioned CSs. It can be used as a requirement for any SoS that integrates the mentioned CSs types.
The idea of mixing OCL and temporal logic originates from the need of specifying static and dynamic properties of object-based systems. In [26,27] OCL has been extended with CTL and (finite) LTL, respectively, without support for real-time properties. The work in [28] is more similar to ours and it is based on ClockedLTL, a real-time extension of LTL. ClockedLTL is slightly more expressive than BLTL, because it allows unbounded temporal operators, whereas BLTL is decidable on a finite trace.
As mentioned previously, patterns are motivated by infeasibility of direct use of BLTL, which is a very low level language and commonly used properties may require the nesting of several temporal operators, with the risk of making requirements hard to understand and error-prone. Patterns provide a convenient way to represent frequently occuring and well-identified schemes, while avoiding errors due to the complexity of the underlying logic. One can expect to have a library of patterns, which is what the methodology developed during the DANSE project prescribed [23], in order to capture the most relevant temporal constraints for the considered domain. The methodology developed during the DANSE project prescribes to have a library of patterns [23] that captures the most relevant temporal constraints for the considered domain. In the rest of this section we are going to discuss a number of these pre-defined patterns. Clearly, their choice is inevitably subjective and depends on the field of application as well as on the test case at hand. The expressiveness of BLTL, jointly with OCL, makes the patterns library easily extensible to cover future, domain-specific needs. Figure 1 shows some of the behavioural patterns of GCSL (all the patterns can be found in [23]). Patterns 2 and 3 are typical safety properties, where Ψ denotes an argument where an OCL property can be used to describe a state of the SoS. The translation of pattern 2 into BLTL states that the atomic property Ψ should be true at every real-time instant within simulation end time k. Pattern 3 is translated similarly. Pattern 8 shows the joint use of events counting (the n indicates a number of occurrences triggering the pattern) within a real-time interval, such as [3.2 seconds, 25.7 minutes]. Its translation relies on the BLTL operator occ and, even if it does not involve any temporal operator, it cannot be decided on a single state but it needs to be checked across the entire trace. In this case we have no explicit mentioning of the simulation time k but we have the overall constraint on all patterns requiring a, b to have appropriate values (a ≤ b ≤ k). Pattern 12 is used to express a liveness property which is triggered by an initial condition Ψ 1 occurring at time t and discharged by a following condition Ψ 2 occurring within the interval [t + a, t + b] that is relative w.r.t. the time t of occurrence of Ψ 1 . Its translation into BLTL underlines a number of important aspects of this pattern. First, the pattern is verified on the entire simulation up to time k − b. This is needed because if a condition Ψ 1 occurs at k − b, we need to have enough remaining time (actually b) to verify whether either it is discharged by a condition Ψ 2 (making the pattern true) or not. As a consequence, a condition Ψ 1 occurring after time k − b would not require any following discharging condition. Second, on the occurrence of a condition Ψ 1 the pattern requires a shift to time a (as close as possible, depending on the actual states produced by simulation) which is indicated by the next operator X ≤a . From that point onwards, we can check for the occurrence of the condition Ψ 2 in the remaining interval using F ≤b−a .

Contribution to GCSL
Before illustrating the new patterns, we enrich the set of atomic predicates of BLTL with adequate timing operators.
On extending BLTL Usually atomic predicates describe properties of system states, e.g. by comparing a variable with a constant. We propose here an extension where atomic predicates also depend on the past (i.e. from states before the current one). In particular, we are interested in measuring the amount of time during which a given atomic predicate has been true. The syntax for our predicates is as follows: occ(AP , a, b) | N exp ± N exp Here, • contains the usual boolean connectors , ± the usual arithmetic operators and the usual comparison operators. Given a trace τ = (σ 0 , t 0 ), · · · , (σ k , t k ) and a step i, our predicates are interpreted as follows:

[[occ(p, a, b)]](τ, i) = a≤t j ≤b 1 {true} ([[p]](τ, j))
The dur function computes the amount of time during which the predicate p has been true since the beginning of the trace. The #T ime notation returns the simulation time at the current point. The occ function computes the number of steps in which a predicate holds within the given time bound. For instance, G ≤t (dur(UP) > 0.9 · #T ime) is true iff for every step between 0 and t, the amount of time during which UP holds is at least 90% of the elapsed time.
On monitoring extended BLTL In order to support the new #T ime and dur constructs, we extend the atomic predicates available in BLTL. Concerning the #T ime variable, we first recall that each state in a trace contains a timestamp, according to Definition 1. Indeed, this value is necessary for verifying patterns involving time bounds. At a given state, each predicate is evaluated by replacing #T ime by the timestamp of that state. The predicate dur(φ) is evaluated by accumulating the amount of time during which the predicate φ evaluates to true, according to its semantics defined above.
On GCSL extension We are now ready to present our new contribution to GCSL, that is patterns 13 and 14. Those patterns are suitable for expressing safety and reliability constraints, such as the availability of a SoS. As shown in Figure 1, the BLTL translation of these patterns relies on the novel #T ime and dur. Pattern 13 checks that, at each simulation point during [a,b], the amount of time during which Ψ has been seen to be true so-far is at least e% of the currently elapsed simulation time. We use the operator dur to accumulate the overall time during which Ψ is true, and we compare the accumulated value with the required portion of the current simulation time, extracted with the operator #Time, at each simulation point. Pattern 14 checks that, at time [b] the amount of time during which Ψ has been seen to be true is at least the required portion of b. Figure 2 shows a (simplified) portion of the grammar defining the OCL integration within the GCSL atomic properties (indicated by OCL-prop). Properties are constructed using usual boolean operators (•) and basic arithmetic comparisons ( ) between expressions. Attributes of Constituent Systems can occur in properties or expressions and can be accessed by using their Fully Qualified Name (FQN), such as SoS.Sensor03.isOn.
OCL propositions can describe properties about sets of Constituent Systems that are unknown at requirements-definition time. These sets are left undetermined because (1) the requirements may apply to several variant architectures of the same SoS and ID Pattern (below, k is the simulation time and a < b ≤ k)

OCL-coll
::= attribute | csName | its(type) | connected(type) | OCL-coll . OCL-coll (2) the SoS architecture may evolve during the SoS life-cycle. Indeed in [1] one of the five SoS-specific properties of a complex system is that the number and type of systems participating to a SoS may change over time. Ideally, the specifications of the system should remain independent of the number and type of CSs, which is exactly what OCL provides as a feature within GCSL. In order to support properties that are parametrized by the SoS architecture, GCSL provides the quantifiers forAll and exists that allow us to instantiate properties over finite collections (OCL-coll) of Constituent Systems (the var ranges over these collections and occurs in the OCL-prop which is the scope of the quantifier). A corner case of quantification is provided by the set operators empty and notempty that simply return a truth value after testing the emptiness of the collection. Standard OCL allows to concatenate object names (here indicated as csName) by the "."-containment relation, until reaching an attribute. In GCSL we add (1) another (weak) containment operator (its) that allows to navigate the systems hierarchy in terms of CSs types and (2) an operator (connected) that allows to navigate the neighborhood of a CS, again in terms of types.
Quantifiers occur also in expressions (OCL-expr) and allow to aggregate the values of (equally-typed) attributes. E.g. the simple expression (SoS.its(Sensor).temp → sum( ))/(SoS.its(Sensor) → size( )) can be used to compute the average temperature in a SoS where the number of CS of type Sensor is unknown or time-dependent.
Since BLTL does not interpret the OCL syntax (except for FQN), the translation from GCSL to BLTL is done once the architecture is fully known and fixed. OCL operators can be nested and their elimination is performed by induction over the structure of the formula in an outside-in fashion. Conceptually, this can be done by repeated application of three steps: (1) resolution of the outermost OCL collections (that is, replacing a collection with a finite set of FQN), which eliminates its and connected operators, (2) elimination of the universal (existential) quantifiers, replaced by the corresponding conjunctions (disjunctions) of the instantiated scopes, (3) elimination of the remaining operators by replacement with corresponding Boolean or arithmetic expressions, and (4) recursion on new outermost OCL collections. Termination of this procedure is trivially proved. In practice, this procedure is reiterated each time the SMC analysis is called in order to capture every architecture change.
In practice the tool-chain dynamically evaluates the formula as the architecture changes. termination is ensured by the fact that at each quantifier elimination the number of quantifiers nesting decreases ++++ Mention the potential combinatorial blow-up of the formula and the need of an onthe fly evaluation algorithm. For example, assume that the requirement considered previously is checked on the architecture in Figure 3. The collection SoS.its(CriticalComponent) would be resolved into the set {SoS.cc01,SoS.cc02}. Then, the new outermost collections would be SoS.cc01.its(TempSensor), SoS.cc01.connected(CoolingFan), SoS.cc02.its(TempSensor), and SoS.cc02.connected(CoolingFan).
That This elimination-based approach is justified by the fact that SoS development with architecture reconfiguration has, typically, a larger time-scale than the operational level of the CSs. So architecture snapshots can be individually analysed, while architectures alternatives are generated from generic architectural pattern.
After having illustrated the main features of GCSL and how they support the SoS specification and dynamic evolution challenges we discuss, in the following section, how to obtain a unique behavioural model of the SoS by integration of single and heterogeneous CSs behavioural models.

Performing Statistical Model Checking
This Section describes the work-flow and the toolchain (see Figure 4) we have set-up to accomplish statistical model checking analysis on Systems of Systems models. The tool-chain has been adopted by the industry partners of the DANSE Project: CARMEQ, IBM, EADS, and Thalès. The core of our SMC tool-chain is composed of three main tools: IBM Rhapsody is the tool implementing the UPDM language to model SoS architectures, DESYRE is the tool providing the joint simulation engine for SMC and PLASMA [5] is the tool providing the SMC analysis engine. Other modelling tools like Modelica and Simulink/Stateflow surround this core tool-chain to provide behavioural models in the form of FMUs. Figure 4 shows our SMC analysis work-flow. The work-flow starts with two parallel branches. The topmost block of the left branch represents the parallel development of Constituent Systems' models, each with its best-fitting modelling tool. The topmost block of the right branch represents the development of the SoS architecture in UPDM as described in Section 2.2. Once each CS model gets ready (e.g. for jointsimulation and SMC analysis) it is exported from its modelling tool into the FMI compliant format. This step is represented by blocks named "FMUs" in the left branch of Figure 4. On the right branch, when the UPDM blocks in the SoS architecture are 1) in synch with their corresponding CSs models (e.g. same interfaces) and 2) correctly interconnected, our Rhapsody exporter plug-in can be invoked to translate the UPDM model into a form that DESYRE can use to un joint-simulation. The two branches merge into the "DESYRE Internal Representation" being the set of FMUs and intermediate format of the SoS architecture. At this point of the work-flow all the information needed to run the statistical model checking analysis is available. To launch the SMC analysis the user has to provide a proper SMC analysis configuration.

IBM Rhapsody
IBM Rational Rhapsody [29] is a model-based system engineering environment implementing industrystandard languages such as UML, SysML and UPDM. Referring to Figure 4, IBM Rhapsody represents the starting point of the SMC analysis work-flow. The SoS architecture is specified within this tool by using the UPDM industry-standard language, conveniently extended to meet SMC analysis specification needs. Figure 5 shows how the UPDM extensions described in Section 2 have been implemented in IBM Rhapsody. During DANSE Project IBM has extended the Rhapsody tool to generate FMI 1.0 compliant FMUs from most of UPDM behavioural diagrams, thus making it join the pool of those candidate modelling and simulation tools able to export CS models for our SMC analysis purposes.
Exporting information for SMC. IBM Rational Rhapsody provides a Java API set for integration with external tools. We developed a Java Exporter Plug-in to translate information from the UPDM SoS architecture model, along with all the information related to SMC analysis i.e. GCSL requirements, FMU names and URIs, and traceability settings to a format intelligible by DE-SYRE, the joint simulation engine. Our Exporter Plug-in integrates into Rhapsody as an UPDM extension profile («DANSEExporterPlugIn» profile in Figure 5). The effect of loading our «DANSEExporterPlugIn» profile is that a menu item named "Export Model 4 DANSE" is added to the "Tools" menu of IBM Rhapsody. Once invoked from the IBM Rhapsody "Tools" menu our Exporter recursively parses both the UPDM Block Definition Diagram and the Internal Block Diagram to translate all the information needed to perform SMC analysis into the DESYRE internal representation format (XMI).

SMC Analysis in PLASMA
PLASMA is a tool for performing SMC analysis. Contrary to existing tools, PLASMA offers a modular architecture which allows to plug new simulators and new input languages on demand. This architecture has been exploited to verify systems and requirements from various languages/specific domains such as systems biology [30], a train station [31], or an autonomous robot [32]. The core of PLASMA is thus a set of SMC algorithms, which includes those presented in Subsection 4.2 as well as more complex ones [5]. This core is completed by two types of plug-ins, that are controlled by the SMC algorithm. First, the simulator plug-ins which implements an interface between PLASMA and a dedicated simulator to produce traces from a dedicated input language on demand. Second, there is a checker plug-ins that verify whether a finite trace satisfy a property.
In this paper we extended the facilities of PLASMA as follows. First, we built a new plug-in between PLASMA and DESYRE in order to produce traces from FMI-FMU model. Second, we used the BLTL checker plug-in that we enrich with the two new primitives dur and #T ime in order to monitor those traces and report the answer to the SMC algorithm. As a last implementation effort, we also implemented a compiler from GCSL to BLTL.
Interaction between PLASMA and DESYRE. Before invoking PLASMA, one has to launch DESYRE and load a SoS model as well as the corresponding FMUs. Then DESYRE GUI allows the designer to set and run the SMC analysis. In particular, the designer selects the type of SMC algorithm to use and the corresponding parameters (number of traces for Monte Carlo, precision and confidence for Chernov Bound . . . ). The designer also selects the properties to be evaluated among those embedded in the UPDM model.
Once the analysis is launched, DESYRE translates the GCSL properties to BLTL, as described in Section 5. It then invokes PLASMA and transmits the list of properties to check, as well as the algorithm and parameters to use. PLASMA then requests DESYRE to start the first execution. The execution is controlled step-by-step by PLASMA. At each step DESYRE provides a new snapshot of the system state, by giving the current value of every observed port in the architecture. A detailed description of the interaciton between PLASMA and DESYRE is provided in [3]. Exporter plug-in terminates the translation of the UPDM SoS architectural model it prompts a "Save" window to store its output on the user's file-system. Figure 7 illustrates how the user imports the DESYRE Internal Representation into DESYRE. The import is accomplished in two steps: the Intermediate Format of the SoS architecture exported from IBM Rhapsody is imported and then all the FMUs needed for SMC analysis are imported. To run a DANSE statistical model checking analysis the user creates a statistical model checking configuration in DESYRE. Figure 8 shows DESYRE's SMC Analysis Configuration Editor. From the editor the user selects one SMC Analysis Method out of the ones provided by PLASMA. Figure  8 shows that the Montecarlo method has been selected. According to the selected analysis method the editor    values for variables the user has to select or specify the correct path in the System model field (top of Figure  8). GCSL contracts specified in the IBM Rhapsody UPDM model are available in the DESYRE intermediate representation and can be imported into the SMC configuration editor by clicking the Import button highlighted in the bottom right of Figure 9. In this case the user might not be interested in validating all the imported GCSL contracts through the SMC analysis she/he is configuring. To address this need we decided to make the imported GCSL contracts selectable by means of check-boxes. Such check-boxes are visible in the lower part of Figure 9. If needed, further GCSL contracts can be added by clicking the Add button in the Contracts form, as shown in the top-right corner of Figure 9. This action invokes the GCSL Contract editor window , where the user can specify the new GCSL contract in its three components: Assumption, Promise and Owner, the latter being the model element to which the GCSL contract predicates on. When the configuration is completed the user can launch the SMC analysis campaign. The Play button to run the SMC analysis is shown in the top-right corner of Figure 8.

The SMC Workflow from the User Perspective
After the SMC analysis has terminated the user selects the SMC Analysis Results view, that appears as shown in Figure 10. SMC analysis results are displayed per GCSL requirement as shown in the bottom-right corner of Figure 10.

A Case Study
This section illustrates the application of our technology to a concept alignment example that was defined for the DANSE Project. This case study has the particularity to embed all the difficulties of the case studies proposed by EADS, THALES, and CARMEQ that, for confidentiality reasons, cannot be described in this paper.

Modeling
We modeled an emergency response SoS for a city fire scenario in UPDM. The city is partitioned into 10 districts, and we focus on a few fire-fighting constituent systems (CS). We consider the following CSs: the Fire Head Quarter (FireHQ), the Fire Stations, the Fire Fighting Cars and the Fire Men. The behavior of the CSs Figure 11. CS behaviour modelled in other tools (e.g. OpenModelica) Figure 12. SoS architecture in Rhapsody has been modeled in several FMI-compliant authoring tools. For example, the FireMan has been modeled in OpenModelica (as shown in Figure 11) which is an open-source multi-domain modeling tool based on the Modelica language. Other CSs have been modeled using Rhapsody state-charts. The CSs rely on the new UPDM profile to include probabilistic behavior. For instance, each District models occurrences of fires by randomly choosing the time before the next fire according to a exponential distribution. The time before a fire is reported to the head quarter is also randomly chosen. We remind the reader that the objective of this work is not to learn the probability distribution itself (this has to be done via observations), but rather to show that it is conceptually possible to incorporate such information within the model.
The SoS integrated architecture was built by instantiating the CSs and by specifying how to connect them through an Internal Block Diagram, shown in Figure 12. The SoS architecture is exported to DESYRE using a DANSE-specific exporter plugin. Each CS behavioral model is exported from the corresponding authoring tool into FMUs, according to the FMI standard. This enables the DESYRE platform to simulate the whole SoS model and to plot some selected variables -see Figure 13 for an illustration.
The simulation is parameterized by its duration, expressed in the time of the model. For our experiments, we choose to simulate 10 000s of execution. Since our model of computation is event based, the computation time needed for running this simulation depends on the number of events occurring during the simulation. Whenever there are few events, such as in the top of Figure 13, the simulation takes a few seconds. In that case, there is no event between two pikes, corresponding to two fires that are very quickly extinguished. Simulations involving more events, such as the one at the bottom of the Figure, require a few dozen of seconds to complete. In that case, fires are not extinguished and the time between two events is kept small to describe the evolution of the fire.

Expressing Goals of the SoS
Our main objective is to check that the fire area remains small enough. In order to define "small enough" independently of the number of components, we require that the fire is less than a given percentage of the total area.
In our model, each district has two variables of interest, its area and the fire area.
Our first formulation states that the fire area is always less than X percent of the total area. The total fire area is the sum of the fire area in each district, which can be expressed in GCSL by SoS.itsDistricts.fireArea->sum(). We define Pattern 1 as follows: always [SoS.itsDistricts.fireArea → sum() < (X/100) * SoS.itsDistricts.area → sum()] As Pattern 1 might be too strong, we propose an alternative formulation. More precisely, we allow the fire area to exceed X% of the total area, but no more than 10% of the time. For technical reasons, we define Pattern 2 as the negation of the above property, namely: Pattern 2 is true whenever the fire area is above the threshold for more than 10% of the time, that is when the SoS behaves incorrectly. As we want the probability that the system behaves correctly, we have to compute the probability of the complementary event. This is done by subtracting the probability that Pattern 2 holds from 1.

Unwanted Emergent Behaviors Detection and Evaluation
One of the challenges in SoS design is the detection and analysis of unwanted emergent behaviour. In our case, simulation allowed us to detect an (undesired) emergent behaviour which is depicted in the lower part of Figure 13. Our analysis of this emergent behaviour is the evaluation of the probability of its occurrence. The first step is to define a GSCL pattern that characterizes the absence of the emergent behaviour. One key characteristic of this behaviour is that fires spread over entire districts. We assume that the emergent behaviour does not occur if there is no area where the fire has taken the whole district. This is specified in Pattern 3:

Analysis and Discussions
In this section, we use SMC to compute an estimate of the probability for Pattern 1, 2 and 3 to hold. We use the PESTIM method which is parameterized by an allowed error , and a confidence 1 − δ. We chose an error of 0.1 and a confidence of 99% (δ = 0.01), which requires 265 simulations traces. These traces are obtained by running stochastic simulations of the model. The length of a simulation is set to 10000s. We present the analysis results and time for Pattern 1 in Table 1. The analysis result is an estimation of the probability that Pattern 1 holds, based on the 256 traces. As expected, the probability that the fire remains smaller than X% of the total area increases when X increases. Indeed, "the fire area remains smaller than X% of the total area" implies that "the fire area remains smaller than Y % of the total area" for any Y ≥ X. However, the probability returned is an approximation, with an error up to 0.1 with a confidence of 99%. Therefore, the fact that the probability decreases from 0.96 to 0.95 when X increases from 0.01 to 0.1 is not significant. Indeed, the difference between the two values is less than the error. On the contrary, the difference between the probabilities obtained for X = 0.0001 and X = 0.001 are significant since they are more than twice the error. In our model, the total area is about 23 square kilometers. Therefore the two last lines of the table correspond to respectively an area of 23 and 2.3 square meters. In order to obtain the probability presented in Table 2, we subtract from 1 the probability that Pattern 2 holds. We obtain the probability that the fire is smaller than X percent of the total area for at least 90% of the time (over 10000s). Again, since for each value of X we ran a different set of simulations, it is not clear that the probability that the pattern holds increases when X increases. With this more permissive definition, we see that even small fires have a low probability to stay on for more that 10% of the simulation time. By comparing with Table 1, we can conclude that frequently occurring fires (i.e. very small ones) are quickly extinguished, because the probability of the last two lines are significantly higher in Table 2.
Finally, we evaluate the probability to obtain the unwanted emergent behavior depicted in Figure 13, that is the probability that Pattern 3 holds. The returned result is 0.9622, which means that the probability that the contract holds is between 0.8622 and 1 with a confidence of 99%.
We showed here how our tool chain is used to evaluate whether a given pattern holds. By evaluating the probability of Pattern 1, 2 and 3, we were able to discover that small fire occur often (last two lines of Table 1) but are not likely to last long (last two lines of Table 2). Finally, the emergent behavior occurs with a probability between 0 and 0.14, which explains why Pattern 1 and Pattern 2 do not occur with a probability of 1. This problem could be resolved by studying the causes of the emergent behavior and evolving the SoS to avoid it, for instance by adding more fire fighting cars.
At this level of analysis, a precision of 0.1 is sufficient to obtain a good general idea about the probability that each of the patterns occur. In general, using SMC requires to find the appropriate trade-off between the required precision and the time available for the analysis and subsequent re-engineering. Our Patterns are independent on the actual number of components. Indeed, adding constituent systems such as districts or cars, even if they have a new behavior, do not require specifying new patterns. The analysis is still possible on the modified SoS model.
In the framework of the DANSE project, Industrial Partners built models of their SoS under analysis. SMC and other methods provided them a higher confidence in their models [33]. More precisely, one Partner verified Mean Time Between Failures (safety) requirements in an Air Traffic Control case study. Another Partner verified sufficient water availability (robustness to failures) in a water distribution system of national scale.

Conclusion, Future work and related work
This paper proposes a full tool-chain for the rigorous design of Systems of Systems via formal reasoning and Statistical Model Checking.
Recent work promotes simulation techniques as the principal way to perform SoS analysis. In [34] the authors use discrete event specification (DEVS) concepts and tools to support virtual build and test of systems of systems. Their MS4-Me environment enables modeling and simulation (M&S) of SoS by allowing the user to specify constituent systems' behavior in terms of a so-called Constrained Natural Language. The tool is implemented in Eclipse and employs Xtext, Eclipse Modeling Framework and the Graphical Modeling Project.
Recent work in [35] provides an overview of the underlying theory, methods, and solutions in M&S of systems of systems, to better understand how modeling and simulation can support the Systems of Systems engineering process. However, simulation is an incomplete analysis and it is not able to assess the likelihood of the simulated behaviors. This is not acceptable from the point of view of SoS analysis, as it does not provide to the designer sufficient confidence of correctness. Other approaches to verification of complex systems are based on exhaustive formal analysis, such as model checking, or simulationbased formal analysis, such as run-time monitoring. Industrial model checking techniques [36] are not adequate to the complexity and dynamicity of SoS. Runtime monitoring does not seem to be adequate to the context of SoS, where failures detection should provide a likelihood estimate of the failure and sufficient time for devising failure-avoidance corrections. In this perspective, a very promising approach to provide sufficient coverage of the SoS behavior while keeping the analysis cost low is based on Statistical Model Checking (SMC) [4]. SMC is a simulation-based formal analysis providing an estimate of the likelihood of requirement satisfaction and a tunable level of confidence in the accuracy of analysis results.
Some frameworks, such as BIP [37] or Reo [38] allow the user to describe architecture of systems using alternative composition operators. Both these frameworks theoretically permit composition of heterogeneous system, by including existing code into the components. However, they provide no standard such as FMI/FMU allowing to compile a particular component independently of the architecture, which make them unsuitable for an industrial environment. They both include a stochastic extension [39,40], which is not part of the core framework.
Contracts for reasoning about heterogeneous systems are presented in [41]. However, these contracts are considered at a very abstract level, and thus are not as useful as FMI/FMU from an industrial point of view.
Future Work Our objective is to improve our solution by exploiting rare-event techniques [42] that would allow us to detect rare emergent behaviors with a minimal amount of simulations. Our second future work is to automatize the relationship between the outcome of SMC and the reconfiguration process, in order to automatically find an architecture satisfying sufficiently the GCSL contracts.