Northumbria

Few modelling studies have been carried out to investigate patients’ involvement in the decision-making process in a healthcare system. Here we perform theoretical and simulation analysis of a healthcare business model involving three populations: Public Healthcare Providers , Private Healthcare Providers and Patients . The analysis contributes to healthcare economic modelling by analyzing the dynamics and emergence of cooperative behavior of agents within the three populations. Resorting to agent-based simulations, we examine the effect of increasing behavioural mutation and providers’ capacity on patients’ cooperative behaviour. We show that the former introduces more randomness in agents’ behaviors enabling cooperation to emerge in more difﬁcult conditions. Moreover, when the providers’ capacity to meet patients’ demand is limited, patients exhibit low levels of cooperation, implying a more difﬁcult cooperation dilemma in a healthcare system that needs addressing.


Introduction
The healthcare system in the UK is regulated by the Department of Health where explicit rules, activities and mechanisms are imposed and enforced by the state to manage social behavior [21]. From the perspective of UK health policy, the tendency is to understand an incentive as an economic inducement or benefit, whether tangible or intangible, resulting from a behavior that impacts the performance of the healthcare system, whether positively or negatively. This narrow definition tends to exclude non-monetary incentives or benefits [10,21]. This issue can be solved pragmatically through the introduction of Evolutionary Game Theory (EGT) modelling [23], with various similar examples such as the Public Goods Game (PGG) and different forms of incentives (both monetary and non-monetary such as reputation-based ones) [18,22,24].
In this research, we investigate the evolution of cooperation in a complex system, namely the healthcare system in England, which is made up of populations * Corresponding author. Email: t.han@tees.ac.uk consisting of different healthcare providers interacting with patients. Our investigation examines individual agents' behaviors as viewed by an external policymaker, in this case, the Health Department. Generally, policies are initiated and managed by the Health Department, which allocates a specific budget to interfere.
To this end, our analysis here is carried out based on a baseline model we developed in [2]. This previous work focuses on numerical analysis of the proposed healthcare model, studying evolutionary dynamics of three well-mixed finite populations. The analytical approach we adopted therein relied on the assumptions of rare behavioral mutation by the agents, and that all populations having an equal size. Our finding shows that agents from all three populations tend to not cooperate (i.e. defect) [2]. While these simplified assumptions allowed us, as a very first step, to provide clear mathematical analysis, they prevented us from analysing some important factors. Namely, mutation or behavioral exploration, where agents can freely experiment with new behaviors, has been shown to play an important role in enabling cooperation in the context of social dilemmas [3,6,11,20,26]. As the world is facing a rapid increase in populations, most healthcare capacity is struggling to cope with attendant mounting demand and to accommodate growing populations' needs. Moreover, some scholars have warned that, in reality, healthcare providers might not be able to meet the treatment demand from all patients [8]. It is, therefore, viable to investigate this intensifying trend and study the ratio of population to hospital bed availability. Currently in England, UK, the number stands at 474 patients per hospital bed 1 [19]. Therefore, in this paper, we develop agent-based simulations of this baseline model to investigate how relaxing the above-mentioned assumptions affects the cooperation outcomes in providers' and patients' populations. Namely, we seek to develop a more insightful understanding of the dynamical behavior of agents in each population, the effects of applying a large mutation/exploration rate on the abundance of cooperation, and the influence of reduced sizes of the healthcare providers' population on the agent's behavior within the dynamic system and their population.
The rest of this paper is structured as follows. Related Work section reviews the most relevant literature. Model and Methods section presents our healthcare model and methods applied. Results section discusses agent-based simulation results. The conclusion summarises our findings and explores their implications for future work.

Related Work
Researchers eager to understand the behavior of different agent representations within the healthcare system use AI [30], game theory [1,28], multiagents system [5] and big data [16] which are used to predict and understand behaviors within systems. However, little effort has been made to study the dynamics of cooperation and other decision making in the healthcare domain, which usually involves different actors in the decision making processes. Our previous work [2] and this work aim to bridge this important gap by resorting to population-based methods from EGT to develop an understanding of the dynamics of cooperative behaviour in this domain.
The rapid development in research on the learning of social behavior has significantly increased our understanding of the dynamic interaction among individuals from different populations [23]. Cooperation is one of the fundamental aspects to measure the strength and dynamism of a population [7,14,25]. It can be studied by applying EGT to different mechanisms, such as reciprocal behaviors, kin selection and costly incentives [13,23]. EGT has 1 England population number was obtained from Office for National Statistic ONS (correct as of mid-2019), available at:https://bit.ly/2MCb3yc. The number of hospital beds in England (correct as of 2018/9) was calculated based on data provided by the King's fund report "NHS hospital bed numbers: past, present, future": https://bit.ly/2UfgeZ0; (accessed 6 June 2020).

Parameters' description Symbol
Reputation benefit for the Public and Private healthcare providers

Healthcare Model
In this section, we summarise the model presented in [2], where we consider three populations: Public providers P1, Private providers P2 and Patients P3. While P1 represents the NHS or the public healthcare providers, P2 represents independent healthcare providers P2 selling healthcare services, and P3 represents those who seek personal treatments. Agents from each population P1, P2 and P3 can choose from two strategies: provide/accept sustainable treatment identified as cooperating, otherwise the game will be dominated by alternative treatments from other providers. On each game encounter or iteration, an agent's payoff is acquired based on the strategy played by each agent/individual from the three populations. In this game, an interaction happens among three agents, one from each of the three populations, see Table 1.
An agent in each of the three populations has two strategies, cooperate (C) and defect (D). Here the strategy coincides with the action of an agent, despite the fact that there is only one decision to make, which can be interpreted 2 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 05 2020 -10 2020 | Volume 7 | Issue 24 | e4 Toward Understanding the Interplay between Public and Private Healthcare Providers and Patients: An Agent-based Simulation Approach as follows. Public healthcare providers: • C: offers treatment paid for from taxpayers' money, which consists of paying a cost c I , and in return gets a reputation benefit.
• D: does not want to pay for the treatment.
Private healthcare providers: • C: offers treatment either paid by Public (when Public cooperates) or self-paid by Patient (so the main cost involved is management cost c M ), and obtains a reputation benefit b R . In case of cooperation with the Patient, Private commits to investing c I from its revenue.
• D: does not want to offer the treatment. Patient: • C: accepts being treated and pays for the treatment c T when treated by the Private; Patient obtains health benefit b P if at least one provider cooperates and extra health benefit when both providers cooperate ε.
• D: rejects the treatment and looks for alternative treatment mostly overseas, where above conditions are correct, and: b P > 0, ε > 0 and c T > 0.
Thus, given the strategies of players in each population, there are 8 possible strategic combinations or scenarios, represented by XYZ where X, Y, Z can be either C or D. We summarize a few of them below (full details can be found in [2]) • CCC: individuals from all three populations choose to cooperate. The Public pays for the treatment provided by a Private healthcare provider and the Patient accepts the provided treatment in pursuit of her/his own benefit or well-being. The Public, which bears the costs of investment and pays for the Patient's treatment (c I + c T ), covered from its allocated budget, gains a reputation benefit b R . Reputation benefits are derived from the Patient's satisfaction with the provided service. On the other hand, the Private healthcare provider will provide the required treatment to the Patient and receive the payment covering the costs from the Public. The cost of investment c I of the Private healthcare provider is to invest in staffing and healthcare facilities, while c M refers to administrative and operational costs. Also, the Private provider will obtain a reputation benefit with the Public b R . Patient gets extra health benefit εb P as both healthcare providers are cooperating (ε represents a synergistic factor). • DDD: this scenario consists of all three agents choosing not to interact with one another, thus all the agents get a zero payoff.

Method: Evolutionary Dynamics for Three Populations
EGT method is adopted to study the evolutionary dynamics and interactions among individuals from three distinct finite populations: P1, P2 and P3. The populations are of fixed sizes N 1 , N 2 and N 3 , respectively. In each time step, from a randomly selected population, a randomly selected individual A with fitness f A imitates another randomly selected individual B with f B fitness using the pairwise comparison rule, a popular and standard approach to implementing social learning in EGT [23,27]. Namely, the probability ρ that A adopts B's strategy is given by the Fermi's function where β > 0 represents the 'intensity of selection' or 'imitation strength'; β = 0 represents neutral drift where imitation decision is random, while for β → ∞ the imitation decision is increasingly deterministic. In our results, we set β = 0.1, which is in accordance with previous theoretical and experimental works [20,29]. When mutation or behavioral exploration [6,9,20] is not rare, we consider that, before A considers to imitate B, with a probability µ, it changes to a randomly selected strategy from its behavioural space (in this case, C or D). That is, with probability (1 − µ) A considers to imitate B as above.
Individuals in each population have the choice to play C or D. As described above, there are eight possible scenarios corresponding to the the eight possible combinations of the strategies within the three populations: CCC, CCD, CDC, CDD, DCC, DCD, DDC, DDD. Denoting the numbers of cooperators in P1, P2 and P3 by x, y, and z, respectively, the payoff of each strategy can be written as follows:  where P xyz is the payoff for the strategy selected by individuals from one of the stated populations, and (x,y,z) represents the selected strategies C or D. For instance, individuals from P1 have the options to play C or D. The selected strategy will replace the s at x vertex, while y and z vertices remain unchanged for selected strategies for the Public population in this instance.

Agent-based Simulations
As mutation is not rare, we perform agent-based simulations to study the evolutionary dynamics. Initially, agents in the three populations are given a random strategy (C or D). In each generation or time step, agents' fitness is calculated as in Table. 1. Namely, the fitness of an individual adopting a strategy s within a population is derived from the average obtained from the tripartite oneshot game described in Table 1. A randomly selected individual in each game obtains an average payoff given by [7,22]: where f P S (x, y, z) represents agents' average payoff in a population P while adopting strategy s, assuming that the population is present in the vertices (x,y,z). P S ABC denotes the payoff that an individual playing in a group with a strategy profile derives from state S where public plays A, private plays B and patient plays C . These payoffs are shown in Table 1.
Each simulation of the stochastic modelling runs for a number of generations (at least 40,000), where a stable state is reached in general (see examples in Fig. 2). To ensure accuracy, the accumulated results are then averaged over the last 100 time steps, and furthermore, for each parameter configuration, the results were averaged over 50 independent realisations. As mutation is adopted in the simulations, no absorbed monomorphic state is reached .

Results
In this study we conduct agent-based simulations (see Methods), where agents from the three populations P1, P2 and P3 interact in a one-shot game and learn how they influence the level of cooperation to sustain cost-effective services and better patient satisfaction. Figure 1 shows the cooperation levels adopted in the populations for varying the sectors' benefit b R and patient's 4 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 05 2020 -10 2020 | Volume 7 | Issue 24 | e4 Toward Understanding the Interplay between Public and Private Healthcare Providers and Patients: An Agent-based Simulation Approach benefit b P . In general, cooperation in all populations is most abundant when both benefits are sufficiently large. The two sectors have similar cooperation for the whole parameter space, except when both benefits are small, where the private sector has a slightly higher level of cooperation.
To better understand the detailed dynamics, in Figure  2 we show the evolution of cooperation over time in each population for different combinations of the benefits. Indeed, we can see that when b R is large (see b R = 6, bottom row), all the three populations quickly converge to full cooperation even for small b P . For intermediate b R (see b R = 1, middle row), P1 converges to defection, while P2 still maintains some level of cooperation and P3 have high levels of cooperation when b P is large (i.e. equals 3). When b R is small, all populations converge to defection. That is, P2 population is more willing to cooperate with P3 for quite low b P (where b P ≤ 1 and b R ≤ 1). On the other hand, P3 requires better reputation benefit b R to cooperate with P3 and provides the desired services. As a consequence, P3 has to pay extra cost equivalent to c T as represented in Table 1, i.e: DCC for the Patient's payoff (b P − c T ) compared to the other scenarios, CCC or CDC, where the cost equals zero.

The effect of mutation
We now study how increasing the mutation rate µ affects the cooperation outcome in each population. Note that so far we consider rare mutation or low mutation rate µ = 10 −4 , recovering results from our previous work using small mutation dynamics [2]. Fig. 4 shows results for higher mutation rates, namely, µ = 10 −k , with k = 1.5, 1 and 0.5. As expected, as larger mutation leads to greater levels of randomness in agents' behaviors, we observe more defection in scenarios where cooperation is abundant (high benefits), and vice versa, i.e. more cooperation in  previous scenarios with abundant defections. Interestingly, for intermediate µ (µ = 10 −1 ), P1's cooperation mostly depends on P3's benefit, while P2 is more dependent on the reputation benefit. This observation can also be seen in Fig. 3, where we plot the frequency of CCC. For sufficiently large µ, the frequency of CCC always converges to 1/8 regardless of the values of the benefits, which is when all populations converge to 50% cooperation.

The effect of unequal population sizes
In reality, the availability of hospital beds is very limited for the increasing number of patients; i.e: winter long waiting time in A&E or in a pandemic (e.g. as is the case with the COVID-19). We study the effect of this factor by looking at different providers' population size ratios to that of the Patient's population. Fig. 5 shows simulation results for two different providers' sizes, 10 & 5. The results reveal that P2 cooperates more with lower providers' and patients' benefits. Whereas the P1 population still prefers to cooperate with higher reputation benefit b R . Conversely, P3 shows a significantly different behavior in the decision making process by defecting upon low health benefit 0.1 < b P < 1 and willing to cooperate when b P < 0.1 and b P ≥ 1 for larger providers' population sizes.
These observations show that, in a more realistic setting where both P1 and P2 usually have a lower capacity than what is required by P3, it is more difficult to ensure 5 EAI Endorsed Transactions on Industrial Networks and Intelligent Systems 05 2020 -10 2020 | Volume 7 | Issue 24 | e4 Figure 5. Cooperation levels in each population based on the ratio of hospital bed availability provided by healthcare providers compared to the size of the P3 population(N 3 =100). Where the size of the P1,P2 is: first row N 1 = N 2 = 10 and second row N 1 = N 2 = 5. Other parameters as in Figure 1.
cooperation from the patients. That is, the cooperation issue is more severe, which therefore requires additional supporting mechanisms such as incentives to maintain cooperation. We are currently studying this important setting.

Conclusions
We have studied cooperation outcomes and evolutionary dynamics of a three-population healthcare model using extensive agent-based simulations. We analyze cooperation outcomes for incremental rate of mutation, and compare them with those when it is rare, as reported in our previous work. This result can represent realistic scenarios derived from some elective treatments such as hip replacements [15]. To link our model to realistic scenarios it will make use of data published by the NHS and complemented with figures obtained from the Patient Reports Outcome Measures [17] and the Care Quality Commission [4].
We found abundance of cooperation closely linked with high benefits. Interestingly, for sufficiently high mutation rates, more cooperation also emerged in scenarios that are previously dominated by defection in all populations.
We explored the effect of reducing the size of the healthcare provider population on cooperation between the three populations. We found that the Patient is less likely to cooperate (small abundance) when the Providers' populations are small, which means that providers have a limited capacity to accommodate the patients' needs. This finding reveals a critical cooperation problem that needs to be resolved. This is an important issue since in reality providers might not always be able to meet patients' demand, especially when health crises occur such as in the time of a pandemic (such as . Again, our findings reiterate the important role of patients in healthcare decision making processes, which has been pointed out in previous reports [17] but usually omitted in modelling works.
As future work, we plan to explore new mechanisms to improve cooperation in this important and realistic setting, such as via positive or negative incentives in the form of peer and institutional rewards and punishments [2,12,24]. Moreover, in this paper, as a first step, we assumed that providers have the same capacity. However, in reality, the public and private sectors may have very different capacities, which might depend on different regions and their local demands. We plan to provide a more comprehensive analysis of how difference in providers' capacities, in relation to the patients' demand [19].