Deploying serious games for management in higher education : lessons learned and good practices

Deployment of serious games (SGs) and their insertion in higher education (HE) curricula is still low. The literature lacks papers describing deployment of SGs in HE critically showing educational benefits and providing guidelines and good practices for their use. With the present work, we intend to make a first step in this direction, by reporting our experience in using state of the art managerial SGs in MSc engineering/business courses in four different European universities. In order to describe and analyse the educational characteristics and effectiveness of each game, we propose to use two models that we have straightforwardly extracted from two major pedagogical paradigms: the Bloom’s revised cognitive learning goals taxonomy and the Kolb’s experiential learning cycle. Based on our experience, we also propose a set of lessons and good practices to incentivize and better support deployment of SGs in HE courses


Introduction
While there is a certain consensus about the educational potential of Serious Games (SGs) (Bellotti et al., 2010;Greitzer et al., 2007;De Gloria et al, 2012) in higher education (HE), the deployment rate of SG in HE and their proper insertion in meaningful curricula are still quite low. This is generally attributed to an undefinedteacher's reluctance towards the use of games. However, there is also a lack of papers in literature describing deployment of SGs for HE in detail, critically showing their educational benefits and providing guidelines and practices on their use, in comparison with other educational tools/techniques.
With the present paper, we intend to make a first step in the direction of better characterization of the effectiveness and the use of SGs in HE, by reporting our experience in using managerial SGs at different European universities, namely: Genoa (Italy), Bremen (Germany), Nottingham (UK) and Open University of The Netherlands. In particular we describe the deployment of three games, selected because of their quality and ability to cover the course's managerial topics, that are being used in MSc courses in different engineering areas (civil, electronic and industrial). In order to describe and analyse the educational characteristics and effectiveness of each game, we propose the use of models that we have straightforwardly extracted from the major pedagogical paradigms.
Several pedagogical theories and learning models have been employed to inspire SG design and to assess validity of SGs. Among the knowledge models we highlight the Nonaka SECI model (Nonaka et al., 2000;Nonaka, 1994) is mentioned as a theoretical basis for the use of SG-based workshops, at least in the fields of business, management and manufacturing (Anghern and Maxwell, 2009), and Kirkpatrick's "The Four Levels of Learning Evaluation", which is a popular learning impact assessment model, involving the following levels: reaction, learning, behaviour, results (Kirkpatrick, 1998). A fifth level of evaluation has been added in new versions of the model by (Phillips, 2007) and by (Watkins et al., 1998), considering also return on investment and impact on clients and society, respectively.
In our work we have focused in particular on describing SGs through two models that we consider complementary, simple and particularly useful to analyse SGs: the Revised Bloom Taxonomy, which is the most popular cognitive approach to SG evaluation (Luccini et al., 2012) (Table 1); and the Kolb's Experiential Learning model (Figure 1), which systemizes the work rooted on Piaget's cognitive developmental genetic epistemology (Piaget, 1929), on Dewey's philosophical pragmatism (Dewey, 1933), and on Lewin's social psychology, putting the experience at the centre of the learning process.

Case studies
This section describes three case studies of serious games that the authors deployed in higher education contexts in four countries, namely The Netherlands, Italy, UK and Germany.

Estuarine systems: the Scheldt
The Scheldt is a web-based, role-playing, single-user game developed via the EMERGO methodology and toolkit (Nadolski et al. 2008). EMERGO-games are developed in such a way that the user-interface can be easily replaced without changing game-structure or game-content. Content resources are also separated from the game-structure. This enables easy maintenance and supports sustainability. The learning objective is to analyze, understand and explain the problem of the soilwater systems in the Scheldt (see Figure 2). This concerns a natural science approach towards the threats to our society, and complex spatial and temporal interactions between soil and water. During gameplay, the student takes the role of a junior researcher-trainee at a virtual company. He receives tasks and feedback from a senior researcher (embedded NPC) during the analysis of increasingly more complex problems and must propose/find workable solutions. He may use webbased tools / GIS sites, multi-various data and models to work towards his solution. This occurs by watching the phenomena and visually inspecting the area (e.g., video, satellite images, GIS sites) in order to solve the question on "Why is land reclamation or loss necessary from a scientific point of view?" Tasks and feedback are given via company-mail, or via video. Feedback can consist of completed examples and discussion. Students need to compare their own solution these; such as such tasks don't have unequivocal solutions. Then, based on an examples, a consecutive task will be given. In other tasks, feedback is given in a very natural way (for example, reactions from NPC's when consulted during task execution). The Scheldt (0,7 ECTS) has been embedded in a distance learning course on soil and water (4.3 ECTS) since 2010 at the Open University Netherlands (OUNL). There has been no needed revision of the game. The Scheldt is meant for independent self-study, so there are hardly any restrictions concerning the number of enrolled students. However, normally 30-50 students are enrolled every year. The central theme for the game case is "a field study focused on research and exploration of the Scheldt estuary towards relevance for naturalness, accessibility and security." This case concerns a step-wise approach towards the solution of the question "Why is land reclamation or loss necessary from a scientific point of view?" The case is highly realistic and centres on authentic tasks.

Support for Bloom's cognitive learning goals
Analyzing the EMERGO game, we can see that it supports several learning goals, as reported in the following table.

Support for soft skills
As anticipated, the game supports several soft-skills aspects such as problem solving, strategic thinking, meta-learning.

Any Business: a highly configurable online multiplayer business simulation
GoVenture Any Business (http://goventureanybusiness.com) is an instructor-customizable business simulation platform that can be used to simulate virtually any type of business, within any industry and any market. It is playable both individually and in teams. The game objective is to successfully manage a business while competing with other companies, managed by other players or by the computer. The Simulation Manager (usually an instructor) has a lot of freedom to configure the simulation, creating scenarios that can range from very simple to very complex/difficult, The Simulation Manager is also able to model specific events or situations to target specific learning goals.
The gameplay consists of making business decisions, which means setting several parametersprice, product features, marketing, human resources, business ethics, among othersbefore the deadline of each period of the simulation. After the deadline, the simulation advances to the next period and the game presents the results of the previous decisions in terms of sales and profits. A performance score is provided as a weighted sum of different dimensions, and the instructor receives a detailed report with all the activities performed by the students. Every simulation is different (e.g., economic and market conditions), which makes performance not perfectly comparable, but allows for more engaging challenges. Teams compete against each other in the same settings, as in a strategy game, and computer-driven competitors are also generated, creating a good model of the market. GoVenture Any Business is one of the serious games being used in the second edition of the course on "Entrepreneurship through Serious Games" (eSG) at the University of Genoa, Italy, for the Electronic Engineering M.Sc. degree. The course -which is presently in progress -aims to stimulate entrepreneurship in university students, especially future information technology engineers with little previous academic instruction in economics. The 3 ECTS course includes a series of lectures/workshops that introduce the theoretical foundations of entrepreneurship, discuss case studies and present the main features of the serious games that are used in the course. The games are played in teams as part of each week assignments, and in addition students are required to fill in questionnaires about the game and the concepts presented in the lectures. By the end of the course, students will have played a total of seven different simulations in GoVenture Any Business. The students' actual performance in the games is considered for the final course grade.

Support for Bloom's cognitive learning goals
Any Business shows a good capability for covering all the levels of the Bloom's taxonomy (Table 4).

Support for Kolb's learning stages
The authors do not know whether the game was designed on accordance with Kolb's cycle. However, it is quite well supported, as shown in the following table

Support for soft skills
The game supports mainly strategic thinking and decision making. In cases where the game is played in teams, it also supports interpersonal relations, as the decisions made must first be negotiated among all team members, who may also specialize and consider different aspects of company management (e.g., human resources, finance, etc.).

Seconds: a role playing game to improve decision making skills
The game Seconds, developed at the University of Bremen, is used to train students in decision making on supply chains (SC) and in distributed production environment. It is a facilitated multi-player, role-based, online game. The game creates a safe learning environment in which the students can apply different approaches for improving the flexibility and efficiency of manufacturing and analyse the impact on the SC. It is configurable, and the level depends on the knowledge level of the player (preconfigured). The goal is adaptable (depending on course setting), but is mostly used to produce a specific product in cooperation, while, taking all costs and expenditures into account. A simplified accounting system is implemented, i.e. the game delivers several performance indicators that are used for the analysis and calculation. The gaming scenario evolves as the players play the game. Depending on production volume and time, the player can gain experience and skills needed for producing higher quality. Target users are master students from industrial, production and system engineering and MSc logistics and operational management. The game is used at the University of Bremen as part of a 3 ECTS lab course on "Decision making in distributed production environment", which uses a blended learning concept. This part is comprised of 6 units-one for introduction to the basics of SCM and a tutorial on the gaming environment and five for playing. Methods for strategic decision making are successively introduced into the course. Each session lasts 5 hours. On average, the play time is 3-3,5 hours for each session and at least 30-45' for debriefing and reflection. For two years, Seconds has also been used at the University of Nottingham. There, it is used as a supplement to a post graduate course on Supply Chain management. Therefore, there is no introduction to strategic decision making in supply chain or in the basics of SCM, since this is knowledge already known to the students, i.e. the students come solely to play the game. So far, the students had have played twice. The sessions have taken around 2,5 hours with an additional 30 minutes for debriefing. In this case, the students received predefined scenarios with all company processes already implemented; i.e. the degree of freedom for taking decision on production sites etc. was lower than in the German case Support for Bloom's cognitive learning goals Actually, in the design phase of Seconds, Blooms taxonomy was hardly considered. Consequently only the higher levels are supported. This is typical for this type of game; it emphasizes on the two highest levels of evaluating and creating.

Support for Kolb's learning cycle
The use of Seconds uses an extension of Kolb's learning cycle: it uses the BIG (beyond the information given) defined by Perkins (1991, p. 20) BIG constructivism. Following the BIG approach, a facilitator directly introduces the concepts, provides examples to the students with concrete experience in activities that challenge them to apply, generalise and refine their initial understanding in multiple activities. This approach presents information to the learners but stresses the need to go beyond the information given.

Lessons learned and good practices
Quantitative results from the deployment of the games are not yet available from each game. However, the experience gained from the field allows us to make some considerations that we believe could be useful for educators.
In general, in order to guarantee successful deployment it is important to carefully align gaming goals with course goals and course assessment (i.e., constructive alignment). Deploying a new game is a complex and time-consuming activity that ideally requires the development of an ad-hoc deployment plan, specifying goals (educational and in-game) and context of use. Also student feedback should be carefully considered, in order to tune the game in terms of contents, difficulty levels, pace, etc. Finetuning the parameters for the games and deciding how much playing will be employed in a course can be a challenge. It is difficult to organize a sequence of game matches/sessions that continuously engage the students, while representing a proper educational path usable during a whole course, or part of it. Also, the number and duration of the sessions is relevant to the manner of interaction between students, especially regarding collaboration. For example, in the game Seconds, it was observed that the students' willingness to make compromises/trade-offs and to make strategic collaborations is higher, on average, in the groups having five sessions than those having two (thus showing a collaboration learning effect).
For competitive games, the teacher should thus support the weaker teams, in order to enhance the overall competitiveness, whereas for a collaborative game setting, it is more important that the game environment is able to support different competence levels within the same gaming scenario. In facilitated games, the facilitator's role is essential.
Documentation should be easily accessible online, in particular during the game and also the game developer support should be available, at least in the phases of course design and early deployment.
While the term SG is appealing, in particular for students, state of the art SGs have generally limited entertainment features, especially if compared with the best selling videogames. Tools like AnyBusiness are frequently referred to as a business simulations, without specific serious games mechanics (i.e., able to join fun and instruction). However, inter-team competition even through the simple mechanics of score and other performance indicatos (e.g., cashflow, profit and loss, etc.) are an excellent motivator.
SGs for complex scenarios should typically be used in blended learning settings, with briefing and debriefing sessions in order to complement and reflect on the experience, possibly also with questionnaires. Games designed to be facilitated should not be played in single mode or without facilitator, since intra-team relationships are very useful and the overview of an expert is very important both for the contents and for the game procedures themselves.
In addition to the time factor of each game session, another critical factor concerns the instructions given to the students before and during the game (both concerning the contents and the game itself). It is possible to allow the students to freely manage their roles. Howeverthis is a common rule in education -freedom should be limited for students having less knowledge about the target domain, in order to make the learning process more efficient and to help them overcome hurdles, concerning both the playability and the contents. Moreover, in-game knowledge (typically procedural and intuitive) should be complemented with other type of information, typically verbal and objective.
The facilitator or the teacher should pay attention at the students' learning outcomes after (and possibly also during) the game, in order to detect misconceptions, that are likely to appear, according to our experience, given the students' procedural and empirical approach.
A crucial step when preparing a course exploiting SGs is the actual choice of the games. The first step involves the collection of requirements related to the course and the curriculum. Addressed items include: target group, credits, learning objectives, which skills and competences should be trained, connection to the overall curriculum, underlying technical infrastructure, course setting, embedding with other learning material, use of blended learning concepts or not, number and length of units, feedback and assessment needs, pre-requisites (compare Nadolski, 2008).
The candidate games' features will need to be analysed in the light of the above mentioned requirements. Typical criteria for selection include various factors, such as: coverage of the needed educational topics; matching between the course's learning objectives and the game's features; costs (both in terms of software and of deployment and of maintenance); usability; quality of user assessment and provision of feedback (Bellotti et al., 2013); game adaptability; knowledge transferability; in house competencies and time availability in case development of a new serious game was considered; degree of freedom for players and teachers; support to collaboration; SG's learning curve, difficulty level and long-term playability; competences and effort needed on the teacher's side; availability of additional educational material related to the game. A main challenge in the selection process is the difficulty in having a critical and complete overview of existing games.
Depending on the weight of each one of the above criteria, existing games can be matched, and a "make or buy" decision may also be done. In the second case some of the collected requirements may need to be adapted. In the case studies presented in this paper, different needs led to different choices. In the first case (the Scheldt), the game had perfectly to comply with distance education and should partly serve as a replacement of fieldwork. Consequently, an ad-hoc development was deemed as necessary. The second case study (Any Business), a comparative analysis of several state of the art games was carried out, based on the main criteria of entrepreneurship topic coverage and per-license costs lower than €30. For the third case study (Seconds), a review was performed of the few available games on supply chain management and of several simulation environments as well. Since simulations were considered very good for mapping the real world processes, excessively difficult for the students, and the games not complex enough, in-house development was decided, allowing also the implementation of collaborative features, mirroring the work carried out in the production network.

Conclusions and Future Work
All the three studied games typically focus on the higher levels of the revised Bloom's taxonomy (analysing, evaluating and creating), in different ways and to different extents. The Scheldt emphasises the analysing level, whereas Any Business the evaluating level. Seconds focuses on supporting students to learn to create new knowledge. These differences substantially reflect the targets of the games and the corresponding courses.
Our observations (though still qualitative) highlight that this has a strong relationship with the role of the teacher/instructor/facilitator. The higher level to be achieved according to Bloom, the more emphasis has to be put on supporting the abstraction process, which typically requires the teacher's intervention. This process is only partly supported in the three games and related documentation, so it is very important that the teacher designs a proper pedagogical plan to make the game experience profitable from a real learning point of view. Also during the lectures, the presence of a teacher is very important, introducing and explaining topics and giving indications and discussing the experience with students. In Seconds, furthermore, the gameplay itself is accompanied by a teacher, while Any Business has been played at home by the competing teams and feedback has been provided by the teachers during the debriefing discussion. In The Scheldt, in-game feedback is provided by an NPC player, which aims at nurturing the thinking process, even if with a lower quality than through a human teacher. This game is made for an online university course, and thus it is also only offered in single user mode.
We do not agree that new education practices should turn the teacher from a "teaching machine" into a consultant nor a simple facilitator. We believe that SGs are powerful and complex tools that need the adult expertise and competences. Overall, the adult's presence is necessary for the educational role of leading students to knowledge and understanding of/access to reality. A proper use of SGs, instead of limiting the teacher's role, requires even better prepared teachers, and able to introduce students to aspects of reality by using a potentially powerful simulation tool. In the Any Business course, lectures involved the presence of three other researchers for supporting the official teacher by monitoring the teams' behaviours during the de-briefing (probably one would have been enough, but we preferred two given the experimental case).
Our analysis has shown that all the three games seem to correspond to the Kolb's learning model, even though the model had not been explicitly taken into consideration at the design level. Whereas The Scheldt emphasises concrete experience and reflective observation, the other two games, at least in the multi-player settings with debriefing, seem to be more focused on an abstract conceptualisation.
Regarding the effectiveness of the games, it can be reported that the students who had used The Scheldt received high grades on the final exam that was conducted after completing the game (average score 8.9 (maximum 10)). For Any Business, the preliminary results after four game sessions of one week each show that the game supports strategic thinking and requires and stimulates a deep understanding of the simulation environment. It can also be seen that the different reports generated by the game help in analysing and taking decision. Moreover, competition is really compelling for the majority of the players. The market conditions are determined by the abilities of the competing teams. Seconds has been used for decision making for more than 6 years. The available results mainly show that the game helps the students in applying methods and constructing new knowledge, and support strategically thinking.
Our experience in deploying the targeted games indicates some important reasons for the current low penetration rate of GBL in HE. The game that seems to be most easily to integrate is The Scheldt, for which only few adjustments were necessary. For the other courses that involved the presence of one or more teachers, it can be concluded that it is difficult to set up a course integrating the use of a SG, since the playing time, the length and plan of the sessions, the modalities for keeping the motivation etc. are difficult to estimate in advance. Thus, these courses often have to undergo an iterative design process, adapting the course set-up depending on the evaluation of the learning outcomes. This requires a continuously monitoring and a proper working experience. Taken into account the cost of this work, the experimental design, and the low number of students in each class it is still a question if the resources used for implementing GBL are efficiently used.
Despite the consensus on their potential (in particular due to the technological/graphic appeal, interactivity and the huge data processing/storing capabilities), the deployment rate of SG in HE is still quite low. We argue that this is due to the fact that games are more naturally suited to children than to adults. Moreover, educational effectiveness of games is easier to achieve with simpler content, while more complex and costly games are necessary in order to efficiently and effectively achieve the needed educational targets. Finally, integration in existing curricula is not straightforward and requires a careful pedagogical planning and a smart usage of games. So, we think that a lot of work is still to be done, in particular to understand how to use it and how to design its insertion in the course so that it is really effective for students.