Towards a new software quality model for evaluation the quality of gamified systems

Due to the emergence of the Gamification concept in various domains, this article studies how to improve the quality of gamified systems with the software engineering discipline. Recent studies have shown a significant gap between design frameworks and gamified products, reflected in a lack of consistency, integrity, measurement and a comprehensive process that covers the different phases to achieve a quality gamified product. For this purpose, a specific quality model software for evaluation is required. Our contribution consists of making an analytical and qualitative study of the existing Gamification frameworks and platforms, with the aim of proposing a model for quality evaluation of the gamified applications. This proposal is based on the patterns and attributes determined during the conducted bibliographic study and comparative studies, with an attempt to integrate the ISO 25010 quality model.


Introduction
Launched a few years ago, the Gamification concept is now experiencing a new evolution and seems to be emerging as a strong and scalable trend in many domains, including marketing, business, training, and human resources management. This concept can be described as the use of game elements in a non-game field to enhance the engagement of the user's experience [1]. As an example, it can be used in marketing to attract customers and gain their loyalty, or to engage employees in the business field. Practically, all sectors of activity could benefit from Gamification as it can help achieve three main business objectives: 1) changing behavior; 2) developing skills; 3) Or enabling innovation [2].
This trend has upset many markets and offered new development opportunities. According to a recent study conducted by the Technavio research center, the market will exceed 6 billion dollars by 2019, with a growth rate of 48% since 2016. It is also noted that the US and Asian markets appear to be the most dynamic in the subject [3]. According to the report, one of the main drivers of market growth is the need to improve customer's interaction [4].
This "Gamification" has already been successfully adopted in several large organizations and had many positive results, including better long-term collaboration, creativity, productivity, loyalty and learning. [5] On the other hand, many software projects still fail today to deliver on time, with the expected costs and scope. One of the main reasons behind the failure of these projects is that they do not meet requirements, often caused by insufficient stakeholder collaboration, incomplete understanding of needs, insufficient knowledge and other [6]. 2 In 2014, Gartner predicted that 80% of Gamification initiatives in the company will fail, pointing out that bad design is the main cause of these failures; hence the importance of establishing a clear design strategy to successfully implement Gamification [2]. However, existing studies focus primarily on experiences in areas such as education, marketing, sales, energy and health. Little research has been initiated on the relationship between Gamification and software engineering discipline; even less have targeted the use of software engineering as a tool at the service of Gamification to design products of good quality. Therefore, this article will specifically focus on the quality of processes and framework of gamified software development, and on the contribution of the scientific software engineering discipline to a systematic and effective development of this type of application. This study focuses on the existing quality models in order to propose a new quality model specific to the Gamification concept, while responding to these different characteristics.

Related works
At this point, it can be argued that it is necessary to evaluate the current state of the art of quality models of the Gamification concept. It is important to determine the most consolidated body of knowledge and the most remarkable gaps to be filled. It seems that researchers have not shown the same interest in evaluating these qualities despite their growing interest in gamified software.
After analyzing the state of the art of the gamified software quality, we noted a remarkable lack of quality assessment tools for gamified products. Only few quality models take into account quality characteristics and attributes specific to gamified software, and more precisely the models supporting our proposals. Despite the wide scope, we have explored various well-known scientific databases including IEEE, ScienceDirect and Spring, only one attempt to customize the quality models for the Gamification frameworks has been identified; which is the QU-GamSoft model. The latter represents an adaptation of the Use Quality model (ISO, 2011b) that takes into account the specific quality attributes and characteristics of gamified software.
To fill this gap, we propose a customization of the ISO 25010 model, consisting of nearly the same data quality characteristics, to assess the quality levels of data used in Gamification projects. Unlike the QU-GamSoft model [28], our proposal is distinguished by: -A justification for the choice of the basic model after studying a set of existing quality models. -A multi-dimensional attributes mapping approach (conceptual, analytical and qualitative) -Proposal of a new quality model, richer in features and characteristics, to cover different aspects of a gamified software product, and to target more precise measures at the application level; unlike the QU-GamSoft model which is more interested in the use quality of a product regarding its users (satisfaction, risk, ...) [27] The proposed model can be integrated into any Gamification project because it is independent of any preconditions or technologies. A more detailed sheet of the proposed model specifics is to be presented throughout the article.

General Research Methodology
This article studies the integration of the software engineering discipline with Gamification in order to build a well-designed, more engaging and motivating gamified software. Our contribution is limited to studying the quality of the existing frameworks and gamified products towards proposing a new personalized quality model.
The research will be carried out qualitatively through reviewing the results of comparative studies of existing quality models, the characteristics of the ISO 25010 quality model, conceptual frameworks and experimental studies on gamified products. The following figure summarizes and lists the steps of our research: Step 1 : Definition of Gamification attributes This first step is to define the attributes for Gamification concept. Based on existing studies of different frameworks and Gamification platforms, we have extracted a set of patterns considered important to the evaluation of a gamified product.
Step 2: Choose a basic quality model In addition to the previously proposed attributes, we have chosen to reinforce our proposal with a software quality model. It is a question of choosing a quality reference EAI Endorsed Transactions on Creative Technologies 11 2017 -01 2018 | Volume 5 | Issue 14 | e2 model on which our attributes will be projected. The choice was mainly based on the recommendations of the comparative studies on the existing quality models for the software engineering domain, in addition to the following two main criteria: -The reference model must be rich in features in order to increase the possibility of covering the majority of the Gamification attributes and minimize the additions, modifications or adaptations to be brought.
-The model must cover the maximum of phases of the software lifecycle.

Step 3: Customize the ISO 25010 model for Gamification
Given the particularity of our field of study, "Gamification" and the "Software engineering" discipline, the quality model chosen as referent requires certain modifications, deletions or adaptations of its characteristics or sub-characteristics to respond to the context of study. This step aims to customize the ISO 25010 quality model following a mapping between its characteristics (or sub characteristics) and the Gamification attributes.

Gamification Attributes
The definition of the Gamification attributes required an analysis that includes both design frameworks and Gamification platforms. This scope was chosen in order to cover the characteristics of a product during the different phases of its life cycle. The analysis of the Gamification design frameworks allows a first vision of the main patterns that concern the phases of needs expression, design and needs analysis. This vision has been completed with a second study that is oriented to Gamification platforms, and that covers the other phases of testing, implementation and continuous improvement.

a) Gamification design patterns:
As mentioned before, we start with an extraction of the list of patterns from an analysis of the Gamification design frameworks. As shown in Figure 2, the following steps outline the approach used to obtain the list of Gamification design patterns: Step 1: Research and analysis of articles on Gamification design frameworks for pattern extraction related to the Gamification concept. The research was based mainly on: • The most well-known scientific databases in this field, including: IEEE Xplore Digital Library, Springer, Science Direct, ACM Digital Libray, Google Scholar, etc. The following search string: Gamifying (OR Gamification OR gamified) AND Framework (OR model, OR process, OR software, OR design). Eventually, we obtained 92 candidate articles for analysis.
Step 2: From the lists of articles obtained, we extracted the keywords explicitly specified by the authors and others following a diagonal reading of the whole document. 327 keywords were obtained.
Step 3: In this step, a first level of filter has been performed, consisting of deleting duplicates and classifying keywords according to several criteria, for example: -Application area: Learning, e-commerce, Health, ... -Life cycle: Design, Implementation, Improvement -Framework: Octalysis, Serious Game, user experience,.. -Psychology like: Fun, Self-determination, ... -Characteristics (patterns): Player types, Intrinsic motivation, Badge, game elements, measures, ... This step has allowed us to reduce the keywords list to 150 items.
Step 4: Based on the inclusion and exclusion criteria defined to meet the need of our study, 26 keywords are obtained (see figure 3). Below, more details about the criteria used in the filter are to be found: Inclusion Criteria: Recurring keywords that may represent one of the Gamification characteristics related to one or more phases of a product's life cycle. Exclusion Criteria: keywords that do not represent Gamification characteristics, especially those that specify the application domain, phycological reference theory, other frameworks (example: Octalysis, Serious Game, user experience , ...) This second section targets a recent study by Philipp Herzig that focuses on an analytical study of the Gamified products using new tools that were proposed and validated by 10 experts actively working in the field of gamification, particularly: To analyse these tools, Philipp Herzig [25] and al proposed the following evaluation tool: This study has shown the absence of a specialized tool to monitor and adapt gamification designs. Field experts have confirmed the use of customized solutions according to the need eventhough these solutions remain costly. Hence, having a common tool for the evaluation of the gamified software quality is needed. The next sections present our contribution to this study.

c) Gamification design and experimental attributes:
After studying the frameworks and platforms of Gamification, identified patterns have to be consolidated and classified in order to get the final list of Gamification attributes. This list will be used in the next steps to establish a characteristics and sub characteristics mapping of the quality model ISO 25010. More details of the obtained results are found below:

Choice of a reference quality model software: ISO 25010
To build a quality software, the use of quality models is necessary. According to ISO / IEC IS 9126-1 [5], a quality model is "the set of characteristics and relationships that serve as the basis for the specification of quality requirements and for evaluation". The main idea of these models is to decompose the quality of software into characteristics (representing the fundamental factors) and sub-characteristics (the sub-factors), while specifying the metrics corresponding to each of them. Following our literature review, we found numerous articles that compare the existing models, which we examined to select the most suitable for our study. The figure below summarizes the research recommendations in this area: Following these studies, the majority of the recommendations were oriented towards the ISO9126 model and its new version 25010, which justifies our choice of the latter. These conclusions have been drawn:  The ISO9126 model is a generic framework that allows users to develop their own criteria [12].  The ISO 9126-1 quality model is the most useful as it was built on the basis of an international consensus and agreement of all ISO member countries [12] [19] [20]  It contains the last standardized terminology [17]  The ISO-9126 model has received input from previous models and set standards to evaluate the quality of the software [17] [20]  It can be adapted to any type of software product and is oriented towards evaluation and improvement [17]  The ISO 25010 model is the most complete among the basic models, as it covers 26 of the 28 characteristics [17]  The ISO model seems to be more complete and with less defects than other models. The most important features of the ISO model are hierarchical structure, evaluation criteria, complete terms and expressions, simple and precise definitions, and a one-to-many relationships between the various layers of the model [26] As a result, the ISO model has been chosen as a base model and customized to become more suitable for evaluating a particular application domain, which is a gamified software application 6 New quality model for gamified software After presenting the results of the comparative studies of the software quality models and choosing the ISO 25010 (fig 4) model as a reference model, this section presents the different steps that were followed in the analysis and customization of the characteristics and subcharacteristics of this model. Following a detailed analysis of the description of each sub-characteristics of the ISO 25010 model (also called SQUARE); we classified them according to the following 3 statuses (table 2): Acquired '-': Corresponds to characteristics / subcharacteristics that have no particularity for a gamified product. Adopted '=': These are the sub-characteristics we have adopted without modifying the measure elements to judge the quality of gamified software. Modified 'x': Represents the state of the characteristics important to the evaluation of a gamified software, but requiring a modification of content or data...

EAI Endorsed Transactions on
Creative Technologies 11 2017 -01 2018 | Volume 5 | Issue 14 | e2 Modularity -Reusability -Analysability x Modifiability x Testability x Portability Adaptability -Installability -Replaceability -This list of characteristics has been revisited to give an oriented definition of the qualification of the gamification system.

Functional completeness
Generally, this must be translated by accounting for and defining the business objectives with a projection on the user's tasks, while taking into account the particularity of the player profiles. This also implies that a balancing is ensured between the user's objectives and business objectives, as well as the associated measures.

Functional correctness
Through the KPI application and indicators, the process must be able to provide the expected result with the expected precision.

Time Behavior
The aim is to ensure the rapidity and availability of information according to the type of users and their needs. According to Philipp Herzig [25], the relevant measures in this aspect are: -Time to Completion: The time period between the start of user existence and gamification element completion; -Time to Assignment: the time period between the start of user existence and its assignment to the gamification element.
-Time Active: the time period between assignment and completion of the gamification element. Generally, a user completing a mission faster than expected could be an indicator of the adjustments needed to design the mission.

Resource utilization
Availability Degree of tools needed for analysis, evaluation or other Usability :

Appropriateness recognizability
Considering the different player profiles in the system according to the particularities of each user.

Learnability
It concerns: -Degree of learning quality provided without risking user's satisfaction. -Degree of system recognition of the users' level and purpose in order to familiarize them with the application.
-The ability of the system to motivate and engage learners in the learning process in order to achieve the specified quality and efficiency objectives. 7 from the first experience to achieve the desired state, called «Win-state". According to Yu Kai in his Octalysis model: "The first Win-state happens when the user first says, "Wow! This is awesome!" [28] User interface aesthetics Extrinsic effect: Ergonomics Degree of the user's interface by implementing the game mechanisms (dashboard, level, points, ...) in the user's interface to ensure a pleasant and satisfactory user's interaction. Intrinsic effect: Degree to which the system allows users (or customer business vision) to quickly engage with the application objectives Accessibility Adaptation Degree of the framework to the user's characteristics and advanced level (mastery)

Analysability
This involves assessing the availability degree of the user's behavior analysis and evaluation, namely: -The availability of data and user's progress statistics in the dashboard, based on gamification elements such badges, levels or missions, ... -The application must issue events that inform the gaming solution analysis about relevant user's properties such as age, areas of interest, personal motivations, functions within an organization, which can help optimize a gamification design for specific target groups within final users. [25] -Tools Availability to monitor the achievement of business objectives, analysis of the gamification state, and adaptation of the gamification design in case of deviations or changes in the objectives.

Modifiability
The ability of the system to adapt to changes in order to improve the functionality of the gamified application as users experience it.

Testability
Degree of effectiveness and efficiency testing of criteria to assess user's actions, behaviors and achievement of business objectives. The testability is not limited to this aspect, it is also necessary to several parameters: -Particularty of user profiles, for example, Bartle's Four Player Types (Achievers, Explorers, Socializers, Killers) -Mastery level (Progress, Purpose, Relatedness, Mastery, Autonomy, ..) - The test phase (Discovery, Onboarding, Scaffolding and Endgame)

Conclusion and Future Work
This study focuses on the advancement of research in the gamification concept, especially for the software engineering field. We noted a gap between the process and the product. Indeed, the majority of theoretical frameworks are limited to the general aspects of design rather than other phases. In return, several gamified softwares are found but do not share their frameworks and development processes in full, which raises the need to produce a new framework with a process that covers all phases of a software lifecycle. The new model will be the result of a comparative study of existing models.
Our work stands as a preparatory phase of this comparative study. In this article, we have targeted the existing software quality models, among which we have selected the ISO 25010 quality model that seems the most suitable to our needs. This model has been revised to adapt to the Gamification concept.