Augmented Reality Meets Tangibility : A New Approach for Early Childhood Education

Augmented Reality (AR) has been recognised as one of the promising technologies for the gaming industry. In this study, the authors intend to apply AR technology to develop an interactive educational game. This paper presents an AR featured educational game specifically designed for 4-7 years old pre-school children. The principal objective of this game is to enable children to learn various abstract concepts, such as colour mixing, mathematics and 2D-3D geometrical shape recognition. This game allows users to interact with both onscreen (intangible) and physical objects (tangible) at the same time; different interaction forms including the touch screen (click) and AR game (rotate) are designed for better interaction with the real world and learning. This paper focuses on the details of the design and interactive behaviour. Furthermore, beyond the needs of children, this game also serves for parents through the Token Economy method; parents can control the kids’ contacting time with portable devices, and track and modify their everyday learning patterns. A pilot study implementing mix method was used to gather user’s feedback is also described in this paper.


Introduction
Different learning styles suit various stages during growing up.Before the age of seven, children learn from imitation and play [4].Playing is a requisite part of children's growing process [6], and games provide a fun way of learning important concepts and life skills [4].In the past, 'educational' was a negative word in the toy business, but that started to change over the last decades [15].According to studies, well-designed digital educational games can play a significant role in early childhood education.They perform more efficiently and effectively than traditional toys [3,10,17], and also enable 'playful learning' [1].Moreover, bridging interactive technology (e.g.Augmented Reality (AR)) with traditional toys can efficiently enhance the educational value of children's playing [6].One of the challenges in recent child education is to enable them to think on an abstract level; such sophisticated knowledge only can be transferred to children through integrated methods and tangible manipulations [12].For instance, AR can significantly enhance early childhood education by providing exciting experiences of 'playful', as it can integrate virtual objects and additional information to real objects.However, till recently, there are only limited successful cases to support serious learning activities, such as 2D-3D geometrical shapes, mathematic and colour mix.Most of the education-focused cases are focusing on either onscreen exercises or pure virtually interactive environments and objects.Also, none of them is focusing on bridging theory and abstract concepts with real world feedbacks.In the design of this AR game, the following aspects have been specifically targeted: 1. Game design principles based on Montessori education theory 2. Physical object interaction with screen 3. Learning abstract concepts The following sections describes details of this study: Section 2 illustrates the background through analysing the existing products in the market; Section 3 presents the design concept of the game, technological features and system structure; Section 4 explains the interactivity features in the game; section 5 uses the Mix method to test the pilot study; Section 6 gives detailed discussion and future work; and Section 7 presents conclusions of this project.

Background
Montessori theory is an educational approach, which emphasises independence, observation, prepared the environment and the absorbent mind [8].Enabling children to be independent and be able to do things by themselves are the goals of Montessori education, and this aim is achieved by facilitating a well-organised learning environment and providing opportunities for children to act [5].Therefore, various materials/ tools to support Montessori education are designed to enable auto-education and self-correction [11].For instance, Sensory box is a game for children in their early years who are not yet familiar with objects and the exterior environment.Sensory box is designed to broaden their learning scope and reduce the fear of unknown things.The game of Pouring Water not merely trains the ability of independence, but also helps to develop the ability to concentrate through facilitating the action of pouring water into a small bowl.Cylinder Blocks & Pink Tower provides different size and shape of blocks, which helps to refine visual-motor coordination and train positive and comparative capability.The advantages of these toys are low price and high quality of educational theme.
A combined method of tangible objects manipulation, virtual content recognition and imagination enhancement will provide an ideal learning experience [9,16].These can all be achieved by utilising AR technology to enhance 'playfulness' and 'abstract learnability'.The numerous educational benefits of AR technology include providing engaging and efficient learning, improving motivation and attention, developing imagination and cooperation, and enriching interaction [16].
For instance, the AR picture book provides solutions for positive AR learning performance [2].Wu et al. indicate that the AR environment can contribute to teaching mathematics and science, since abstract concepts can be converted to visualisation constructs [14].An Educational Magic Toy with different puzzles, flash cards and match cards was tested to teach objects, colours, numbers and shapes for 5-6 age children.The result showed positive effects of boosting educational value of AR learning [16].Similarly, a case study was launched in a kindergarten in Hong Kong [7].
Based on their research, AR-based learning offers a great fun experience and enhances teaching of abstract concepts, but teachers and parents have concerns about visual impairment and insufficient guidance.
To recognise the features of AR technology that have been applied in children's educational market, a series existing products have been analysed.Table 1

Design Rationale
Our design AR BLOCK is an AR-enabled construction toy kit based on Montessori education method.AR BLOCK aims at introducing four to seven years old children to mathematics, colour mixing and 2D-3D geometrical shape recognition, which can bring long-lasting benefits in their whole life [8,11].

Integration of Tangibility and AR
Comparing with existing products, AR BLOCK emphasises interactivity, playability and methods variations, which could enhance children's experience when exploring the game.This no longer requires a common AR game with display card information but works through manipulating tangible 3D objects.
The Montessori method considers touching as a good solution for bridging abstract concepts with physical objects.The ideology of this method considers a wide range physical interactive behaviours with an emphasis on physical objects manipulations to enable touching.Our system is designed with a number of tangible cards and blocks, to enable the following three teaching purposes.Teach children: 1. how to observe differences between 2D and 3D shapes through touching them 2. how to do mathematical calculations 3. how to recognise colours and understand the principle of colour-mixing.Overall, this system is based on a fundamental training principle, which is to transfer the physical sensory stimulations into abstract concepts and vice versa.For instance, the Colour Blocks provide an intuitive colour mixing process, touching two Colour Blocks can give a new colour; touching solid shapes is the key of teaching 3D geometrical shapes.

Design of Tangible Components
The AR BLOCK game consists of tangible construction blocks/cards and intangible application for tablets.The physical object component contains four primary functions:  At the current stage of the design process, timber was chosen to manufacture the tangible blocks for our original prototype.Timber comes from nature, and exposure to natural materials is suitable for children.This material can be preserved for a long time, and is widely used in the construction blocks toy market.Children are familiar with wooden toys.However, we also consider another material: silica gel, which is wildly used in medicine, toy business and other fields.Silica gel has many advantages, such as being non-toxic, with no pungent smell, and possessing chemical stability.In the future, we will continue to discover the differences between timber and silica gel, considering the characteristics of AR BLOCK to finally choose the better solution.For the software, Vuforia-an open source AR platform was used to develop the AR features, due to its cross-platform and game engines supporting features.
Comparing with other open source platforms, the functions of Vuforia come closest to our needs.For example, Vuforia can recognise and track objects, user-defined images or frame markers, which are ideal to use for customer products and game pieces.As shown in Table 2, blocks and cards have different markers' design.These markers will be printed on the timber, so it is very important to choose a safely printed material.Based on current research, we chose soybean oil ink.Soybean oil ink is an environment-friendly material.It has many advantages, such as a lower price than other materials, non-toxicity, recyclability, and is already widely used in children's products.Because of the restrictions on the size of Vuforia markers (>30 mm), the size of cards is 55 mm square and blocks have different types, such as 55 mm cube and 55 mm triangle.

Learning Modes
The intangible/onscreen component is designed for digital media devices, such as an iPad or Android tablet.AR BLOCK application has three modes: Parent Mode 2.
Family Mode As is well-known, long periods of using electronic equipment brings negative impacts to both the physical and mental health of the young user.According to usability tests, the playing time issue has been raised by most of parents.It is important for parents to supervise their kids' playing time.In Parent Mode, parents can choose different behaviour medals and each medal has three stars.Based on children's everyday behaviour, parents can provide stars for their kids; one star equals five minutes playing time.Better behaviour can earn longer playing time.This method is called the Token Economy, which is an excellent behaviour modification tool.Through Parent Mode service, parents not only can control kids playing time but also can track their everyday behaviour, in order to correct behaviour problems immediately.Children can play games in Kids' Mode or Family Mode.Kids' Mode is time-limited, and needs stars to unlock.Family Mode does not have time limits, but it needs parent's password to unlock.

Figure 1. System architecture of Kids' Mode and Family Mode
These functions have been achieved through implementing the Unity3D game engine and the Vuforia AR platform.Figure 1 illustrates the different interactive behaviours, feedbacks and system of Kids' Mode and Family Mode.If children have already been awarded playing stars in the Parent Mode, the Kids' Mode can be unlocked and they can play; otherwise, users have to return to the main page.When playing times out, users cannot continue to play the game.There are three buttons in the game interface.'Home' button directs the user back to the main page and 'Screenshot' button can save children's work to the tablet's gallery.Considering that self-directed learning is an important concept of the Montessori education method, a 'Help' button is facilitated to assist children learning independently.The 'Help' button is functioned with various tips, such as 2D and 3D geometric shapes, the results of mixing two colours, and how to learn mathematics.When the game starts, children will choose a Foundation Card first, and the screen will show a 3D frame of a house.Next, children should click the yellow number button beside the house on the screen.If users forget to click the button but choose another card directly, the system will present visual and audio feedbacks, in order to remind users to make the correct behaviour.This method can help children to focus on their existing work.After they finish the existing work, they can go to next.

Interactivity
The concept of interaction in this project divides into two sections: a visual and auditory interactivity, and an operational interaction aspect.For instance, in terms of parent's operation gesture, the main behaviour focuses on the screen.The click gesture allows choice of different functions and the swipe choice of different medals.Conversely, in the play environment, the users' primary behaviour will focus on the physical blocks/cards.
In most cases, the system provides feedback to users when they have operation behaviour.These visual or audio feedbacks are crucial for guiding the user's interactive behaviour [7].For example, suppose that after choosing a Foundation Card, instead of clicking the screen to choose the house, children just continue to choose another block.The system will show a visual feedback 'Please click the number to choose which building you want to change' and an audio feedback 'Oops'.If the user chooses a Building Card with a shape different from the Foundation Card, the user will see a message 'Please choose the right card' and hear an audio feedback 'Wrong card'.
When the user interacts with the application, Click is the most commonly used operation gesture.However, in the game environment, Rotate will become to the main gesture.Through rotating blocks, the tablet camera will recognise different markers and the screen will show different feedbacks.

Pilot Study
The pilot study of AR BLOCK was tested in the International Tangible Interaction Design Lab (ITIDLab) at Monash University, and involved five volunteer parents.All of the parents have 4-6 years old children and have experience with using tablets or smartphones, but none of them has any experience with using AR application.As a design-based research, this pilot study mainly focuses on products function and interactive methods.
Parents were required to operate APP and tangible toys by themselves without any training.After completing the test, surveys and short interviews were given, based on their experiences of using AR BLOCK.The whole process of users' test was video recorded.

Data collection
Data were collected from multiple sources (video observation, interviews with parents and questionnaires).
Video observation was used to explore users' operation flow and any weaknesses of the game design.Based on the interview, we measured five principal features, which were scored (1-10) by users.Figure 5 presents the results of users' experiences with: educational theme, affordances, playful exploration, instruction (for parents/children) and meaning of markers design.The questionnaire is given in Table 3.
Table 3. Questionnaire protocol 1. Are you in favour of AR BLOCK with your child at home?Why? 2. What pros/cons that your can think of AR BLOCK? 3. Do you think it is hard to play for your child? 4. Do you have the budget to support AR BLOCK?

Results
The video observation indicates that all of the parents would start from Parent Mode.During their exploring, there are two common problems of Parent Mode design.Firstly, the registration process is different from normal application.Users were only required to input password and click 'Register' button in order to reduce unnecessary registration information.However, subjects always directly pressed 'Register' button and waited for a pop-up window or feedbacks without input password, because the system only contains text feedbacks, which are routinely ignored by users (Figure 3).Secondly, after clicking 'Save' star button, subjects were confused because the   Concerning interviews and questionnaires, parents have expressed that AR BLOCK provides a great fun learning environment, and showed a positive attitude towards Parent Mode since it is a workable solution to control the screen time.Moreover, based on parents' answer to question 4 (Table 3) and the average score of affordances of 7.3 (Figure 5), parents appear willing to spend money on AR BLOCK.According to educational theme and playful explorations features' average scores, which are 7.6 and 7 respectively, parents widely believed that AR BLOCK creates a fun and interactive learning environment and contains powerful pedagogical factors.However, as described in video observation result, subjects were confused by some design problems in Parent Mode; therefore the average score of instruction (for parents) is 6.6.The participants also discussed disadvantages of AR BLOCK.First, they worried that there was insufficient guidance to direct children when they were playing.AR BLOCK system had image tutorial pages in Parent Mode and Kids' Mode, but even for parents, it is difficult to understand the game rule when they play for the first time.So the average score of instruction (for children) is 4.6.Second, there are some limitations of markers design.As described in Table 2, different cards and blocks have different markers design.Subjects considered that the design of Number Blocks and Colour Blocks were reasonable.But parents expressed their concerns about markers design of Foundation Cards and Building Cards.Children should compare the shape of Building Cards with Foundation Cards, only choosing the same geometrical shapes to go to next step.The markers design meaning of existing prototype is unclear; therefore, the average score was 4.3.

Discussion and future work
This paper provides an overview of the design of AR BLOCK; we started our study with a survey of existing AR products and discovered weaknesses with them.After the initial users' test, our prototype was again tested in a MADA exhibition in 2015 (Figure 6).In the exhibition, researchers, educators and parents trialled AR BLOCK.They provided different comments based on their experiences.Through observation and conversation with users, some features have been modified.For example, most of the users consider that the interaction behaviour of AR BLOCK is very fascinating.Their playing method is no longer limited to the screen but returns to the original operation behaviour.Instead of Drag, Click or Swipe, they can assemble physical blocks, exactly the same as with traditional construction block toy's operation mode.Moreover, rotating Colour Blocks to change colours provide an intuitive and efficient method for learning colour mix knowledge.However, the prototype is lacking a better way of guidance as well as previous users' test results.The majority of users do not know how to start the game when they play for the first time.We designed a tutorial page in Parent Mode; we hope that after parents understand how to play, they can teach their children by themselves.In fact, children still do not know the game rules.From the surveys and users' tests, the prototype is different from other existing products in interactive operation methods.AR BLOCK pays more attention to the interaction of physical objects with the screen.In terms of learning, AR BLOCK provides a unique colour mix method.Users can choose different colour blocks and rotate any of them, which will produce a new colour.Children can learn each colour's name and observe the result when two colours are combined.

Figure 6. 2015 MADA exhibition test
According to outcomes of two users' tests, an improved tutorial function is designed in another project, which is an AR educational training approach for autistic children.As shown in Figure 7, the design of tangible components contains different shapes cards, which can assemble together as well as puzzle pieces.And same functional cards have their special colours in order to help children to identify various functions of cards.An animated tutorial page is provided to users.The system will show different function cards on the screen one by one, and users should choose the correct cards shown on screen in order to proceed with the tutorial.The new tutorial function was tested with same participants.All the subjects indicated that the new tutorial method was a huge improvement compared to the previous one, and the average score of instruction (for children) increased from 4.6 to 7.5.

Figure 7. The physical components design of an AR educational training approach project for Autism children
As the next step, based on previous outcomes, the future development of AR BLOCK will focus on building a more effective guidance system, in order to provide a better user experience.For example, in the playing environment, instead of 'Tutorial' function page, using a cartoon character guides users' game process and provides visual/audio feedback of users operation.Moreover, the interface design of existing Foundation cards and Building cards can easily mislead users; they do not know how to pair the right card.We should redesign them and display information more efficiently.

Conclusions
To sum up, the main aim of this project is to provide a solution to enable children to learn abstract concept through interacting with tangible blocks and intangible inscreen elements.According to Montessori theory, blocks and cards of AR BLOCK can transfer abstract information to children, such as addition, 2D or 3D geometrical shapes and colour mixing.Instead of in-screen exercises, children can assemble or rotate these blocks, and the screen will give feedback based on their operation gestures.AR BLOCK can provide a better interactive experience when children are playing and learning.
Through playing with distinctive blocks or cards, children can learn different concepts.For example, when children observe Foundation Cards, they can only see 2D geometrical shapes, but the screen will show the 3D geometrical shapes.These Foundation Cards can help children with 2D and 3D conversion ability.Children can learn mathematics from Number Blocks.Number Blocks have different shapes and figures; if the Foundation Card shows a house frame with square and triangle shapes and number 8, the user should choose a cube and a triangle Number Block.Then they calculate numbers on the cube block and triangle block, letting the result equal 8.Moreover, children can exercise their matching ability through comparing Building Cards with Foundation Cards.The same geometrical shape can complete the house.Lastly, each Colour Block has three colours: yellow, red and blue.Children can rotate Colour Blocks to change the colour of a building and two Colour Blocks colour can mix to give a new colour.

EAI
Endorsed Transactions on Creative Technologies 09 2017 -09 2017 | Volume 4 | Issue 11 | e2 For instance, if the user wants to change the house colour and learn colour mix knowledge, the user can use Colour Blocks.If the camera recognised No. 1 Colour Block marker as yellow and No. 2 Colour Block marker as red, the house's colour would show as orange.If the user rotated No. 2 Colour Block and changed the marker to blue, the house colour will change to green.

Figure 2 .
Figure 2. Rotating Colour Blocks to change a building's colour

Figure 3 .
Figure 3. Registration page of Parent Mode

Figure 4 .
Figure 4. 'Save' star button of Parent Mode

Figure 5 .
Figure 5. Users' experience with 5 main features

Table 1 .
The existing products analysis

Table 2 .
Different tangible blocks or cards design table • Learn colour mix • Combine two colours • Create a new colour