Potential for Augmented Reality in Education : An Overview

_______________________________ *Corresponding author. Email address: shengbin@cs.sjtu.edu.cn


Introduction
People have already recognized the great potential value of AR technology through its excellent performances in various fields including education.According to the Horizon Reports of year 2010 and 2011, Augmented Reality will see widespread application on US college campuses within few years [1,2].As never before, through AR technology, learners will have a chance to get access to a wide range of information from a variety of sources immediately [3].This new approach enhances the effectiveness and attractiveness of education and changes the way we acquire, comprehend and even interact with knowledge [4].For example [2], the lecturers may engage and motivate students to explore learning materials from different aspects [5].Several subjects where students could not feasibly gain real-world first-hand experience could be conveyed through AR approaches [6].Collaborative relationships among students could be strengthened by AR techniques [7].Meanwhile, the creativity and imagination capability of the students could be fostered if AR teaching mode is employed in their education [8].Based on Augmented Reality, the learning progress could be controlled by the students themselves [9].There are still many aspects that could benefit from the utilization of AR for the teachers and learners.This paper contributes a comprehensive survey of AR in education domain.Our objective is to provide a useful insight into the state-of-art AR technologies and applications for instructors and learners.This paper is organized in five separate components.In part 2, we introduce a brief definition, a historical review and a taxonomy of AR.In part 3 and 4, AR applications and key technologies that have been developed are presented respectively.We conclude our survey with a prediction of the educational AR futures.

Starting out: A Definition
Originally, the Augmented Reality was defined with respect to the display technologies like head mounted displays since the display devices played a vital role in an AR system at that time.This definition is similar to the restricted one proposed by Milgram [10].However, this kind of definition limit Augmented Reality a narrow sense of sight.Afterwards, Azuma [11] defined Augmented Reality as any system that has three characteristics required: 1) combination of the real and virtual objects, 2) interaction with the environment in Table 1.A brief historical review of Augmented Reality real time, and 3) registration of three dimensions.This definition put more emphasis on the integration of the real and virtual world and the interaction with human beings.Similarly, Wu et al. argue that viewing Augmented Reality as a concept rather than a type of technology would be more productive for educators, researchers, and designers [22].Taking these characters into consideration, we view the Augmented Reality as an emerging form of experience in which the users are served with a composite view of the real-world physical scenes and computer-generated virtual scenes, without replacing the real environment being experienced.Synthetic virtual elements are superimposed upon the real world with which users may interact.

Historical Review
More than four decades ago, Ivan Suntherland, a pioneer in the computer graphics fields, devised a head-mounted three dimensional display [23] which has been regarded as the first prototype of a see-through head-mounted display (HMD) of Augmented Reality (AR).Inspired by Suntherland, an increasing number of AR prototypes have been brought forward during the past forty years of development in this field.To some extent, with the help of these AR techniques, our lives become more colorful, and our ability to acquire and process information has been enhanced.We summarize the brief developing history of AR technology in Table 1.

Further Study: A Taxonomy
The terminology "Augmented Reality" is well categorized in the virtuality continuum taxonomic model, firstly proposed by Milgram et al. [24].virtuality continuum "connects completely real environment to completely virtual ones", as the authors claimed.A general schematic view is depicted in Figure 1.In this taxonomy, the Augmented Reality is part of the Mixed Reality which involves the mergence of virtual and real environments.The real environment is the world that we can be perceived physically using our senses.Everything we experienced is part of our physical reality.On the opposite side is the virtual environment which is a

Educational AR Applications
Due to great efforts by researchers, AR technologies have been applied in a variety of fields.In 1997, Azuma summarized a few typical applications in his survey [11].These applications belonged to six principal areas including medical, manufacturing and repair, annotation and visualization, robot path planning, entertainment and military aircraft.While it appears that a few areas are not scientifically classified, this summary still cast a good direction for further development.Ludwig and Reimann classified these potential applications into three major categories [25]: 1) presentation and visualization, 2) industry, and 3) edutainment.More up-to-date applications have been produced along with the development of new AR techniques.In their supplementary survey [26], Azuma et al. grouped the new applications into three areas which are mobile, collaborative and commercial applications.Krevelen and Poelman [27] added several rising domains like navigation, touring, training and office applications.The advancement in these areas have already changed our lives greatly and are believed to continue making a difference to the future world.However, of all these areas, the exertion of AR technology in education seems to be the most meaningful one and currently, developers have created many AR applications for educational purposes such as reading, location-based learning, objects modeling, and skills training.In this paper, we briefly introduce several innovative representatives of the educational AR applications.

AR Reading
MagicBook [28] is a good example for the AR techniques to be used in the primary education.By blending reading and virtual reality, the MagicBook enables children to see 3D virtual models appearing out of the book pages from any perspectives by moving themselves or the book.Another useful reading tool is the Virtual Pop-up Book brought forward by Taketa et al. [29].This picture book based on AR technology is said to be user-intuitive without involving any markers that had been used extensively.The authors argued that characters in their book look lively and new means of expressions is possible since 2D and 3D rendering are involved [29].Never before have educators obtained such a new highly visual and highly interactive form of learning.prototypes are reported.For example, a new edutainment medium was introduced by Papagiannakis et al. [31].They proposed an innovative revival of life in ancient fresco paintings in ancient Pompeii and created of narrative spaces of the ancient people's lives.Similarly, when incorporating the AR techniques, museums can build interactive learning scenario in which the visitors can transform from passive viewers and readers to active actors and players [32].A useful exploration in this application is introduced by Miyashita et al. [33] who presented a guide in a museum environment.This kind of applications based on field learning provide learners with more exciting and immersive learning experience meanwhile exciting the learner's curiosity to the unknown.

Objects Modelling
Augmented Reality can also be used to model objects, allowing learners to view items from different perspectives.
An evangel for the students studying mathematics and geometry courses is the Construct3D designed by Kaufmann et al. [30].This 3D geometric models construction tool allows students to design, view and interact with the virtual models so that it encourage learners to experiment with geometric constructions and helps to improve the spatial ability.Students receive immediate visual feedback about their ideas and designs in the way that they manipulate the virtual models directly.Coffin et al. [30] help to strengthen students' comprehension by augmented physical props with virtual annotations and illustrations which the students and instructors may interact with.Their system enhanced the physics lessons by demonstrating the instantaneous moving and forcing conditions on top of the physical objects.Their system can also create molecule models during chemistry lectures available for remote education.

Skills Training
Another educational function where Augmented Reality will shine is in the area of skills training.Henderson et al.
[34] report a prototype AR application to support military mechanics conducting routine training and maintenance tasks.The prototype uses a HMD to augment a mechanic's natural view with text, labels, arrows, and animated sequences designed to facilitate trainee's comprehension and execution.

AR Technologies
Even though the technological demands for AR are much higher than for VR [27], key components required to build an AR system like displays, trackers, graphics and software, and the interactive user interfaces, remain essential techniques in many AR system designs.In this paper, we will review several key techniques with which users may communicate with the augmented environment.Notice that the advances in educational AR technologies depend on the development of the whole AR research area.So when it comes to the analysis of the technological advancements, we are not limited in education.We provide theoretical explanations and analyses of the advantages and shortcomings of each techniques.We choose the following discussed technological directions based on two major considerations: On the one hand, in current studies, two major directions are popular among researchers and developers in the AR field.One is the advancement of new devices and technologies for better combination of the real world and virtual data.The other is the development of applications involving existing technologies.According to Zhou et al., who conducted a review of the currently published research fruits at AR related conferences, AR researchers primarily focus on five core areas essential to deliver AR applications [35].We re-express their finding in the form of a pie chart, like Figure 6.On the other hand, even though theoretically human beings have five different channels to perceive the surrounding world since we are born with five different senses, the olfactory (smell) and the gustatory (taste) ways have still not been fully made use of by current technologies.In addition, the visual (see) and haptic (touch) ways are more efficient in accessing information and thus they have been extensively developed and applied in AR technologies.The sub-topics to be discussed in the following parts are closely related to human visual (display techniques) and haptic (interface techniques) senses.

Display
As we mentioned before, Augmented Reality superimposes the virtual elements upon the real world and both are integrated into a seamless synthetic environment.Hence, the display technologies, which allow users to perceive the real and virtual contents at the same time, play a vital role in an AR system.Basically, three visual displays are available when we are exposed to an AR environment.

Optical See-Through Displays
This kind of display can be ascended to the head-mounted three-dimension display created by Sutherland [23] four decades ago.The first optical HMD with miniature CRT is shown is Figure 7. Optical see-through HMDs work by placing optical flakes in front of the user's eyes [11].Users may see the real world directly through the flakes since they are transparent.Meanwhile, the flakes are also partially reflective so that the virtual objects generated by computer graphical devices and projected from the user's overhead can be reflected into the user's eyes.These functional flakes act like half-silvered mirror.They enable part of the real environmental light penetrate through itself and reflect the artificial environment to the users at the same time.
The optical see-through displays not only demonstrate the real world as it is, they also have the advantage of being safe.Even if the total apparatus are powered off, users can still see the real world, which makes it more suitable for emergency.However, the optical see-through displays don'ts have sufficient brightness and contrast to seamlessly blend a wide range of real and virtual imagery since only a reduced amount of light can penetrate through the "mirror".So this kind of display does not work well in the outdoor environment.Furthermore, the view field is narrowed in a limited range and thus may cause incompleteness of the synthetic images at the edges of the "mirror".But recent advancement in technology shows that the problem of low brightness and contrast and narrow field-of-view could be solved when using retinal scanning display (RSD) techniques [27].

Video See-Through Displays
The second type of display that are extensively used in the AR system is video-see through displays.The single or multiple cameras bundled on the head-mount display equipment provide the view of the real world to the users, together with the artificial objects.The composition results are then sent to the monitor in front of the user's eyes inside a closed-view HMD.
Since the real world is also brought to the users through the camera, a modification to the real world information becomes possible before the users could perceive.For example, the elimination of the unnecessary objects appearing in the real environment become easier.And the brightness and contract problem can be handled more appropriately.Nonetheless, the drawbacks of this display prevail over its advantages.Besides the shortcomings of low resolution of reality, narrowed field-of-view, and parallax trouble to the users, if powered off, the device is nothing but a black box setting in the user's head and causes potential harm to the users.Moreover, users may feel difficult to adapt to this display because the view is significantly different from what they are normally used to.

Projection Displays
Projection-based approach for constructing AR/MR uses physical objects such as walls, books, plaster ornaments and whatever the computer generated contents can be optically projected onto, namely, projection makes it possible to use real objects as displays [36].One simple case to utilize the projective technique is to superimpose the desired image to certain projection surfaces, ranging from plain walls to complex models.Another approach for projective AR relies on head-worn projectors.However, target objects have to be painted with retro-reflective materials and users cannot see anything except along the line of projection.

Tracking
According to [35], the most cited paper is a tracking paper [18] with over 50 citations per year since publication and five of the top ten ISMAR (International Symposium on Mixed and Augmented Reality) papers with the highest citation rates are tracking papers.This phenomenon shows that tracking is one of the fundamental enabling technologies for Augmented Reality.Rolland et al. [37] presented a top-down classification of tracking technologies, described each physical principle and discussed the advantages and limitations of these implementations in detail.Generally, AR trackers used in AR applications need to provide high accuracy, low latency, and provide robust operations under a wide range of environmental variations.Three kinds of tracking techniques are normally utilized in AR systems which are sensor-based tracking, video-based tracking, and hybrid tracking techniques.

Sensor-based tracking
For example, Sutherland tracked the subject through a mechanical device suspended from the ceiling.[23] They named this device as the "Sword of Damocles" who have the ability to send and receive ultrasonic signals to locate the target.Welch et al. from University of North Carolina at Chapel Hill presented a promising novel mathematical method for tracking a user's pose [38].They achieved a high accuracy by properly assimilating sequential observations, filtering sensor measurements, and by concurrently auto calibrating source and sensor devices.

Video-based tracking
The most active area of tracking research in ISMAR with over 80% of past tracking papers described the computer vision methods, which is also known as video-based tracking approach.In computer vision, most of the available tracking techniques can be classified into two types: feature-based approach and model-based approach.The key point of feature-based approach is to find the connections between 2D image features and their corresponding frame coordinates in 3D world.For example, Neumann et al. [39] tracked natural scene features like points and region features automatically and adaptively for direct scene annotation, pose stabilization, and tracking range extension.
Lee et al. [40] designed two kinds of interactive markers which are polka dot and color band.Afterwards, pattern recognition algorithms are utilized to detect the markers and an interaction result is generated artificially.Comport et al. [41] analyzed the visual features for model-based tracking and proposed a novel 3D model-based tracking technique using robust virtual visual servoing.Tracking of objects in the scene amounts to calculating the pose between the camera and the objects.Virtual objects can then be projected into the scene using the pose.Stricker et al. [42] devised an algorithm to track linear feature of landmarks or objects.They only tracked a few sample points which defined a 4-bit bar-code.Through two steps of calibration, they accomplished AR applications in the domains of exterior construction, 3D user interfaces and games.

Hybrid tracking
Even though computer vision alone is able to provide a robust tracking solution, the hybrid methods, which combine several tracking technologies for general purposes, is able to solve some specific problem unexpectedly, just like the case in many other research areas.For example, a system providing a robust experience that surpasses each of the individual components alone consists of an edge-based tracker for accurate localization, gyroscope measurements to deal with fast motions, measurements of gravity and magnetic field to avoid drift, and a back store of reference frames with online frame selection to re-initialize automatically after dynamic occlusions or failures [43].Newman et al. [44] carried out an experiment to dynamically fuse data from widespread and diverse heterogeneous tracking sensors.Theoretically, Klinker et al. [45] proposed the distributed user tracking concepts to support the hybrid tracking method.Another example is provided by Yokokohji et al. [46].Their system combines accelerometers and vision-base tracking techniques and demonstrates accurate registration property even during rapid head motion.

Interactive User Interface
The traditional WIMP (windows, icons, menus and pointings) is no longer applicable to AR systems.Hence new interactive user interfaces have to be designed to support an intuitive and immersive experience in AR systems.There are currently four principal ways of interaction in AR applications: tangible, collaborative and hybrid.
Tangible Interface Ishii et al. [47] introduced the concept of Tangible Bits, which is the origin of tangible user interfaces (TUIs).In their work, the authors defined the tangible bits as "allowing users to grasp and manipulate bits with every physical objects and architectural surfaces".In other words, the physical manipulation in tangible user interfaces provide an intuitive way for users to interact with the virtual content directly.For example, Regenbrecht et al. [48] [55] applied the AR technology into the traditional team game air-hockey.Different from the traditional air hockey game, in their AR2Hockey, the puck is in a virtual space and thus collaboration between the players become a great challenge.As introduced before, Construct3D [30] is also a typical example in collaborative AR applications.In order to support various scenarios within which teachers and students may interact with each other, they implemented flexible methods for context and user dependent rendering of parts of the construction.
Hybrid Interface A hybrid user interface combines a variety of different, but complementary interfaces.Rekimoto et al. [56] developed a rudiment of the original hybrid AR user surface.Using an interaction technique called hyper-dragging, users can transfer information from one computer to another, by only knowing the physical relationship between them.Other prototypes like EMMIE [57] (Environment Management of Multi-user Information Environments, an environment that enables users to share 3D virtual space and manipulate virtual objects that represent information to be discussed) which combines a variety of different technologies including 3D widgets, tracked displays etc. and VITA [58] (Visual Interaction Tool for Archaeology, an experimental collaborative mixed reality system for offsite visualization of an archaeological dig) etc. are also good example of hybrid user interfaces.

Future Trends and Conclusions
As we demonstrate, AR technology is able to bring fundamental changes to a variety of fields and education is an area where this technology could be especially valuable.The seamless interaction between real and virtual environments, the use of a tangible interface metaphor for object manipulation, and the ability to transition smoothly between reality and virtuality [1] make the educational experience supported by Augmented Reality different from other areas.Decades of development in this field have roughly sketched out the future trends of the AR technology.
In this paper, we summarize several directions that the educational AR technology may be heading to.

Ubiquitous Learning
From an educator's perspective, ubiquitous learning is a new educational paradigm made possible in part by the affordances of digital media [59].However, in our point of view, ubiquitous learning is a specific practice of the ubiquitous computing where computing services have integrated into the physical world seamlessly.Portable devices made a great contribution to the rising of ubiquitous learning.Modern mobile devices like smartphones, wearable devices (e.g.Google Glass), tablets etc. make it possible for people to get access to the information service wherever they go.Typically, Henrysson et al. [60] presented a conceptual framework for developing Ubiquitous Mobile AR applications named UMAR.The implementation of UMAR enables learners to receive class information without constraints.Furthermore, learners are also able to communicate or collaborate with their instructors and classmate through a face-to-face AR on mobile phones [61].

Immersive Learning
Studies have shown that immersion in a digital environment can enhance education in at least three ways: by allowing multiple perspectives, situated learning, and transfer [62].Immersive experiences can help learners achieve greater awareness of multiple perspectives of the problem discussed [2].Furthermore, through the situated learning implicit in AR and VR problem solving environment, learners show an increased ability to apply their known into practice.

Collaborative Learning
Augmented Reality can also be used to enhance collaborative tasks.It is possible to develop innovative computer interfaces that merge virtual and real worlds to enhance face-to-face and remote collaboration.These AR applications are more similar to natural face-to-face collaboration than to screen based collaboration In this paper, we have presented a comprehensive survey of the Augmented Reality technologies utilized in the education domain, with analysis of both advantages and disadvantages.We introduced a definition and a brief history of AR followed by a detailed enumeration of the educational AR applications.Even though the AR technology is not new born, it is still at infancy and thus there are many problems to overcome.For example, the creation of 3D models for AR is difficult since significant technical knowledge is required.Nonetheless, advancement in AR will offer more solutions to these difficulties and take ubiquitous, immersive and collaborative learning styles into our lives.We believe that AR will bring more photorealistic experiences and higher-efficiency mode to the learners.

Figure 2 .
Figure 2. (Left)A women is operating The MagicBook handheld display[28].(Right) Students are working with Construct3D in the standard lab, studying the vector algebra.Images generated as live video capture with computer overlays.[30]

Figure 6 . 5
Figure 6.(Left)A pie chart of the proportions of each study directions.We can see that the five core areas are tracking, interaction, calibration, applications and display.[35]Figure7.(Right) The first head-mounted display in the history.[23]

Table 2 .
Common applications of VR, AV, and AR

Table 3 .
A brief summary of pros and cons of each display technique.
designed a MagicMeeting featuring face-to-face communication, collaborative viewing and 3D-model 7 Potential for Augmented Reality in Education: An Overview EAI Endorsed Transactions on Future Intelligent Educational Environments 09 2014 -06 2015 | Volume 1 | Issue 2 | e2 manipulation.They set all virtual contents attached to tracked physical objects in order to leverage the efficiencies of natural two-handed manipulation.Another example is provided by Lee et al. [49] who applied AR technologies to physical blue foams, entitled Augmented Foam, so as to handle the intangibility problem in computer aided designs.A notable star in the tangible user interface technologies is the Pinch gloves.It is a remarkable new interactive style using gestures for a wide range of control with 3D simulations.Several AR or VR systems [50-52] based on the Pinch gloves interaction reported to be more flexible and convenient for the users to operate.Collaborative Interface Before Schmalstieg et al. [53] designing the first applicable collaborative AR interface in 1996, most existing augmented applications are single user setups, or do not exploit the multi-user character of their systems.Billinghurst et al. [54] introduced a similar approach named Share Space for CSCW (computer supported collaborative work) two years later.Ohshima et al.