The Immersive Education Laboratory : understanding affordances , structuring experiences , and creating constructivist , collaborative processes , in mixed-reality smart environments

In this paper we describe how the iClassroom and other technologies are providing the testbed through which we are able to design, develop, and research future intelligent environments. We describe the process of distinguishing between the technical and pedagogical aspects of immersive learning environments, while simultaneously considering both in the redefinition of effective intelligent learning spaces. This paper describes how our laboratory is working on specific projects that increase our understanding of the distinct advantages of technical design elements, like immersive visual displays, and pedagogical design elements that need to be in place as we go through the process of structuring learning situations that create constructivist, collaborative experiences. We describe specific technologies and their design across these multiple dimensions and the ways in which they are helping us better understand how to maximize technological affordances for increased positive learning outcomes. Finally, through this design research process, as we begin to better understand the affordances and iteratively create design guidelines, our hope is that eventually a prescriptive framework emerges that informs both the practice of embedded technology development and the deliberate incorporation of technical attributes into both the educational space and the pedagogy through which students learn.


Introduction
The iClassroom is a research testbed within the Immersive Education Laboratory (iEL) in the School of Computer Science and Electronic Engineering (CSEE) at the University of Essex in the UK.This paper describes how the iClassroom is being used to carry out research into future intelligent educational environments.We first provide an overview of the problem space being explored and how the concept of immersive education relates to intelligent educational environments.We then identify some key themes that characterise the research being undertaken and describe a number of exemplar projects within the lab that illustrate these research themes.A key aspect of this is the need to conceptualize a framework for understanding and defining immersive learning environments.An initial outline for this framework is presented, with the intention of developing practical tools to help practitioners and researchers make informed choices in this emerging field.Finally we discuss the implications for future research into immersive education and intelligent educational environments.As we consider the design of intelligent environments for learning, the deliberate distinction should be made between the parts of the embedded technology that function in a way that gives the user wider or better access, helps make a task easier or more connected, or represents a construct in a new form.Immersive learning that is built into intelligent environments will need to be designed in a way that maximizes the added value of the technological advantages built into the world around us for meaningful learning.That is to say, because we have added technology that may create a more immersive environment, the design of our classrooms must also include the purpose and function of new technologies in the specific role of facilitators or even the context for interactive, constructivist The Immersive Education Laboratory: understanding affordances, structuring experiences, and creating constructivist, collaborative processes, in mixed-reality smart environments 3 learning.Our projects demonstrate the progression of our design based research in the development of not only classrooms that are intelligent and immersive, but that also bring value add to learning

The iClassroom
Dooley in [7] describes the early ideas around the creation of the iClassroom laboratory at the University of Essex.A core objective of the iClassroom was to provide an ambient intelligent environment (AmI) to support teaching and learning activities.AmI techniques and algorithms were previously utilized in other smart environments research at the university.For example [8] describes how embedded intelligent agents were deployed into the user environment (the iSpace) so that they could control the living space according to the needs and preferences of the user.A novel fuzzy learning and adaptation technique was developed to implement the agents that were embedded in the environment.Building on this previous work, the goals of the smart classroom (from [7]) were to:  Construct and develop an intelligent classroom through the deployment of ubiquitous computing [24] and ambient intelligence (AmI) [25] that facilitates learning/knowledge transfer. The deployment and evaluation of technology to locations outside the classroom that permit interactive and immersive remote participation.
A 3D model of the iClassroom is illustrated in figure 1.To facilitate the deployment of necessary technologies, both as part of the original design and as later augmentation, false walls and ceilings provide hiding places for embedded devices/sensors.These are then over-populated with power and Ethernet sockets in support of the electronic artefacts they will eventually yield.All Ethernet sockets are wired to a central patch panel and are interconnected to form a network that is isolated from the rest of the university.A single access point provides secure wireless access to the iClassroom network, while a gateway/firewall provides Internet access, basic network services (such as DHCP) and also allows certain service requests to be handled from outside the iClassroom.Overall, this forms a raw skeleton into which ubiquitous computing can be embedded.We have reused many of the technologies developed in our previous works and have deployed computer controllable lighting, heating and ventilation (HVAC), door-locks, RFID readers and ambient displays in addition to an array of sensors that are all exposed through middleware to the network where intelligent agents can discover, monitor and manipulate them based on embedded AI.As part of our ongoing research, both the middleware and the agent-based techniques can be swapped out and replaced by others -this permits the evaluation of many approaches, models and methods in various permutations.Thus, the space itself is as much a subject of research as the human activity that it supports.
To enable familiar human interaction, we have added projectors, a large interactive whiteboard, wall-mounted touch-screens, handheld/tablet/pad devices and a desktop PC (as part of a lectern setup that aids in the delivery of presentations).In combination with a multi-speaker audio setup (where each speaker is embedded in the ceiling and able to render an individual audio stream), the iClassroom is equipped for multimedia delivery, interaction and control.
To complete the design of the iClassroom; additional equipment has been deployed that provides various video streams (360° top down, 180° fly-on-the-wall, movable high definition and thermal spectrum) and affective monitoring of participants (galvanic skin response sensors, heart-rate monitors, embedded seat sensors, brain-computer interface headsets, etc).It is intended that this overall deployment can provide a starting point for the development of new technologies across the whole spectrum of ubiquitous computing and AmI within the context of teaching and learning.

Research themes
The iClassroom provides us with a highly configurable experimental space for carrying out ambient intelligent research into immersive education.However this domain is potentially too large and diverse for any one research laboratory to consider in its entirety.Therefore we have refined this domain into a number of key themes that characterise the research activities within the Immersive Education Laboratory.
As a starting point we feel that there is very little support for practitioners, designers and researchers in creating intelligent immersive education spaces.A key issue is to ensure that design decisions are based on sound pedagogical principles that aim to maximise the affordances of these environments.So our first research theme is rooted in understanding the affordances of this technology.This aims to combine the collective wisdom across the disciplines of computer science, human-computer interaction, psychology and education so that we can build more effective immersive learning environments.
Our next two themes are rooted in the need to focus on the process of teaching and learning, rather than the end products.A key issue for teachers is being able to design structured educational experiences.This can be challenging for traditional education and e-learning activities, but becomes particularly more difficult when we start to consider the use of intelligent immersive education environments.The second aspect to this process oriented view of the world is then to consider the social context for teaching and learning and how intelligent immersive environments can support collaboration between students and teachers throughout this process (rather than just focus on the end products arising from the content generated during this process).
Our final theme is then looking at a specific technical aspect of immersive education, which is the boundary between the real and virtual environment.We are particularly interested in the opportunities arising from combining the real world and the virtual world into a mixed-reality smart environment.The rest of this paper provides more detail on some our projects that are addressing these themes.

The affordances of virtual environments
Taxonomies have been developed that describe the technical aspects of immersive technologies [4], in addition to frameworks for describing learning affordances of virtual learning environments (mainly virtual worlds) [5,6], but none have been able to sufficiently and completely capture the multiple levels and complex interactions between them in terms that can be beneficial to designers and researchers for accurately describing the technologies with which they are working and the affordances of those technologies for learning.As we move forward in the design of immersive learning environments, being able to better define and classify new tools and research is imperative.
The most commonly cited work in describing augmented reality is Milgram's continuum [4], which is helpful at providing an initial framework for describing immersive education applications but is insufficient since it does not move past describing the visual display characteristics.Particular affordances of augmented reality for learning lie in the technology's ability to represent abstract concepts, display content that is invisible or no longer exists, and delivers content in context.Not all technologies, as described in this paper are designed for a learning purpose, necessarily.However, while there is a need to distinguish these applications, it does not mean that one is necessarily less valuable than the other.
Technologies can be described on multiple levels.Some technologies are adept at increasing access.While others make particular tasks more manageable, easier, or faster.These technologies can be described as utility applications.While the application of some technologies, by design, necessarily and purposefully contribute to and meaningfully impact the way in which learning happens.That meaningful application requires designers to both harness the affordances of the display technology and apply it in a way that impacts learning [41].Some of the affordances of the technology providing the context for learning include increased opportunities for interactions with other students (social constructivism), collaboration [40], and improved spatial understanding [5].For example, using augmented reality for the to visualize the internet-of-things has value, but the display of information does not mean that there has been an interaction between interface and user that impacts learning.
Elliott and O'Shea [41] described another perspective of defining immersive learning technologies based on their learning purpose or function.Making the distinction that not all technologies that fall under the description of "immersive" or "learning" necessarily accomplish either of those goals.For example, in this study, over 300 educational augmented reality applications available on the current Android and iOS markets were analyzed to determine the designed purposed for the technology in the context of learning.Building on the work of Schrader, who described the dimensions of technology as learning about, from, with, and within technology, Elliott has developed an evaluation tool that will help measure and define the use of immersive learning technologies.The purpose and function of the technology are defined across levels from basic utility, content delivery, and assessment, to experience.The framework describes the most shallow (and most prevalent) use of augmented reality in education applications as a trigger mechanism that either launch another website, movie, or other (generally) static image or video.The interactions usually required the use of a QR code that users viewed with their mobile camera, launching an image, video, or website.The next categorization is Utility, which means the technology serves a functional purpose that is not directly related to learning.The third dimension describes the function of the immersive environment as Content Delivery or Information Access, the technology is used to over content, deliver content or access data, but does not require the user to interact with the content in a way that is meaningful for learning.When the technology is used in a way that delivers content, but also requires the user to respond interactively it is categorized as Content Delivery+ Assessment.The last dimension of the framework, Content Delivery+Assessment+Experience/Context describes the interaction with the technology where the learner experiences content, learns from it, and does so in meaningful context and through and experiential process.Although certain dimensions of the framework are less valuable than others, the framework is not hierarchical and categories are not necessarily mutually exclusive.Using this framework as the analysis tool, Elliott and O'Shea found that less than five of the 300+ applications used augmented reality for a purpose beyond Trigger or Utility.
Researchers [5] have created frameworks for understanding the learning benefits of 3D virtual learning environments (VLEs) using a learner-computer approach that accounts for the role of representational accuracy (see The Immersive Education Laboratory: understanding affordances, structuring experiences, and creating constructivist, collaborative processes, in mixed-reality smart environments 5 figure 3).They cite increased collaborative and experiential learning opportunities, as some of the learning benefits of 3D VLEs, however, they describe environments that cannot truly be defined as 3D from a visual display perspective because they are housed in a 2D desktop computer [42].
Dalgarno & Lee [5] posit that the technical capabilities of the technologies create immersion and the individual's cognitive response to the 3D VLE creates presence.They argue that immersion and presence should be considered as individual constructs since they are both the result of psychological interactions with the technology.Dalgarno & Lee define characteristics of 3D VLEs as falling under two categories, 'representational fidelity' and 'learner interaction', which lead to 'identity', 'presence' and 'copresence', which lead to afforded learning tasks that result in learning benefits.

Designing structured educational experiences
Our focus on supporting the process of teaching and learning is mainly aimed at the teacher by helping them to design and deploy structured learning experiences in smart environments.We then need to support the student in undertaking these learning activities.This work is rooted in our previous research into the configuration of ambient intelligent environments.For example in [9] we introduced a vision for a new type of domestic appliance, a softappliance, constructed from aggregations of elementary network services.This vision was based on the possibility of 'deconstructing', logically, conventional home appliances such as TVs into their elemental functions which may then be combined in novel ways with other deconstructed services to generate soft-appliance of a person's own choosing.An essential component of this vision was a concept called a MAp (meta-appliance/application); a semantic data template that describes the soft or virtualappliance that can be instantiated by manufacturers and endusers in a way that redefines the nature of an appliance and which can be created, owned and traded.These MAps could be created by an explicit process of end-user programming which uses a variant of Programming-By-Example (PBE) [10] called Pervasive interactive Programming (PiP) [11].
Pervasive interactive Programming differs from PBE in that, firstly it aims at real rather than graphical objects, secondly it is directed at distributed computing rather than a single processor, and thirdly it spawns distributed non-terminating sequence independent MAps (soft-appliances) rather than creating macros or other procedural structures.The creation of these MAps using PiP addresses many of the same issues that users (mainly teachers) will have in creating structured learning activities in smart spaces.Essentially it provided a relatively easy way for nontechnical users to configure their smart space for any given activity.If we replace the smart home with the smart classroom then PiP could be used by teachers as a mechanism for creating structured tasks that make use of the infrastructure and 'services' available within that environment (for example in a teaching context this could be the configuration of the smart board, projector, networked PC, etc for a teaching session).However, the MAps developed using PiP so far only exist within the real world.Section 4.4 below discusses how we can combine real worlds with virtual spaces to provide mixed-reality environments for teaching and learning.The next challenge we are exploring for PiP is to investigate how it can be used as a mechanism for not only configuring the real world, but also the associated virtual spaces.A key issue that will need to be addressed is how to effectively synchronise real and virtual spaces that provide alternative representations of a single realitythis is discussed further in section 4.4 below.Also the PiP approach is mainly targeted at the creator of the program, which in our context is mainly focused on the role of the teacher in creating learning activities that will then be undertaken by a group of students.So far, PiP does not differentiate between these roles treating each user in a similar way.However it is clear that the teacher and students will have different needs and constraints that would need to be fulfilled in order to use this approach with the context of formal education.
In many ways the PiP approach to manually creating MAps (which describe how an intelligent environment should be configured in order to achieve a given task), is directly counter to the type of approach described in [8] where intelligent agents attempt to automatically create these programs by observing the behavior of the user within the environment.While some believe that agents should have very minimal autonomy and should only act as directly instructed by the user, others consider providing agents with autonomy to be an essential aspect to building intelligent environments.This is also true for the creation of structured learning activities.Should the user (eg.teacher) be in charge of creating these activities, or should intelligent agents automatically generate these activities?To address some of these issues we have developed the concept of Adjustable Autonomy [12], which aims to enable human users and agents to collaborate in managing intelligent environments as a team.With this approach we were seeking to develop an adjustable-autonomy agent in an effort to explore the user acceptance of pervasive computing (and the use of autonomous agents therein), as well as aiming to improve the robustness and reliability of future intelligent environment systems.We are currently [13] applying these techniques as a way of allowing students to control the sequence of learning activities that they choose to study.In this application we are creating a flexible learning environment which allows the student to either have full control over the sequence of their learning tasks, or to allow the system to automatically choose the most appropriate sequence, or to allow the student to adjust their level of autonomy during the selection of course modules.We are developing a system called iPerSeq (an intelligent Personalised Sequencing system) that uses machine learning combined with adjustable autonomy to intelligently personalise and adapt the learning path for individual students based on an analysis of their previous contributions and behavior.
The ability for teachers to create a specification for a structured learning activity and deploy it across different learning environments has been an objective for the IMS Learning Design (LD) specification for some time [15].A number of tools have been inspired by this to provide a mechanism for teachers to easily author learning tasks based on the IMS-LD specification.For example, LAMS [16] the Learning Activity Management System, is an open source Learning Design system for designing, managing and delivering online collaborative learning activities.Also the Open University of the Netherlands has developed an IMS LD engine for playing LD called CopperCore [17].
However much of the emphasis of this work on Learning Design has been around delivering these learning activities into 'traditional' e-learning environments such as Moodle.Very little if anything has been done on investigating whether the Learning Design approach could also be applied to immersive education spaces such as 3D virtual worlds.This is particularly important as one of the identified barriers to the adoption of 3D spaces for learning has been the perceived difficulty for teachers to construct or author meaningful learning activities which could be deployed in these environments.This issue was addressed by our research project [14], which attempted to combine the robustness and ease of authoring of LD with the capabilities that a 3D environment offers.Our approach was to deploy the task design (IMS-LD Units of Learning) sequence created in a 2D authoring interface such as LAMS into a 3D virtual world using Open Wonderland [18].Once an activity sequence is created in LAMS (which may contain a collection of activities with data and transitions) it can be exported to an IMS-LD level B [15] conforming XML document.The XML file consists of the markup describing the content and the order of each activity in the sequence.A separate upload interface was created in the web administration page of OpenWonderland (a 3D virtual environment) and a predefined 3D world was designed to hold the optimum number of LAMS activities in preset locations.During the upload and parsing of the XML file, each LAMS activity renders the corresponding module in the predefined positions in OpenWonderland (using the 'snapshot' capability in OpenWonderland).Figure 2 illustrates different learning tasks as they are rendered in OpenWonderland.The student can then begin the learning activity, which will place them in the starting location (room) for the first activity in the LD specification.The transition from one activity to another is handled by OpenWonderland portal modules that are configured with the location coordinates of the next activity in the 3D world (which have been pre-configured by the LD specification).The capabilities feature of the container holding a particular activity will not let the learner move outside the container without first completing it.In this way, the learner can 'jump' between each step in the learning activity.Currently the rendering of the 3D space for the learning activities is limited to a predefined design format and a maximum number of activities.However, we hope to make this more dynamic in a future version of the tool.The project demonstrates a new approach to the creation of dynamic learning activities in a 3D virtual world based on XML data conforming to the LD specification.Our aim is that this type of toolkit could be practically used as an adapter to any 2D LD authoring environment as a way of deploying structured learning activities into 3D virtual worlds.

Focus on collaboration rather than content
Much of our research work is based on a constructivist view of education.Constructivist approaches emphasise the active building of understanding through the performance of learning tasks in which the learner decides how to proceed, based on his or her current understanding of the task and of the domain of knowledge in question.Often the task will involve some kind of problem solving, although this can take many different forms.The goal is for learners to build their own knowledge and is a much more learner centred view of education.The focus is on supporting the learner in the performance of tasks that have been designed to engage the learner in active problem solving, questioning and conceptual manipulation [19].Technology can be used to support the learner through this process, and this has been discussed in the preceding section on structured learning activities.The clear focus here is on supporting the learner through the process, rather than just simply focusing on the instructional content being used or the outputs produced by the student.Although this can be a singleton activity, technology can be used to explore new ways of supporting constructivist learning activities involving the collaboration between students at different locations.In this section we discuss our research into immersive education that is concerned with supporting collaborative learning activities.
Our work on the +Spaces project [20] provides a good example of this approach.The project explored the use of virtual worlds to support online role-play as a collaborative activity.As a first step in creating a generalizable role-play simulation framework, the project developed a number of role-play templates to help the policy maker or teacher to The Immersive Education Laboratory: understanding affordances, structuring experiences, and creating constructivist, collaborative processes, in mixed-reality smart environments 7 devise an appropriate role-play simulation to support a given issue.The project created both 2-dimensional (web-based) and 3-dimensional (virtual world) environments to support these synchronous role-playing simulation events.For the role-play simulation, the 3D environment was implemented in Open Wonderland and the 2D environment used the Twitter service.When a user enters the Open Wonderland role-playing chamber it clearly displays the current phase of the role-playing simulation session as well as the current topic.It also displays the participant's own role to the other participants.A 'moderator' user controls the process of completing the role-play and only they have access to a toolbar that allows them to step through the stages of the different role-play activities.Figure 3 illustrates the Wonderland role-play chamber and a brainstorming post-it wall that is used during one of the role-play activities.
It was interesting that some of the participants preferred the experience of the role-play compared to their previous experience of taking part in an Open Wonderland pilot of a debating application.Because the process in the role-play was much more structured they felt that it was easier to follow than in the debate (which was only loosely structured).They also felt that it was easier for the participants to fully engage with the activities because they had a clearer idea of what they should be doing at each stage (ie.they felt that they could participate more fully).

Figure 3. Online role-play and brainstorming
The tools developed by the +Spaces project to support online collaborative role-play can also be used in classroom based learning activities.Simulations have long been used to support constructivist-learning tasks, particularly based around participatory models of learning [21].However, the 'black-box' nature of these simulation models is recognized as a limitation in their use for teaching and learning, where students can often get frustrated by the hidden nature of the underlying simulation models.There is also evidence that it can result in 'superficial understanding' or 'factually wrong conclusions' about the topic [22].Contributory, 'glass-box' based approaches to discovery learning are therefore encouraged.The +Spaces role-play tools also take this approach.By facilitating online role-plays, we envisage that students can go beyond the superficial understanding of complex topics, to become more engaged with and ultimately achieve a better understanding of the subject matter.When combined with the use of 3D virtual environments, we hope to provide more highly engaging immersive collaborative spaces for teaching and learning to take place.
A key requirement for the +Spaces project was to provide users with access to the collaboration tools via a range of different online environments (OpenWonderland, Twitter, Facebook, Blogger).This was based on the recognition that some environments are better suited to synchronous collaboration (such as OpenWonderland), whilst others are better suited to asynchronous collaboration over more prolonged periods of time (such as Facebook and Twitter).In +Spaces we essentially provided variants of the same tools in these different environments.However, most users make use of different environments for different purposes.For example a Facebook group provides a persistent place for sharing resources amongst group members and asynchronous communication via chat and email, whereas a virtual world may be more suited to live synchronous collaboration where discussion and immediate feedback is required.To investigate this further we carried out a further project [23] that investigated the potential of using a social network group alongside a 3D virtual collaborative learning environment.The challenge was to find a novel innovative approach to allow learners to seamlessly switch between these two environments.This involved the development of a new Wonderland module to integrate these two platforms.The implemented module communicates with the Facebook Group via an Access Token that was used to manage the authentication and authorisation process between the two environments.To generate an Access Token, a new Facebook Application was also developed which employed the OAuth 2.0 technique to link the user to their Facebook group.Additional in-world applications were created for the 3D environment to allow users within that space to easily post a new message or add a new comment from the 3D world to their Facebook Group.The overall feedback from this study was that by integrating the social network group within the 3D virtual collaborative environment it could better support the need for learners to use different environments for both asynchronous and synchronous collaboration.One implication of this is that both social group interaction and the concept of accessibility should be taken into account when designing a 3D collaborative learning environment.

Combining real and virtual in a mixedreality smart environment
So far we have given examples of projects that address issues concerned with supporting collaborative learning and supporting the process of learning through structured learning activities.These two issues are highly related and indeed collaboration could be seen as part of a process described within a structured learning activity.So far, much of this discussion has also been around supporting the learner within an online learning environment.However, not only does learning normally occur with participants who are co-present with each other, there is also the added dimension of using technology to augment or support the real-world experience.This now brings us to probably the most challenging part of our vision, which is to explore how we can combine elements of online (or virtualised) learning with the real-world.The particular challenge it to find the sweet-spot between the combination of the virtual and the real which can best support the needs for a given learning activity.In section 4.1 above we discussed the need to understand the affordances of different technologies and approaches so that we can make better design decisions when building immersive education solutions.This becomes more difficult when the dimensions to any one learning activity can be addressed by many different possible solutions.In this section we explore some of our research that is combining elements of the real and virtual worlds to create mixed-reality learning experiences within smart environments.
Our first project that combined real and virtual worlds was MiRTLE [26].The objective of the MiRTLE (Mixed Reality Teaching & Learning Environment) project was to provide an online virtual classroom to augment live lectures.This was inspired by the observation that even if remote students were able to watch a live lecture remotely (for example using video conferencing or other similar technology), they often would choose to watch the recorded session instead.The main reason for this being that there was very little perceived value in their participation in the live event, as often there was only limited means (if any) for them to interact with the people in the live classroom.This meant that the recorded version of the event usually offered an equivalent experience with the advantage that they could also choose to watch in their own time.MiRTLE provided a mixed reality environment for a combination of local and remote students (both dispersed and local students are able to see and talk with each other, in addition to the teacher).The environment was intended to augment existing teaching practice with the ability to foster a sense of community amongst remote students, and between remote and colocated locations.In this sense, the mixed reality environment links the physical and virtual worlds.Using MiRTLE the lecturer in the physical classroom is able to deliver the class in the normal way but the classroom also includes a large display screen mounted at the back of the room that shows avatars of the remote students who are logged into the virtual counterpart of the classroom.Thus the lecturer will be able to see and interact with a mix of students who are present in both the real and virtual world.Audio communication between the lecturer and the remote students is made possible via a voice bridge.A camera is placed on the rear wall of the room to deliver a live audio and video stream of the lecture into the virtual world.From the remote students' perspective, they can log in to the MiRTLE virtual world and enter the classroom where the lecture is taking place.Here they will see a live video of the lecture as well as any slides that are being presented, or any application that the lecturer is using.Spatialised audio is also used to enhance their experience so that it is closer to the real world.They have the opportunity to ask questions just as they would in the physical world via audio communication.Additionally a messaging window is provided that allows written questions or discussion to take place.The MiRTLE virtual world also offers a common room where students can meet socially and access other resources for their course.Figure 4 illustrates the virtual world for the online students in a MiRTLE class.
From the initial evaluations of MiRTLE at the University of Essex, a number of valuable issues were highlighted that have implications for future uses of this technology.It particularly highlighted potential social issues, such as the impact on student motivation and perceptions of crowding and jostling for position in the virtual classroom.Trials showed that there was potential for impromptu and naturalistic social interaction between virtual and physically present students.Teachers also recognized the potential value of the system, reporting that, once students are logged on and settled, the MiRTLE environment had a minimal impact on normal patterns of teaching, and the teachers perceptions of the learning occurring in their teaching environment.An important emerging theory is that the previously described finding of spontaneous social exchanges between virtual and physically present students suggests that MiRTLE can facilitate a breaking down of the barriers between the virtual and the physical, and increase a sense of presence for all learners and teachers involved.MiRTLE is currently deployed in the iClassroom.Also the University of Hawaii is carrying out innovative work [37] to extend the MiRTLE concept.They have developed a HoloDeck system that also allows the physically present students to interact with the virtual students by using a mobile tablet based application.MiRTLE uses virtual reality in the form of avatars and a virtual world to bring geographically dispersed learners together.However, we are also investigating how other technologies can be used to achieve some of the same objectives.Torrejon [27] describes research into how we can replace avatars with video of real people augmented with The Immersive Education Laboratory: understanding affordances, structuring experiences, and creating constructivist, collaborative processes, in mixed-reality smart environments 9 panoramic audio at 2 different levels; one for pure audio transmission to the remote user, and a second to calculate the dimensional position of the source, thereby enabling the audio to be recreated and spatially controlled by the end user.This research is pushing the boundaries of videoconferencing in an attempt to achieve the sensory feeling of "being there".Towards this end, we have created an immersive tele-presence system that facilitates physically dispersed students, or groups, to collaborate around a shared task with a sense of shared presence.At one end of the link, the local space (e.g. a lecture theatre), a 360° mirror lens is placed in the room, from where a spherical image is captured and then transmitted to a remote location in real time.Once this stream is delivered, it is converted from polar to Cartesian coordinates to create a panoramic video that is projected onto a 180° screen.3D audio is also collected in order to reconstruct a more natural sound image for the remote learner by using binaural techniques and directional speakers or headphones.This setup allows remote viewers to participate in events as though they were local participants, enjoying much greater control over their visual and audio context.
It is important to remember that this project does not aim to provide a 3D image that can deceive the brain into a false belief of contextual presence [28] but rather to provide a 3D immersive experience where the users can directly manipulate the direction of view and its field of view without affecting others' field of view (FOV).The panoramic immersive media system is capable of deconstructing and reconstructing remote spaces to give access and additional information to distant learners and local groups.This approach provides key elements for the success of online activities such as learning, by providing communication and engagement, and creating a ludic space that is not limited to the academic activity but to any life learning scenario.Thus, we hope that this work provides a new perspective for online education that goes beyond the current state of the art by offering panoramic real-time video and audio connections that are controllable and more engaging to users.
Both the MiRTLE and panoramic audio/video projects are concerned with increasing the sense of presence for participants in a traditional teaching environment (such as a lecture room) by combining real and virtual participants together into a single mixed-reality space.Both of these scenarios are based on fairly traditional instructional learning that can take place in a lecture.However, we are also investigating the use of mixed-reality to support more complex teaching and learning scenarios that involve students in a more varied sequence of activities.The first of these projects is described in [29] and is based on an approach that uses Augmented Reality (AR) technology.
Here we are investigating how augmented reality can be used to make deep IT technologies (ie.invisible IT entities) visible.We believe that this can provide a valuable view for both learners and developers in terms of gaining a better insight into the abstract concepts of the technology that is often woven into the fabric of our everyday lives.In particular we are focusing on the Internet-of-Things, a paradigm that uses small networked embedded computers (which are largely unseen) to make pervasive computing applications.To reveal these invisible processes an AR model called a View-Point, has been developed to visualize and interact with a small, self-contained eco-system of networked embedded components using a system of Buzz-Boards [30].The approach seeks to enrich the developers and learners experiences by providing a view of the invisible embedded-computing elements surrounding us.Moreover, in support of the suggested framework, a 4-dimensional learning activity task (4DLAT) has been proposed, which assists in structuring the study into a number of different stages, through which progress is made from a singlelearner/discrete-task to a group of learners undertaking a number of sequenced-tasks (as illustrated in figure 5).This framework aims to combine the previously discussed themes of sequenced learning activities (the Discrete and Sequenced dimensions) and the theme of collaborative learning (the Single and Group dimensions).We hope to use this framework as a means for guiding both the design of the educational environment and also as a way of partitioning the educational support provided by the learning systems within, and as such it provides the beginnings of an embryonic design framework.

Figure 5. 4DLAT framework
We are also developing a computational framework which we refer to as the 'Pedagogical Virtual Machine' (PVM) that acts as a manager for revealing educational learning related functions to the students involved in the learning activity (see figure 6).
The PVM not only provides the basis for supporting the student in carrying out structured learning activities in a smart environment, it also combines both technological and pedagogical support within that framework.At the bottom is a Data layer that receives real-time data from devices within the intelligent environment.This is fed into an Abstraction layer that attempts to model the state of the environment using an object-oriented hierarchy.
Brad Cox [31] explained that when he started thinking about objectoriented programming he had the vision that everything in this world could be regarded as an object.This inspired us to think about hardware and software in embedded computing as objects as well.This model implies that all computer objects (hardware or software) contain data that represents the objects state and can be communicated with other objects.This is then fed into a Pedagogical layer that combines information about the learning activity being undertaken with an overall model of the pedagogical process being supported.To implement this we are currently using Learning Design (see above) as the means for specifying the detail of the structured learning activity being undertaken, and the Mayes-Fowler pedagogical framework [19] to provide the context of the learning activity.Finally this is fed into the User-Interface layer that is responsible for structuring the relevant information in the most effective way for the student (currently this is through an augmented reality interface).

Figure 6. Pedagogical virtual machine
An important aspect of the PVM is the unification of the pedagogical needs with the architectural capability of the underlying technology.For instance a student/learner would need to be aware (via visualization) of the active software and hardware behaviors.The idea of the pedagogical virtual machine is to provide a platform-independent interface for students and teachers to access information that is pertinent to learning.In this respect it has some similarities with the virtual machine used to support mobile code in web systems (eg. the Java Virtual Machine).However, it does not execute code (in a programming language sense) but rather responds to a set of generic commands that gathers system information (or instrumented data) from the underlying hardware about the software executing.It aims to provide students and teachers with a portable, common and familiar interface irrespective of the underlying hardware (in that sense it acts as a virtual machinethe 'machine' being the monitoring apparatus).In addition, it will include some customizable features that allow teachers to filter exactly the type of pedagogical information they need for a particular topic or lesson.Augmented Reality is being used to provide the mixed-reality interface that can augment the real-world components with virtualized information.The AR technique provides a virtual object overlay in the real-world domain, and can enable users to feel more immersed in the domain through the interactions facilitated between the real and virtual worlds [32].Thus, AR combines virtual objects in a real-world context.From the viewpoint of the user, the aim is that the objects be rendered complete and harmonized with reality, including presenting the same contextual environment.
This combination of real and virtual objects into a coherent blended-reality learning experience raises new challenges and approaches when building systems for teaching and learning.In [33] we describe our efforts towards the implementation of a blended reality distributed system.To achieve integration between real and virtual objects we have developed descriptions of these smart objects (xReality objects) that can then be used by immersive technology in a mixed-reality learning environment.This research builds on our previous work enabling geographically dispersed learners to collaborate on laboratory activities.xReality objects are smart networked objects coupled to their virtual representation, updated and maintained in real time to create a mirrored state (dual reality).This approach is being examined in the context of a collaborative laboratory activity where students need to collaborate together in order to produce Internet-of-Things (IoT) applications that emphasize computing fundamentals.
We are developing a number of scenarios for learning activities using a combination of virtual objects and xReality objects in an individual or collaborative session.In our first implementation we only considered the possibility of using single services through the 3D virtual world.Figure 7 illustrates this scenario, where there is a single student interacting with both a real and a virtual version of a robot.Future research will include the creation of sequences of services designed by the learners, similar to Chin's virtual appliances approach described above.Therefore 4DLAT's full spectrum of sequenced activities (Single-Sequenced/Group-Sequenced) is not currently being considered in this preliminary learning scenario.
The first scenario examines the use of only virtual objects, either in an individual or collaborative session, which is similar to virtual laboratories where simulation is the key to performing an action.In this case although there is synchronization between virtual representations within a collaborative session, there is no dual reality state.A dual reality state involves the coupling of a real object to its virtual representation, which is updated and maintained in real time [34].
To use the system a learner starts the 3D learning environment and, once they are authenticated the 3D virtual environment, will display the "lobby" space, where they can chat with other learners and arrange a learning session.As soon as the learners join one session they enter into a shared virtual world where they can see each other as avatar representations, and they can see the virtual representation of the xReality object(s) linked to any of the users of the current learning session.These objects will have been detected using the broadcasted list of services available, which is located on the left side of the main screen.A chat window is also located on the right side of the screen to allow the users to communicate during the learning session.The Immersive Education Laboratory: understanding affordances, structuring experiences, and creating constructivist, collaborative processes, in mixed-reality smart environments 11 Our testbed is deployed on an immersive environment using Immersive Display's ImmersaVu platform [35], a composite molded panoramic dome screen, which allows a free-range of head movement without the need of any special instrumentation (such as glasses or other devices) that can interfere with the learning session.Our current implementation manages single dual reality states (ideal and shared), the first in an individual session and the second within a collaborative activity.However the architecture proposed allows the implementation of multiple dual reality states.This opens up the possibility for learning sessions in places where laboratory resources might not be available due to place or money constraints.Our future work will be towards the implementation of multiple dual realities (ideal and complementary) using two or more xReality objects; and the integration of sequenced groups of services to be executed within our Inter-reality Portal, encouraging teamwork, creativity and innovation.
We are also running several other projects that combine real and virtual worlds into smart environments.This includes an investigation into gesture-based control of learning games [36], and also a number of current projects using augmented reality.For example, one project is aiming to simulate simple science experiments in AR, and another is developing a mobile augmented-reality app that provides context-dependent location aware information to users, based on the surrounding buildings.

Discussion
With a plethora of technological advances at our fingertips, we have the ability to increase access to technology and content, display content from the internet-of-things, visualize abstract concepts within immersive environments, interact with peers and colleagues remotely both synchronously and asynchronously, and the list goes on.Part of the difficulty with the rapid advances in technology is our ability to rapidly design, develop and research educational spaces on two levels.Firstly, there is a technical and infrastructure level that needs to be iteratively tested in spaces like the iClassroom so that we can understand the logistics behind delivering working immersive learning environments.Secondly, there is the need to address the use of these embedded technologies for the purpose of learning and the creation of pedagogical situations that harness the affordances of the embedded capabilities in a way that is meaningful for learning.Additionally, a distinction needs to be made between the infrastructural components of intelligent environments and the affordances they provide (such as whether there can be increased utility or enhanced learning capabilities).Hopefully, through the description of our research themes and projects we have demonstrated that this is a very wide space.
Our focus is (technically) at the convergence of intelligent environments and immersive educationfuture intelligent education environments and the forward moving process of iteratively designing, studying and refining both the technical and pedagogical attributes for the future intelligent educational environments.The use of our laboratory classroom, the iClassroom, allows us to design and test, in a real space and with real students the efficacy of our technical and pedagogical designs.This space affords the opportunity to refine and redefine what intelligent spaces look like and how they can best be used to maximize positive learning outcomes.
We recognize that this is a nascent area of study and far too large for a single laboratory to undertake.With that, we call on our peers to contribute to the design and development of future intelligent environments by helping us build on current knowledge and collaboratively redesigning the spaces in and methods with which we learn.
As we begin to better understand the affordances and iteratively create design guidelines, our hope is that eventually a prescriptive framework emerges that informs both the practice of technical development and also the deliberate incorporation of technologies into both the learning space and the pedagogy through which students learn.In this paper we have given concrete examples of two embryonic frameworks, the 4DLAT and the PVM, which are currently being deployed in the iClassroom and that incorporate some of these ideas within them.Also, at a wider level, we hope to have demonstrated how our research is addressing the core themes of understanding affordances, structuring experiences, and creating constructivist, collaborative processes, in mixed-reality smart environments.While recognizing the separate nature of embedded technology and pedagogical design, and simultaneously considering both in the development of future intelligent learning environments, we believe will lead to more engaging, effective and rewarding learning experience for our students.

Figure 1 .
Figure 1.3D model of the iClassroom

Figure 2 .
Figure 2. Structured learning activities in a 3D world

Table 1 .
The use of technology in education