A study of the effects of computer animation on college students ’ learning of Leadership in Energy and Environmental Design-LEED

This paper presents ongoing research aimed at investigating the efficacy of computer animations in improving college students’ learning of building sustainability concepts and practices. The use of animations in educational contexts is not new, however scientific evidence that supports their effectiveness as educational materials is still limited. This paper reports an experiment that explored the impact of an educational digital animation, called “LEED-ERS”, on college students’ learning of Leadership in Energy and Environmental Design (LEED) rating system. Specifically, the animation focused on the LEED category of Sustainable Site. Results of a study with 68 students show that viewing the animation led to an increase in subjects’ declarative knowledge by 15%. Compared to traditional learning methods (e.g. reading assignments with static images), viewing the animation led to significantly higher declarative knowledge gains.


Introduction
The built environment has a significant impact on the environment and the society in which we live.Buildings are resource intensive and consume a significant proportion of society's valuable and limited reserves of raw materials, energy and water.In 2010 buildings and their operation were estimated to use 42% of the country's annual energy consumption [1].Design and construction of buildings that reduce the consumption of nonrenewable resources in their construction, operation or maintenance will have a profound impact on the environment and ultimately on the society in which we live and work.In response to the need for a sustainable built environment, the United States Green Building Council (USGBC) has established the Leadership in Energy and Environmental Design (LEED) rating system, which includes a set of metrics for defining and measuring "green buildings".Construction professionals and designers are recognized as LEED AP (Accredited Professionals) if they can demonstrate, via the LEED exam, knowledge and understanding of the LEED rating system.
As engineers, architects and construction professionals are educated for the future, the complex area of building sustainability and LEED must be included in the curriculum.The engineering community acknowledged in 2012 that sustainable construction is an important topic by adding three sustainable construction experts to their editorial board to manage a new sustainable construction specialty area for the American Society of Civil Engineers' (ASCE) Journal of Construction Engineering and Management [2].Education to prepare students for sustainable construction practice has also been a subject of discussion at every annual conference of the Associated Schools of Construction (ASC) for the last decade.Over the past seven years this international association of college undergraduate and graduate educators has published on average four papers each year in their peer-reviewed proceedings covering some aspect of LEED certification and the teaching of sustainable building principles.This reflects 5-7% of papers each year.Several of the papers describe research reflecting the high value that both construction professionals and construction educators place on preparing students to receive credentials as LEED professionals.Despite this support, the literature shows no clear consensus on teaching methods or curriculum design for teaching and learning building sustainability and LEED.
The work reported in the paper is in response to the need to develop innovative and effective instructional methods for building sustainability and LEED education.The choice of a computer animation as the educational approach was motivated by two considerations.First, it was motivated by the need to incorporate traditional pedagogy with new paradigms that reflect our times -our students live in an information age where computer animations and videos are an intrinsic and ubiquitous part of how they live and learn.Second, as differing opinions about the educational efficacy of animations exist, there is a need to further investigate the educational potential of computer animation.Some research studies suggest that animations can be effective educational tools [3; 4], while others show contrary evidence [5] or limit the instructional efficacy of animations to certain conditions [6].The study reported in the paper advances knowledge in this area.

Leadership in Energy and Environmental Design -LEED
The USGBC began development of the LEED rating system in 1995; its mission is to accelerate the implementation of green-building policies, programs, technologies, standards, and design practices.Funded by the U.S. Department of Energy, the LEED standard was developed by USGBC members on a voluntary basis and is now one of the rating systems accepted worldwide [7].
LEED certification is the recognized standard for measuring building sustainability.USGBC is the committee that approves or denies the building's certification, and projects must pass the LEED technical review to achieve certification.LEED standards provide a straightforward checklist, and the 'greenness' of buildings is evaluated according to these criteria.The number of points received defines the building's LEED rating (e.g.certified, silver, gold, platinum) and LEED points are earned in seven categories: Sustainable Site (SS), Water Efficiency (WE), Energy and Atmosphere (EA), Materials and Resources (MR), Indoor Environmental Quality (ID), Innovation in Design (ID), and Regional Priority (RP) [8].LEED can be used for different types of projects such as new construction (LEED-NC), existing buildings, commercial interiors, core shell development, retail outlets, schools, homes, neighborhood development, and health care.The points and credits of the LEED rating system differ based on the category of the building.In the study reported in the paper, the LEED-NC was used as the reference LEED addenda for the design of the animation and for the content of the reading assignment.LEED-NC covers commercial, institutional, industrial, and large residential buildings (four stories or higher).
In order to pass a LEED professional credential exam (e.g., to become a LEED Green Associate or LEED AP or LEED Fellow) students are required to attend lectures (webinars, online courses or live classes), read lengthy text-based materials and memorize large amounts of information.One of the objectives of the work reported in the paper is to improve current LEED teaching/learning methods through the use of effective computer tools.

Computer Animation and Learning
Researchers have studied the influence of computer animations on students' learning through various experiments and have achieved mixed results.While some experiments recognize animation as an effective learning tool, e.g.[9; 10; 11], other studies suggest that animations can be distracting with little or no positive effect on learning, e.g.[12; 13].
Those researchers who believe computer animation is an effective instructional device argue that the main strength of animation is the fact that it can represent information in different codes: images, words (text or narration) and motion [14]; several codes are more influential than one in learning.Moving graphics can attract the viewers and attain audience motivation [15], and animation can play an effective instructional role by engaging the viewer, guiding attention, representing motion-related knowledge, and explaining complex concepts in simple steps [16].Another benefit of animation is its ability to explain a dynamic and changing process.Through animations, students can develop better mental pictures of an activity or a procedure, which leads to a higher understanding of complex concepts [14; 17; 18].
Several studies that support computer animation as an effective teaching/learning tool for higher education can be found in the literature.Marini and Genereux [19] and Haskell [20] studied the effects of animation on transferring knowledge and learning science and technology concepts.Findings from their studies showed that the use of digital animations increased students' abilities in generalizing their knowledge and applying it to unfamiliar situations.In a study by Mayer and Sims [21], students' performances in mechanical engineering increased because of the use of animation as a

EAI European Alliance for Innovation
A study of the effects of computer animation on college students' learning of Leadership in Energy and Environmental Design -LEED supplementary instructional material.A study by Ferk, Vrtacnik, Blejec, and Gril [3] revealed that representing molecular structures through 3-D animations provided students with a better understanding of these structures in comparison to presenting them with static graphics and text.Khalil, Johnson, and Lamar [4] studied the effects of digital animations for learning medical concepts; findings showed that students in medical-related majors found animations more effective for learning biology concepts and processes than static images.A study by Rias & Zaman [22] showed that the combination of text with 3-D animation is more effective for learning in general, especially for recalling information, than text alone or its combination with 2-D animation or still images.
Animations are also being used in K-12 and informal education to teach a variety of topics ranging from ecology, biology, health, technology, finance, geology, and more [23; 24; 25].Several studies confirm their educational effectiveness [26].Falvo [27] reports that animations can assist K-12 students to better understand dynamic molecular processes in chemistry and biochemistry.However, instructional use of animations and visualizations must be accompanied by pre-and postexplanations and discussions to address possible misrepresentations.Solid foundational (prior) knowledge is also important, as it prepares students to learn and retain structural and process concepts conveyed by animations.A study by Adamo-Villani and Kelly [28] showed improved children's motivation and performance in mathematics as a result of interacting with 3D animations.
Another study by Adamo-Villani demonstrated the effectiveness of animated avatars for teaching mathematics and sign language to deaf and hearing elementary school children [29].
As previously mentioned, a few experiments that do not support the efficacy of computer animation as a teaching/learning tool can also be found in the literature.A study by Owens [5] showed a decrease in students' performance when animation was used as a supplementary material to traditional instructional methods.Kuhl et al. [30] compared efficiency of learning using static versus dynamic instructional tools and observed no difference.Gerjets and Scheiter [31] and Paas et al. [32] argue that 3-D animations lead to a cognitive overload and, subsequently, a decrease in learning.Tversky et al. [14] argue that animations may be ineffective because they violate the second principle of good graphics, the Apprehension Principle, according to which graphics should be accurately perceived and appropriately conceived."Animations can be too complex or too fast to be accurately perceived.…..Animations may be more effective than comparable static graphics in situations other than conveying complex systems, for example, for real time reorientations in time and space".
In summary, literature shows no consensus on the potential benefits and drawbacks of digital animation as a learning tool.Furthermore, many experiments that investigated the educational efficacy of animation have caveats.Some studies did not compare animations with equivalent information conveyed in static graphics, therefore it was not possible to determine if the difference in learning was due to the use of images or the use of moving images.Other studies have not used equivalent procedures, for instance the animation condition allowed interactivity while the static condition did not, so that benefits may be due to interactivity rather than animation (e.g.[33]).In some experiments, the successes of animation seem to be due to advantages in extra information conveyed or additional procedures, rather than the animation of the information per se.
The study reported in the paper attempted to overcome many of the limitations of previous studies, and shed light on the educational potential of digital animations for students' learning of LEED.

The "LEED-ERS" animation: design and development
"LEED-ERS" is a 12-minute-long computer animation illustrating the eight credits of the Sustainable Site category of LEED.Although the animation is not segmented in a way that allows students to select a specific credit to watch, the separation of credits is clearly defined.Each credit starts with a seven-second introduction, where the name of the credit and its related LEED code are introduced (Fig. 1).In the second part, the content of the introduced category is explained both graphically and through text (Fig. 2).At the end of each credit, the number of received points (or possible points) is shown both in text format and via visual tools (Fig. 3).The text included in the animation is based on the LEED Addenda-2009 version.The animation was created in Adobe After Effects and the graphics assets were produced in Adobe Photoshop and Illustrator.

Instructional Design
The instructional design of LEEDE-ERS was guided by the 12 principles for effective use of animation in multimedia learning developed by Mayer and Moreno [34] and Mayer [35; 36].LEED-ERS adheres to 7 of these principles, 3 of the principles could not be implemented because not applicable to the animation.Specifically, LEED-ERS implemented the following principles: 1. Multimedia Principle -Learners achieve higher learning when they receive information from more than one media source at the same time.Concurrency of moving images with narration, either text or audio, increases learning.LEED-ERS presents information from several media sources at the same time: moving images, text, diagrams.
2. Temporal Contiguity Principle -The concurrent presentation of graphics and narration increases learning.
Based on the Cognitive Load Theory [34] the presentation of material separately in time imposes an extraneous load on a learner's working memory.Individuals need to keep the information active for a longer time, receive and process the new information, and mentally integrate the two.This process reduces learning efficiency.In LEED-ERS the students are receiving images and related narration simultaneously.
3. Spatial Contiguity Principle -This principle emphasizes presentation of images and text in a contiguous space.The logic behind this principle is the same as the previous principle, except that this principle refers to space rather than time.As shown in figures 2 and 3, in the LEED-ERS animation text is always contiguous to the images and in the same location on the screen.
4. Redundancy Principle -Presenting a single source of nonverbal information (e.g.moving images) and a single source of verbal information (e.g.text or audio) is better than presenting two verbal information sources simultaneously with nonverbal information [37].In LEED-ERS we chose text as the only verbal source of information.
5. Signaling Principle -Learning is enhanced when key points and the structure of the lessons are highlighted via cues (to draw attention).Signaling helps learners find the key elements that should be paid more attention to.
Various signaling techniques have been implemented in LEED-ERS including: highlighting different areas of the screen with different colors and textures, using arrows to point to certain image sections or text segments, displaying or hiding icons at certain key points in time.Figure 4 shows an example of application of the signaling principle.
6. Segmenting Principle -Learning is promoted when instructional materials are segmented and learners have control over the pace of presentation.Students may become confused with continuous presentations, since they must make mental connections to integrate information from presented multimedia [38].Although segmentation in LEED-ERS is not under the learners' control, the content of the animation has been segmented according to the structure of the LEED system.The eight credits of the LEED first category and its sub-credits are separated in LEED-ERS.Each segment starts with the name of the credit and its codes; then the explanation of that credit is presented, and the end of the segment is marked by a box showing the received number of points for that credit.In addition, although LEED-ERS is a noninteractive animation, students can control the playback speed and have access to standard stop-play-rewindforward controls.
7. Pre-training Principle -Learning progresses when learners have a preview of the teaching materials' key parts' names and characteristics.This preview helps students form a better mental picture of what they learn.LEED-ERS is designed to be used as a supplementary material to traditional lectures, not as the main teaching tool.The lecture plays a pre-training role and then students view the animation.In addition, the first segment of the animation presents a summary of the topics covered.

Formative Evaluation
The animation was created using an iterative development approach that included continuous feedback from a panel of experts and a small group of target users.The panel of experts included 2 faculty in Animation, 2 LEED APs, and 2 faculty in Instructional Design; the group of target users included 6 undergraduate students, 3 from Civil Engineering and 3 from Building Construction Management.At different stages of development, the experts and target users were asked to view the animation and provide feedback via online surveys.The surveys included five-point Likert scale rating questions and openended questions to provide additional feedback.The experts performed an analytical evaluation of the elements of the animation that pertained to their area of expertise.The two experts in animation assessed the overall design of the animation, screen composition, timing and pacing, quality of graphic assets and readability of text; the experts in LEED evaluated the accuracy and completeness of the educational content; the experts in instructional for Innovation A study of the effects of computer animation on college students' learning of Leadership in Energy and Environmental Design -LEED design evaluated the design and clarity of the presentation.The target users were asked to provide feedback on perceived usefulness of the animation as a learning tool; usability; clarity of the educational content; and appeal of the graphics.The final version of the animation received high ratings from the experts in each area: overall design of the animation (MEAN=4.5;1=low quality, 5=high quality), screen composition (MEAN=4.5),timing and pacing (MEAN=5), quality of graphic assets (MEAN=5), readability of text (MEAN=4.5),accuracy of educational content (MEAN=5), completeness of content (MEAN=5), instructional design (MEAN=4.5),clarity of presentation (MEAN=4.5).The ratings from the target users were high as well: perceived usefulness of the animation as a learning tool (MEAN= 4.5, 1=low; 5= high); usability (MEAN=5); clarity of the educational content (MEAN=4.5);appeal of the graphics (MEAN=5).

Summative Study
The goal of the study was to provide an indication of whether using the educational animation LEED-ERS affects students' learning of LEED.Learning is a multidimensional construct that includes cognitive, metacognitive, and motivational components [39].Our experimental study focused on one cognitive aspect, e.g.declarative knowledge.Declarative knowledge is defined as the ability to memorize and recall information.In particular, the study aimed to assess the influence of the animation on student ability to demonstrate knowledge and understanding of the "Sustainable Site" LEED category.We measured this learning objective using pre and post educational intervention competency testing.The specific questions that the study aimed at answering were: 1.Do supplementary materials (e.g. computer animations and reading materials comprised of text + static images) increase students' knowledge of LEED?
2. Do learning gains differ between students having a curriculum supplemented with animations versus reading materials comprised of text+static images?
3. Does students' primary knowledge of LEED have an effect on learning gains in both conditions (lecture supplemented with animation versus lecture supplemented with reading materials)?

Participants
Sixty-eight (68) students enrolled in Building Construction Management (BCM) courses in the College of Technology at Purdue University.

Stimuli
The stimuli of the study included: the LEED-ERS animation, the reading assignment, and the lecture.The animation is described in detail in section 3. The reading assignment included the same text content presented in the animation (20 pages based on the LEED Addenda 2009) and a series of static images corresponding to the main keyframes of the animation.The 30-minute lecture focused on LEED sustainable site and was delivered by one of the faculty in BCM.

Procedure
All students received a pre-test on LEED -sustainable site category; the goal of the pre-test was to assess students' pre knowledge of LEED.After the pre-test, a randomized complete block design was used to divide the subjects into two groups with similar pre-knowledge.In other words, we used the pre-test scores to group individuals in terms of pre-knowledge and then made sure these groups of individuals were equally assigned to the two intervention groups, e.g. group A (control group), reading assignment versus group B (experimental group), animation.One week after the pre-test, all students attended a 30-minute lecture on LEED sustainable site.Then all students took an after-lecture test.Upon completion of the after-lecture test, the students in group A (control) were directed to one of the BCM labs to read the file containing text and images for 45 minutes.The students in group B (experimental) were directed to another lab to watch the LEED-ERS animation for 45 minutes.Lastly, both groups took the final post-test.

Testing instruments and test distribution
As mentioned in the previous section, students were administered 3 tests: a pre-test, used to determine the participants' primary knowledge of LEED; an afterlecture test, used to determine the students' change of knowledge due to the lecture; and a final test used to determine the students' change in knowledge due to the supplementary materials (e.g. the animation or the reading assignment).The comparison between the pre-test and the final test scores enabled us to determine which supplementary material (animation versus reading assignment) leads to higher learning gains.Comparing the after-lecture test with the final test scores allowed us to determine the students' knowledge gain due to the lecture versus the knowledge gain due to the supplementary materials.Although the contents the three tests were the same, the questions were not identical.In order to overcome the problem of potential differences in difficulty level among the 3 tests, two versions of the tests were designed, TA and TB.At the after-lecture testing stage, 50% of the participants in each group took TA and the rest TB.At the final testing stage, those who had taken TA took TB and vice versa.The Pre-Test was a random combination of TA and TB. Figure 5  The testing instruments were adapted from materials designed by the US Green Building Council (USGBC) to help individuals as they prepare to either register a building for LEED certification or prepare to take the certification exam as a LEED Accredited Professional (LEED AP).The primary resources for these materials are the LEED Online Documentation Requirements and review and practice questions from the USGBC LEED AP Building Design + Construction Study Guide.

Descriptive Data Analysis
Results show that subjects increased their declarative knowledge by 13% after listening to the lecture; we refer to this increase as "lecture effect".Subjects increased their declarative knowledge by 15% after watching the animation (group B), whereas reading the text-based assignment led to a slight (-1%) decrease in participants' declarative knowledge (group A).We refer to the change in subjects' declarative knowledge due to the supplementary materials as "the experiment effect".Group A (control) showed an overall increase in declarative knowledge by 13% after listening to the lecture and reading the text assignment; group B (experimental) increased their declarative knowledge by 28% after listening to the lecture and viewing the animation.We refer to the change in subjects declarative knowledge due to lecture + supplementary material as "intervention effect".Table 1 reports Median, Mean and Standard Deviation calculated for the pre, after lecture and final test scores.Table 2 reports the differences in average and STD calculated for the test scores in points and percentages.

Statistical Data Analysis
The statistical analysis answered the three research questions of the study.

1.Do supplementary materials (e.g. computer animations and reading materials comprised of text + static images) increase students' knowledge of LEED?
Within group paired t-tests were performed to determine whether the "lecture effect" "experiment effect" and "intervention effect" were statistically significant.For group B (experimental) the "lecture effect" "experiment effect" and the "intervention effect" were statistically significant (p value= .048;p value= .037;and p value= .000respectively).These findings show that the lecture by itself, the animation by itself and the educational intervention comprised of lecture + animation improved students' declarative knowledge of LEED significantly.Findings also show that knowledge improvement due to the animation alone was higher than knowledge improvement due to the lecture alone.For group A (control) the "lecture effect" was statistically significant (p value =.047), the "experiment effect" was not statistically significant (p value = 0.43), while the overall "intervention effect" was statistically significant (p value = .023).These findings show that the lecture alone and the educational intervention comprised of lecture + reading assignment improved students' declarative knowledge of LEED significantly, whereas the reading assignment by itself did not improve students' declarative knowledge of LEED.

Do learning gains differ between students having a curriculum supplemented with computer animations versus reading materials comprised of text + static images?
Two pooled t-tests were performed to determine whether the difference between the two groups' "experiment effect" and "intervention effect" was statistically significant.In order to decide whether to run Equal-Variance t-tests or Unequal-Variance t-tests, two f-tests were performed to determine whether (1) the "experiment effect" data for group A and for group B had the same variance and (2) the "intervention effect" data for group A and group B had the same variance.High pvalues for the two f-tests (p=0.916 for (1) and p=0.625 for (2)) justified the use of Equal-Variance pooled t-tests.

EAI European Alliance for Innovation
A study of the effects of computer animation on college students' learning of Leadership in Energy and Environmental Design -LEED The results of the two equal-variance pooled t-tests showed that there is a statistically significant difference between the "experiment effect" for group A (control) and the "experiment effect" for group B (experimental) (p value =0.043), with the experiment effect being higher for group B. There is also a statistically significant difference between the "intervention effect" for group A (control) and the "intervention effect" for group B (p value =0.049), with the experiment effect being higher for group B. These findings demonstrate that watching the animation lead to significantly higher learning gains than reading the text-based assignment, and that an educational intervention comprised of lecture + animation is more effective than an educational intervention comprised of lecture + reading assignments with text and static images.3. Does students' primary knowledge of LEED have an effect on learning gains in both conditions (lecture supplemented with animation versus lecture supplemented with reading assignment comprised of text + images)?Two regressions of the "lecture effect" and the "intervention effect" on the pre-test scores were run to determine whether participants with different preknowledge of LEED would be affected differently by the supplementary materials (e.g.animation and reading assignment).Results of the regressions show that the pretest score and score improvement are negatively related.This means that lower pre-test scores correspond with greater improvement scores.In Table 3, lecture and intervention effects have been regressed on the pre-test scores.Significant negative coefficients show that the lecture and the total intervention have lead to higher learning gains for participants with lower pre-test scores (novices).

Summary of results and discussion
Findings from the study support the effectiveness of computer animation as a learning tool; specifically the results of the statistical data analysis show that: -The educational animation by itself improved students learning of LEED significantly -The learning gains due to the animation were higher than the learning gains due to the reading assignment -The educational intervention comprised of lecture + animation improved students learning of LEED significantly -The learning gains due to the educational intervention comprised of lecture + animation were higher than the learning gains due to the educational intervention comprised of lecture + reading assignment -The animation alone lead to higher learning gains than the lecture alone -Pre-test score and score improvement are negatively related, e.g., lower pre-test scores corresponded with greater improvement scores.Thus, students with low preknowledge of LEED benefitted from the animation the most.
The main challenge faced during the design of the study was to make sure that the two supplementary materials (animation and reading assignment) were equivalent in terms of conveyed information and procedures.In our case, the animation presented the same text as the reading assignment; the static images included in the reading assignment were keyframes extracted from the animation, and the animation was non-interactive.Because of this equivalency in content and procedures, the study is valid and proves that the educational efficacy of the animation is due to the animation of the information per se, rather than to advantages in extra information conveyed or additional procedures.However, the study presented one limitation: the relatively small sample size.Because of the limited number of participants we cannot generalize the results and we can only claim that the animation shows promise of being an effective instructional tool.In order to build stronger evidence of the learning effectiveness of the animation additional studies with larger pools of participants and in different settings will need to be conducted.

Conclusions and future work
In this paper we have described the design, development and initial evaluation of an educational animation for teaching/learning LEED.An experimental study of the influence of the animation on students' content learning provided preliminary evidence of the effectiveness of the animation as a learning tool.Watching the animation lead to a statistically significant increase in subjects' declarative knowledge; compared to traditional learning materials (e.g.reading assignments with text and static images), watching the animation lead to significantly greater knowledge gains.
The results of the study are important as they add to the growing body of evidence that suggests that animations can improve students learning in engineering, construction, and architectural disciplines.Future work will focus on developing additional animation sequences and conducting a more rigorous evaluation of the educational effectiveness of the animation with larger groups of subjects and in different settings.If future studies confirm the pedagogical efficacy of the animation, expanding the animation approach to other engineering/construction/architectural concepts as well as other subject domains seems to be a logical step in which to proceed.
Future iterations could also be expanded to provide a mechanism to assess sustainable building knowledge beyond the specific examples within the animation to a more general sense of the domain.If we are able to show a strong correlation between animation and student attitudes and performance in the classes in which it will be used, we expect to extend the tasks to other courses, as well as broadening usage in the target courses.

EAI European Alliance for Innovation
A study of the effects of computer animation on college students' learning of Leadership in Energy and Environmental Design -LEED

Figure 4 .
Figure 4. LEED-ERS' screenshots illustrating the application of the "Signaling Principle".Areas of interest are highlighted with different colors and textures, arrows are used to point the viewers' attention to key locations on the screen; icons appear and disappear to emphasize certain requirements.

Figure 5 .
Figure 5. Chart illustrating the distribution of the 3 tests

Table 1 .
Summary of descriptive data analysis: Median, Mean and Standard Deviation calculated for the pre, after lecture and final test scores.

Table 3 .
Regression of Lecture and Intervention effects on the Pre-Test score.Significant negative coefficients show that the lecture and the total intervention have lead to a higher score improvement in the participants with lower pre-test scores.