Evaluating Moodle use via Smart Mobile Phones . A case study in a Greek University .

The use of learning management systems (LMS) has grown considerably in universities around the world. LMSs can offer a great variety of channels and workspaces to facilitate information sharing and communication among participants in a course. One of the most commonly used is Moodle, a free learning management system enabling the creation of powerful, flexible and engaging online courses and experiences. A course can consist of a number of lessons. Each lesson consists of reading materials; activities such as quizzes, tests, surveys, and projects; and social elements that encourage interaction and group work among students. This study investigated how often students used a mobile phone to access various activities on Moodle. The students’ point of view is important since they are the main users of the offered teaching technique and can cooperate in implementing and improving an e-course as a very important stakeholder in the e-learning process. A survey on self-reported usage was filled by 122 university students in a course offered by the faculty of Preschool Education at the University of Crete. Follow-up interviews were conducted to solicit students’ perceptions on mobile access to Moodle and the underlying reasons. The results show significant differences in students’ usage of various Moodle activities via mobile phones. Students’ responses also suggest that Moodle is used merely as an electronic document repository and not as an effective learning tool due to the limitations of mobile access on usability and reliability.


Introduction
The days of mimeograph machines and chalkboards has long past [27].Information and communication technology (ICT) is increasingly becoming a bigger and more important part of students' everyday life [22,30,33,35,36].Students can increase their learning skills using information technology.With the rapidly increasing popularity of the Internet in recent years, the delivery of learning programs has gradually shifted from local desktop to online-based applications [5].Consequently, the role of e-learning has transformed completely as recent advances in Information Technology (IT) and the advent of Web 2.0 technologies enabled the creation of learning content that is no longer based on textbooks and learning guides [32,33,34,35].E-learning is electronic learning, meaning that it is not traditional learning in a classroom with a teacher and students, plus the board.Elearning involves using a computer to deliver classes or a course [19].The traditional idea of "classroom" now incorporates the use of both physical and virtual space [44].Students who have access to an E-learning system can get course materials in different formats (text, image, Stamatios Papadakis et al. 2 sound, video, QR-codes, etc.), and can interact with their colleagues and lecturers individually and simultaneously via message boards, forums, chat rooms, videoconferencing, etc. [41].
A significant trend in universities has been to implement so-called learning management systems (LMS) (Moodle and other web-based learning systems), which are used as a common platform where students and teachers can interact digitally [8,43,45].LMSs can offer a great variety of channels and workspaces to facilitate information sharing and communication among participants in a course [40].
In the current market space there are many commercially available learning management systems from which to choose.The open source community has also been active in creating alternative learning management system choices that are free of licensing costs [27].Currently the most popular open source learning management system is Moodle.Since its launch in 2002, Moodle has become the benchmark that every learning management system is measured against [6].The social constructionist learning philosophy is at the heart of Moodle [10].Moodle is a free learning management system that enables you to create powerful, flexible, and engaging online learning experiences.
In recent years, instructors have been concerned about mobile devices as the new media for learning content delivery, the collaboration between the members of the educational community, at primary, secondary and tertiary levels of education [18,25,30,33,36,37,38,46]. In this way, they go along with the new generations of students that were 'born with smartphones' and are very familiar with the latest technologies [3,30].It is not uncommon to see university students, as well as those in secondary school, using smartphones to access learning resources on Moodle and other LMSs [20].However, despite the increase in LMS adoption in universities, concern has been expressed as to whether LMSs are being used as effective learning tools or merely as electronic document repositories [7,8].The findings of various researchers indicate that while many university staff worldwide use LMSs to some extent, a large majority fail to make use of the potential pedagogical advantages offered by the full functionality of the software [4].[43] state that LMSs are still inadequate in supporting the level of interaction, personalization, and engagement demanded by the tech-savvy students.
Students' perceptions of an LMS is mainly influenced by how the system correlates with their educational needs and expectations [43].Research results confirm that the LMSs are used more as electronic document repositories than as active learning tools and students are much more likely to assume a passive role rather than becoming active players [4,7].In a research in the University of Minho (UM) Portugal, researchers found that the students seem to appreciate the contribution of an LMS to their learning, viewing it as a complement rather than a substitute for classroom activities [7].[20] in a similar study found that students did not prefer using their mobile phones to access Moodle, due to the limitations of mobile access on usability and reliability.In terms of Moodle activities, it was found that students preferred carrying out easy and low-stake Moodle tasks on their mobile phones, such as accessing learning materials.
The current research aims at examining how students use Moodle via smart mobile devices to carry out different Moodle activities and the possible reasons behind such usage patterns.

Mobile Learning and LMS
An LMS is a tool that performs among other functions: mediation of knowledge appropriation, administration of such mediation, access to educational and communication tools [16].An LMS not only provides academic institutions with efficient means to train and teach individuals but also enables them to efficiently codify and share their academic knowledge [1].As a result, the state of the eLearning market globally continues to shift, grow, and evolve.According to [12], global revenues for selfpaced eLearning (LMS, authoring tools, packaged content, and services) reached $46.6 billion in 2016.In 2004, the eLearning market was worth more than US$18 billion [21].The global learning management system (LMS) market is expected to grow at a CAGR of 24% during the 2016 to 2020 period [12].Currently, there are many brands of web-based learning systems, for example, WebCT, Moodle, OLAT, and Sakai [45].According to literature, there are three LMS generations (see Figure 1) [42].

Figure 1. The three LMS generations
When deciding on the introduction of an LMS and when measuring its effectiveness and usefulness, many factors must be taken into account [3].Successful implementation of LMSs is not an easy step and depend not only on providing training and support for instructors but also on the level of student active engagement and student and instructor satisfaction with the LMS used [7].Motivation and positive perceptions and attitudes of tutors and students toward e-learning and digital literacy should be taken into consideration for successful e-learning adoption [2].The students' point of view is very important since they are the main users of the offered teaching technique and can cooperate in implementing and improving an e-course as a very important stakeholder in the e-learning process [3].There are three e-learning modes of engagement, which correspond to growing levels of complexity and depth in LMS usage (see Figure 1) [15].A cross-institutional study revealed both institutional and cultural barriers for e-learning to transform, rather than sit uncomfortably alongside, institutional practice [17].Previous studies had found that most universities were at mode 1 of the Francis and Raftery model [7].
Similarly, as for the level of engagement to which the LMSs are used, studies had found that the vast majority of students seem to experience no higher engagement than Francis and Raftery mode 1 [7,20].Moodle is a course management system for online learning.Created by Martín Dougiamas, a WebCT administrator, its design is based on collaborative learning, in which a teacher creates a student-focused environment that helps them to build up knowledge based on their own knowledge and skills, instead of simply posting and sending them information that the student is supposed to learn [41].In 2018, Moodle is being used in 232 countries, and is translated into 78 languages, with 132 million users, of which 2 million are teachers [29].
The acronym MOODLE stands for Modular Object-Oriented Dynamic Learning Environment [5].

As we can read from the Moodle website "The word Moodle is an acronym for Modular Object-Oriented Dynamic Learning Environment, which is mostly useful to programmers and education theorists. It's also a verb (in the bigger English dictionaries) that describes the process of lazily meandering through something, doing things as it occurs to you to do them, an enjoyable tinkering that often leads to insight and creativity. A sort of cross between "muse" and "doodle". As such it applies both to the way Moodle was originally developed, and to the way a student or teacher might approach studying or teaching an online course in an ongoing, iterative way. Anyone who uses
Moodle is a Moodler."[29].
It is an alternative to proprietary commercial online learning solutions and is distributed free under open source licensing [40].Thus, an organization or an individual has complete access to the source code and can make changes if need be.It began as a way for colleges and universities to offer on-line courses but is now being used by educators at all levels.Moodle can be used on different devices, at different times, in different places.Devices change in many ways, not only in size but also in the way they display our Moodle course.Moodle courses can be used on anything from a tiny device that fits into the palm of a hand to a huge IWB or smart TV, and plenty of other devices in between [19].
Moodle is a free learning management system that enables you to create powerful, flexible, and engaging online learning experiences.As [10] explain: "The phrase "online learning experience" connotes a more active, engaging role for the students and teachers.It

connotes web pages that can be explored in any order, courses with live chats among students and teachers, forums where users can rate messages on their relevance or insight, online workshops that enable students to collaborate and evaluate each other's work, impromptu polls that let the teacher evaluate what students think of a course's progress, directories set aside for students to upload and share their files. All of these features create an active learning environment, full of different kinds of student-tostudent and student-to-teacher interaction."
Moodle has grown into a mature, sophisticated, and complex software system and covers a wide range of topics [6].In Moodle, all the available information within a course (for a specific subject) is organized in separate blocks.The first block is always devoted to general and administrative information, as well as support contents about the course: timetables, teachers, news, forums, chats and so on.The remaining blocks can be organized in different ways [28].
Moodle can be downloaded, installed and run on any web server software using Hypertext Preprocessor (PHP).It can support a SQL database and can run on several operating systems [19].We can download the latest version of the Moodle from the following URL: https://download.moodle.org/.MoodleCloud, on the other hand, does not need to be downloaded since, as its name suggests, is in the cloud.Therefore, we can get our own Moodle site with MoodleCloud within minutes and for free.It is Moodle's hosting platform, designed and run by the people who make Moodle.In order to get a MoodleCloud site, we need to go to the following URL: https://moodle.com/cloud/.[19].MoodleCloud was created in order to cater for users with fewer requirements and small budgets.In order to create an account, you need to add your cell phone number to receive an SMS which we must be input when creating your site.As it is free, there are some limitations to MoodleCloud, unless we contact Moodle Partners and pay for an expanded version of it [19].

The LMS
The open-source LMS Moodle, (http://moodle.org/)can be found in many segments of education and higher education is no exception.Its popularity, except for the fact it is free, is mainly based on its flexibility, adaptability and the possibility of personalization while, on the other hand, the system contains many standard features which make the learning process easy to implement [3].The University of Crete (Faculty of Preschool Education) has adopted Moodle (version 2.9) as one of its official LMS.One course of one instructor was selected for this study.The instructor was in the faculty of Preschool Education and the course was entitled "Introduction to the Informatics II".
The scope of the course was for preservice kindergarten students to learn on how to use ScratchJr for educational purposes.The instructor used Moodle in multiple roles: as a repository of teaching materials, a platform for making course announcements and as discussion forums for student-student and studentinstructor interaction.Also, the students could submit assignments, take quizzes, conduct group projects and receive feedback from the instructor.Although there is a mobile app for Moodle, it cannot be used directly by the students' due to University security policy.The students had to use their preferred browsers in their mobile devices to access Moodle.

The Sample and the Procedure
This study adopts a mixed method with survey and interview data collected and analyzed.The study design was based on the work of [7] to get comparable results.The participants were 3rd-year students.Although the study targeted about 160 respondents, a total of 122 valid submissions were obtained, (76 % response rate).After the online survey data were collected, the students were voluntarily asked to participate in the interviews.25 students took part in the follow-up interviews.All interviews were conducted face-to-face.
A questionnaire asking about the experience of using Moodle was used for collecting quantitative data.It included two parts: demographic information and frequency of course Moodle use.Part 1 asked for basic demographic information as well as their experience with Moodle and self-perceived IT competency level; Part 2 asked about the frequencies of using different categories of Moodle activities with variables in a 5-point Likert scale: ranging from 1 (never) to 5 (several times a day).Some of the questionnaire questions are as follows: "What did you usually do when you access the Moodle of this course via your phone?", "Did you have any difficulties in using the Moodle of this course using your mobile phone?If yes, what were they?", etc.

Questionnaire Responses
Table 1 shows the statistics of student self-reported usage of Moodle via mobile phones.Access to learning materials was the most frequent activity, while interacting with instructors and other students were the least frequent one.Experience of using Moodle may have affected students' usage of Moodle via mobile access.We used Kruskal-Wallis tests as a non-parametric equivalent of the ANOVA control.The tests revealed that students with different Moodle experience reported significantly different usage frequency (p < .05).Follow-up pair-wise tests revealed that students with "2 years' or more" experience with Moodle reported higher frequencies than those with "less than 1 year'" or "1 year to less than 2 years'" experience.Besides, a difference in the frequency of using Moodle via mobile phones across IT competency was also analyzed.As a statistically significant difference of access frequencies in interaction and collaboration activities was found (p < .05) a follow-up pair-wise test was used by the researchers.It was found that students who rated themselves as "competent" reported significantly more frequent access than those who rated themselves as "not or somewhat competent" or "less competent."There was no significant difference between other pairs of IT competency values.These results are completely different compared to the work of [5].As all the students of the Department of Preschool Education were females in this study, there was no gender-based statistical analysis.

Interview Responses
The semi-structured interviews were conducted after a regular university day.A separate isolated room was used to avoid distractions and outside influences.All students interviewed answered that they used mobile phones to access the Moodle of their courses, as mobile phones allowed them to access the LMS at any place and anytime.Mobile access also enabled them to read announcements, comments, and feedback as soon as they were available online.At the same time, they also stated that they preferred content to be more well-organized and clearly presented in comparison with the LMS access via desktop PCs, due to the technological limitations of smart mobile devices.For example, some said they preferred to see the necessary announcements as soon as they accessed the application to feel more comfortable with the course content or they reported the inability to find information in a quick and efficient way.Students' tendency in using Moodle for resource depository and information retrieval in this study demonstrated consistency with previous studies on students' perception on Moodle [4,43].Students statements (Table 2) revealed that they see LMS as a complement rather than a substitute for the formal class.

Accessing resources
It may be a convenient option to see changes in the material made by the professor.On the other hand, one respondent expressed a worry that the mobile access stressed her out as she felt she should constantly check her phone for new updates, similar to the 'Facebook-syndrome'.Submitting assignments freedom, as I am not obliged to constantly sit in front of a PC.It's very easy to access, so sometimes you end up sitting at a cafe and checking whether there is anything new.

Interaction
If there was no social media, the system would be important for communication with instructors and other students.I prefer to use other social media platforms, such as students' closed group on Facebook.Collaboration Although I own a mobile device with a large touch screen, I prefer for this type of activities to work on my PC.
In general, students indicated that using mobile phones was not a preferred method to access Moodle.Although there are mobile device monitors with larger screenswhich can render higher resolutions at more readable sizes -students still referred to usability issues such as the screen size.Thus, they would only be comfortable to conduct simple and low-stake tasks using mobile access (see Figures 3 and 4).

Discussion
This study set out to investigate how university students perceive the use of learning management systems.During the years, various attempts to evaluate LMSs and especially Moodle as an effective learning tool have been successfully made.The majority of the studies concluded that LMSs -including Moodle -are inadequate in supporting the level of interaction, personalization, and engagement demanded by the tech-savvy students [4,7,43,45].In fact, many studies have reported that if we want the LMSs to be used less as electronic document repositories and more as active learning tools it is necessary to provide to the new generations of students that were 'born with smartphones' and are very familiar with the latest technologies the right tools to access the educational content.At the same time, at least one study has found that students did not prefer using their mobile phones to access Moodle, due to the limitations of mobile access on usability and reliability [20].Motivated by the above-mentioned studies, the novelty in this paper is that it is focused on the use of smartphones by the students.As many studies show, the success of web-based learning systems will not be achieved if students fail to use such systems [45].Both the survey and interview data indicated that students used mobile phones to access Moodle for learning materials much more often than for other uses, which indicates that the use of mobile access to Moodle was still at the lowest level as suggested in [15].To sum up the approaches which, based on student's evaluation, could make the smartphone experience better and thus push them to use the LMS more often and more efficiently are:  Better organized web content  Better organized web structure (e.g.nonuse of cluttered navigation bar and/or Long drop-down menus)  A powerful, flexible, and engaging online learning experience  Familiarity and experience with mobile technology It should be highlighted that the students didn't use the Moodle mobile application to access web content due to University security policy.If the students were able to use the Moodle official application, they should have a better learning experience.Incomplete system use signifies that the LMS is perceived as a tool with a few isolated functions, not as the 'multi tool' the system it is designed to be [43].Like other studies [7,42,43], the major findings from the results show that students' perception of the LMS is affected by several factors related to social influences, perceived easiness to use and perceived usefulness.Students highlighted that mobile phones present usability and compatibility problems while trying to access websites meant for desktop or laptop computers.Thus, this is one of the main reasons why students do not prefer mobile access for accessing the LMSs.If mobile phones are to be used to effectively access LMSs, the LMSs must be optimized for mobile access.This could happen through the provision of a few selected services by the instructors, with the necessary detail for each service.The design challenge is to optimize the LMS in such a way that the mobile site satisfies at least most of the mobile users' needs for the LMS [42].Therefore, managers and developers of e-learning need to improve the content quality of their e-learning systems to encourage students to use them more extensively [45].Furthermore, the findings of this study, like other studies, emphasize the importance of the LMS delivering quality content to its users, as well as the fact that it is up-to-date, easily available and relevant [1,31,43,45].In contrast to other studies such as the ones reported by [43], the students didn't report that the LMS contained a lot of unnecessary information.
In this study, like those reported by [7], the results suggest that simply the creation of Moodle activities that are designed for interaction and collaboration does not necessarily result in more frequent access to those activities via mobile phones.Besides, instructors and teaching assistants need to be more responsive and more active in facilitating student interactive activities.As [43] indicate the teacher's way of using the system is a major source of influence on how students perceive and use an LMS.
The results also revealed that students who had used Moodle for a longer period tended to use mobile access more often to take tests and collaborate on Moodle than those who had used Moodle for less time.In addition, students with high self-perceived IT competency used more mobile access to Moodle for interaction and collaboration activities.These seem to comply with many studies where experience and IT competency are positively associated with technology usage [9,14,24,26].

Limitations
The main source of bias for this study could be the fact that the first author was the person who designed this study, was the teacher in the groups and collected the endtest data.The evident conflict of interests and potential bias could genuinely affect the validity of this study.

Conclusion and Future Work
New technologies provide teachers with many interesting tools that can be used to improve the teaching-learning process [28].Recent technological advances have led universities to an introduction of innovative modes of teaching and learning.Studies have found that students may react differently to the online learning environment, depending on their skill level and attitude [13].Moodle is a leading open source e-learning management system.With Moodle, educators can easily construct richly textured web-based courses.A course can consist of a number of lessons.Each lesson consists of reading materials; activities such as quizzes, tests, surveys, and projects; and social elements that encourage interaction and group work among students [39].In general, students in this study did not prefer using their mobile phones to access Moodle due to the limitations of mobile access on usability and reliability.However, most of them indeed used mobile phones to access Moodle when it was necessary.In terms of Moodle activities, it was found that students preferred carrying out easy and low-stake Moodle tasks on their mobile phones, such as accessing learning materials.The students expressed the need for a more user-friendly mobile access.Thus, in accordance with other studies our research concludes that it is the well-organized and clearly presented content that will make LMS a complement rather than a substitute for the formal class.Moreover, the findings of this study and all reported studies included in this paper as well, point out the necessity of active and competent users of technologies and modern digital equipment available, as well as smart mobile phones and/or tablets.
This study, much like other studies of similar nature, had limitations.A possible limitation is that the data collection was limited to a single university department in Greece.Follow-up studies can expand the sample by recruiting participants from different universities and in different regions.Another limitation is that the findings of this study are solely based on self-reported data from participants, which might be subject to the difference in students' own perception.Future studies could rely on objective data sources such as the usage patterns as reflected in the LMS system logs.Additionally, as it is clear from the results, the students' perceptions reflect issues that are both technical and social in nature, a fact which in many ways supports previous research on user acceptance of the technology.So, it will be of interest in future studies to try to identify, the individual, organizational, and technological factors that could be influencing the use of LMS, using the Technology Acceptance Model.
The feature of blended learning on which we would like to focus is the increasing emphasis on 'learner collaboration' [15].We soon expect the use of LMS in the University of Crete to be being utilized closer to Francis and Raftery's Mode 2 as students make more use of communication and collaboration tools.

Figure 2 .
Figure 2. Three e-learning modes of engagement in LMS usage.

Figure 3 .
Figure 3. Moodle content in a 5.5-inch mobile device screen.

Figure 4 .
Figure 4. Moodle content in a 5.5-inch mobile device screen.

Table 1 .
Students usage of Moodle via smart mobile phones.