EDUCATION QUALITY EVALUATION OF ELECTRIC POWER ENGINEERING SPECIALTIES STUDENTS IN TECHNICAL UNIVERSITY OF RUSSIA WITH THE USE OF INTERNET TECHNOLOGIES

A fuzzy model for the education quality evaluation of electric power engineering specialties students with the use of modern Internet technologies is described, which allows to take into account the different compatibility levels of various quality indicators used to form the estimation.


Introduction
Presently among the priority directions of the higher education development, it is possible to single out the task of developing, implementing and effectively using information and telecommunication educational technologies implemented on the global Internet basis.This task is especially acute for higher educational institutions of a technical orientation, in particular, for electric power specialties, where information technologies, through the organization of virtual experimental installations, make it possible to significantly reduce the cost of the laboratory work, to perform new tasks on a new qualitative level, and also work on studying and modeling of objects and systems of the electric power complex.
Widespread usage of information and communication educational technologies in many technical Russian universities requires the model and methodological support development for assessing the quality of students training in various educational institutions, identifying positive experiences with a variety of Internet technologies for the further wide application in the technical universities of Russia, as well as timely identification of factors that negatively affect the education quality.Currently used methods of monitoring the education quality in technical universities often produce only partial estimates in the form of values set for individual indicators.These approaches not only make it difficult to formulate a comprehensive integrated education effectiveness evaluation, but also do not allow determining the contribution of individual indicators values to the overall integral assessment, which is necessary for developing justified recommendations on the wide dissemination of A. V. Bobryakov, V. V. Borisov and Y. A. Fedulov 2 positive practices of Internet technologies in the technical universities educational process.In turn, the techniques that make it possible to obtain the value of the generalized efficiency coefficient, as a rule, are based on a simple summation of different quality indicators values without any consideration of mutual influences and indicators compatibility.Thus, the task of developing an evaluation model and the methodology for assessing the effectiveness of students education for electric power specialties in higher education institutions using Internet technologies, taking into account the features and different indicators compatibility levels of the subject area, is relevant.The indicators of the development and effective use of Internet technologies in education are often calculated on the basis of ambiguous, contradictory and diverse source data, which imposes significant restrictions on the choice of a suitable mathematical apparatus.In these conditions, the methods of direct evaluation based on the theory of fuzzy sets and fuzzy logic [1,6,11], which allow to obtain a generalized value based on the particular indicators of the evaluated object, has proved themselves well.To reliably evaluate complex system and make informed decisions, generally, it is not enough simply to get the result, it is also necessary to know how the input parameters affect the obtaining of the resulting estimates.The solution of such inverse problems for fuzzy systems is called the inverse evaluation.However, methods designed to solve inverse problems are developed only for a number of particular models.There is no general approach to constructing evaluation models for complex organizational and technical systems with the possibility of either direct or inverse fuzzy evaluation, taking into account the mutual influence of the evaluated indicators.This paper presents the method of constructing a fuzzy evaluation model of the proposed type.The results of its application are presented on the example of the education quality of electric power engineering specialties students in universities with the use of Internet technologies.

Fuzzy evaluation model of education quality
In general, the problem of constructing the proposed fuzzy evaluation models is formulated as follows.Let there be a set of indicators with the values that represent the results of the corresponding properties evaluation for solutions alternatives.It is required to construct the fuzzy evaluation model based on multi-level evaluation structure, various significance of indicators and compatibility relationships between indicators at each level of the model hierarchy [4,5].
The whole set of indicators is divided by levels of hierarchy.At each level of the hierarchy, indicators form a subsets, each of which correspond to the characteristic adjacent to it at higher hierarchy level.Each indicator is assigned to the weight.Indicators belonging to the same q s j j j j qQ j j j j q q q i q n q j j j j j q i s s s m s n s j qi {(( , ) / )}, 1, ..., ; , {1, ..., }.
where Jnumber of levels of the model hierarchy; Qnumber of subsets of indicators at j-th hierarchy level; Snumber of subsets of indicators at (j+1)-th hierarchy level; q n -number of indicators from subset () j q P at j-th hierarchy level; s n -number of indicators from subset ( 1) j s P  at (j+1)-th hierarchy level, associated with i-th indicator () , j qi p from subset () j q P at j-th hierarchy level; () , j qi p -i-th indicator from subset () j q P at j-th hierarchy level; Suggested fuzzy evaluation models are characterized by the following features: flexible hierarchical structure of indicators, allowing to reduce the problem of multicriterial evaluation of alternatives to one criterion or vector of indicators used for selection; allow fuzzy representation of parameters and coherence relations between them; consider various significance of evaluation indicators; provide implementation of the methods of direct and inverse fuzzy evaluation; contain the required set of formalization for a software implementation.

Method for constructing a fuzzy evaluation model
The method of constructing the evaluation model is based on the search of the indicators subsets that compatible with one degree and the convolution of estimates for them in accordance with the strategy worked out for a given degree of compatibility.Depending on the nature of problem being solved, compatibility of concepts can be interpreted as correlation, mutual influence, simultaneous achievement of their criterial values, etc.
Let us demonstrate the main stages in the construction of a fuzzy model for evaluating various quality indicators for the higher education institutions activities.average", "Average", "Above average", "High".The type of membership functions (based on the two-sided Gaussian standard membership function) is shown in Figure 2. Table 3 provides an analysis of compilation for TB and ESN indicators.Table 4 provides an analysis of compilation for ESA and IT indicators.The "+"icon indicates the presence of the combinability of the values, the "-" sign indicates the absence.

Compatibility levels set
Based on the completed tables analysis results, a set of four compatibility levels was compiled for all pairs of indicators: {1, 2, 3, 4} С  . In this set the level 1 -"high" means that the achievement of high evaluation values by one indicator significantly facilitates (but does not guarantee) the achievement of high values on another, and the level 4 -"low" means that attaining high scores for one indicator severely limits the achievement of high values on another.Table 3 shows a low compatibility level between the indicators of TB and EF, Table 4 shows a high compatibility level between the indicators of ESA and IT.

Construction of the compatibility matrix
Table 5 presents the compatibility matrix of the second level indicators of considered evaluation model, built based on completed compilation tables and a compatibility levels set.Similar matrices were constructed for all groups of private indicators of a fuzzy evaluation model.

Construction of a generalized convolution operation of indicators
This stage involves establishing the order of binary convolutions application corresponding to the compatibility level for each compatibility matrix.By analyzing the second level indicators compatibility matrix, it can be concluded that most combinations of indicators correspond to "middle" compatibility levels (2 and 3).Under these circumstances, it makes sense to start convolution operation from indicators associated by medium compatibility levels.The connections recalculation of the minimized vertices with the remaining vertices should be solved by choosing an arc with the medium compatibility value.Thus, the most "average" evaluation strategy is chosen.This strategy benefits from  [10,12] these universities are in the list of top 100 institutions of higher education in Russia.
The summary of general information and students, university graduates and applicants interviews that are in free access [9] was used as a basis for initial data for the evaluation.Separately from sufficiently detailed information about named universities, the source site provides the university evaluation in the field of electro-energy specialties, based on the three separate indicators: "passing score", "number of budget places" and "average cost of education" in university, the value of each indicator is shown in Table 6.To obtain the education quality overall assessment, the source site proffers the calculation of three indicators weighted average as a conventional methodology for considered field.Table 7 presents the results of the universities evaluation, calculated according to the open access data given in [9] as well as their ranking among top 100 Russian institutions of higher education by authoritative rating agency Expert RA (RAEX) [10].The presented data do not agree with the assessment of reputable rating agencies and other sources [10,12].The suggested reasons for this discrepancy are the following drawbacks of the methodology for calculating given in [9]:  the initial sheet of evaluation indicators is too short, and does not reflect all aspects and nuances of the higher education institutions activities;  the source data are not submitted by experts in the given subject area;  the weight of the significant collapsed parameters, such as "the use of Internet technologies", in the resulting evaluation is not fully taken into account;  the mutual influences and contradictions of the evaluated indicators are not considered.
The method proposed in this paper considers the above limitations.As a result of the models and application method described in this paper on the survey data from [9], the education quality evaluation of the three Russian technical universities was procured and presented in Table 8.To quantify the advantages of the proposed evaluation method over conventional approaches, deviations in the estimates for each university were calculated for both methods.The calculated deviations are presented in Table 9.The average deviation of the presented method estimates compared to the baseline values obtained by the considered example is 11%, while one of traditional evaluation methods (weighted indicators summation) deviation is 21%.

Method of inverse evaluation
To the inverse problems solved using the proposed fuzzy evaluation models include the following:

The application of inverse evaluation
The inverse fuzzy evaluation method is designed based on decision trees.It provides ranges of values of particular indicators for given value of the generalized indicator using the rules described earlier [6].The bases of the method are: built fuzzy evaluation model; defined evaluation strategy and convolution structure; values set of the generalized indicator.
To demonstrate the application of inverse evaluation method lets consider the typical problem of private indicators values ranges definition for a given range of values of the target indicator, on example of the integral parameter "the use of Internet technologies" of the constructed fuzzy evaluation model.
In connection with the growth and unconditional benefits of the Internet technologies introduction into the education of electric power specialties students, the contribution of this indicator to the overall assessment is significant; hence good values of this indicator are highly desirable.The use of reverse estimation provides detailed review of which evaluations of private indicators, assure high estimates of the integral indicator.13.

Conclusion
Thus, this paper presents a way of constructing a fuzzy evaluation model of education quality of electric power engineering specialties students in universities with the use of Internet technologies, taking into account the compatibility of indicators and the possibility of both direct and inverse evaluation, and their applications for the purpose of assessing and subsequently choosing solutions based on the analysis of the university activities as a complex organizational and technical system.
In comparison with conventional estimation approaches in the considered field the proposed method has the following advantages:  the initial sheet of evaluation indicators is decomposed, structured, and reflects all aspects and nuances of the higher education institutions activities;  the source data is submitted and initially evaluated by experts in the given subject area;  the weight of the significant collapsed parameters, such as "the use of Internet technologies", in the resulting evaluation is fully taken into account;  the mutual influences and contradictions of the evaluated indicators are considered;  fuzzy model and methods contain the required set of formalization for a software implementation in applied decision support system.
The average deviation of the presented method estimates compared to the baseline values obtained by the considered example is 11%, while one of traditional evaluation methotodologies (weighted indicators summation) deviation is 21%.

2. 1 .Figure 1 -
Hierarchical structure of indicators EAI Endorsed Transactions on Energy Web and Information Technologies 06 2018 -07 2018 | Volume 5 | Issue 19 | e5 As a result of the subject area analysis of education quality of electric power engineering specialties students in universities with the use of Internet technologies, a fourlevel hierarchical structure of indicators is determined.It consists of one general indicator on the first level, four integral indicators of the second level: "educational and scientific activity" (ESA), "technical base" (TB), "the use of Internet technologies in education" (IT), "educative and social work" (ESW), 13 integral and private indicators of the third level, and 19 private indicators of the fourth level, evaluations of which are given in the universal scale.The structure is shown on the Figure 1.The hierarchical structure of indicators for the education quality in university 2.2.Weighting coefficients The vector of weights provides the different significance of particular indicator in a generalized evaluation and the possibility of its re-adjustment.The weight coefficients of the education quality model are obtained by the method of paired comparisons with the help of experts in the subject area.Table I presents the obtained values for the indicators of the second level, Table 2 reflects the weight values of the third level indicators for the integral indicator "the use of Internet technologies".
The method based on an intermediate expert composition of compilation tables for possible separate values of indicators was implemented to determine the compatibility of individual indicators.For the considered model, the representation of the indicator in the form of a linguistic variable with five grades was chosen: "Low", "Below EAI Endorsed Transactions on Energy Web and Information Technologies 06 2018 -07 2018 | Volume 5 | Issue 19 | e5

Figure 2 -
Figure 2 -Fuzzy indicator grades The choice of five gradation for each indicator sets the dimension of the compilation tables: 5 x 5.The compilation table is constructed as follows.An expert evaluation is made of the compatibility of the linguistic values of one indicator with the linguistic values of the other.Presence or absence of compatibility values is noted.The results are tabulated for each pair of indicators.Table3provides an analysis of compilation for TB and ESN indicators.Table4provides an analysis of compilation for ESA and IT indicators.The "+"icon indicates the presence of the combinability of the values, the "-" sign indicates the absence.


definition of values or ranges of values of different indicators for a given value (range of values) of the target indicator; EAI Endorsed Transactions on Energy Web and Information Technologies 06 2018 -07 2018 | Volume 5 | Issue 19 | e5 EDUCATION QUALITY EVALUATION OF ELECTRIC POWER ENGINEERING SPECIALTIES STUDENTS IN TECHNICAL UNIVERSITY OF RUSSIA WITH THE USE OF INTERNET TECHNOLOGIES  finding the best solutions (values or ranges of indicator values) that provide the desired value or range of values of the target indicator.The basis for solving these problems using the proposed fuzzy evaluation models lies in the rules for determining the values (ranges of values) of the arguments k p and l p in parametrized operations of the form med( , ; ) gen p proposed in[14] (see Table10

Table 10 - 11 -
).The rules for determining the values (ranges of values) of the arguments The rules for determining the values (ranges of values) of the arguments The rules for determining the values (ranges of values) of the arguments

Figure 3 -
Figure 3 -Decision tree for given example

Table 1 -
Weights of the second level indicators

Table 2 -
The weights of the third level evaluation indicators for the integral indicator "the use of internet technologies"

Table 3 -
Compilation table for TB and ESW indicators

Table 4 -
Compilation table for EA and LA indicators

Table 5 -
Compatibility matrix of university activities upper level indicators

Table 6 -
University evaluation indicators by free access data

Table 7 -
Education quality evaluation of Russian technical universities by free access data

Table 8 -
Education quality evaluation of Russian technical universities according to the constructed model based on free access data

Table 9 -
Comparative deviations according to the universities evaluations for the proposed and conventional estimation method in comparison with the reputable agency ratings

Table 12
presents the compatibility matrix the third level indicators of constructed evaluation model of the integral parameter IT.EAI Endorsed Transactions on Energy Web and Information Technologies 06 2018 -07 2018 | Volume 5 | Issue 19 | e5 A. V. Bobryakov, V. V. Borisov and Y. A. Fedulov

Table 12 -
Compatibility matrix of the second-level evaluation indicators for the integral indicator "The use of Internet technologies"

Table 13 -
Variants of values for particular characteristics for