Advanced teaching module for the course " Theory of Probability and Statistics " based on a recurrent test system with an incremental level of complexity and dynamical case studies

This paper presents an innovative approach to teaching the discipline ‘Theory of probability and statistics ‘, which based on recurrent test system with an incremental level of complexity and on a dynamic block of case studies generated by information received online (by using Thomson-Reuters informational terminals). The model of the so-called "recurrent testing" allows to increase significantly the student's knowledge level. The idea is to build such a system of successive tests, which takes into account at each subsequent step the results of the previous one, thereby each time modifying the task to the level of the tested and gradually increasing its level of knowledge, and, consequently, the quality of learning as a whole. It is important that, regardless of the starting point the recurrent testing system converges quickly to the trainee's knowledge level. The method is also described by the case study 'Modeling of the stages of development of the company based on real data'. The adequacy of conclusions obtained with this approach have been shown, that is an additional advantage of the proposed model of learning.


Introduction
The traditional approach to testing the student's knowledge level assumes that the quality of the answer to the test task is characterized by a binary value: "correctlynot correct".This approach underlies the classical theory of tests and the modern test theory -IRT (Item Response Theory) (see Neiman (2000)).At the same time, in many cases, the solution of the problem (if it's not about testing when the correct answer is determined unequivocally) is evaluated in full: "solved, but not completely," not solved, but the idea was correct," etc.The demands of modern employers to a young specialist make one think about a more differentiated approach to assessing the level of knowledge.The authors developed a method of recurrent testing, which makes it possible to obtain a differentiated estimation of the level of knowledge.The main result of this part of the work is a constructed mathematical model for assessing the level of student knowledge, the input parameters of which are the volume and complexity of test tasks and a priori estimation of the student's knowledge level.It is assumed that the quality of the test tasks is characterized by a certain continuous indicator.An important, or even the main, aspect is the demonstration that the developed methodology, regardless of the starting point (the primary test), converges quickly to the student's knowledge level.To prove this fact, based on the Birnbaum model (see Sage (1982)), a mathematical model of the process (recurrent testing) is constructed and using Fredholm integral kernels (see Koralov (2007)) can be shown its convergence.The results of this part of the work were first presented at an international conference at the Finance University under the Government of the Russian Federation "Educational programs and professional standards: the search for effective interaction" by a report (01.04.2016), which was later expanded into an article "Step-by-step testing method: joint assessment of the training level and the complexity of the assignment "(see Mochalina et al. (2017-2)).

A new course structure and grading system
Currently, there is a full range of opinions regarding the acceptability of testing in the learning process: from complete rejection to total application.Meanwhile, the advantages of such a method of measuring the level of knowledge are already clearly defined: objectivity, speed, technology, coverage of all the teaching material, the possibility to use mathematical methods for processing results, etc.Further work went in the direction of increasing the variability of the task to ensure the possibility of a more differentiated, and therefore more objective estimation while maintaining a recurrent method of testing.Also, the aspect of relevance was added: the tasks should be brought closer to reality (to increase the interest in the subject).And thus, we come to a system of clinging cases, based on real data.
To implement the training module in practice, a computer program was written "A practical simulator with three levels of complexity in the discipline "Probability Theory and Statistics "on the basis of cases with open content."The software product (information system) is designed to test the received theoretical knowledge and the formation of practical work skills (working with data sets, dispersion and correlation analysis, time series analysis) in the discipline " Probability Theory and Statistics" and used to test the quality of training on this discipline.The simulator is an innovative way to apply testing: the test is not a form of control (more precisely, not only as a form of control), but as a way of obtaining knowledge.The underlying test is part of a set of similar ones that are an indicator of learning (with the obligatory condition of the uniqueness of each test).Work is built in such a way that when executed by the student enters the next level of training.The first block of the simulator is the multi-level tests (important(!):without option of choosing the answer, the simulator allows only put it), the results of which are estimated by a binary value ("solved" -"not solved").Further, by summing the results of the test tasks, the student's knowledge level indicator is determined.We also note, that testing is a method that puts all students in equal conditions, virtually eliminating the teacher's subjective assessment.This approach allows to set the level of knowledge of the student on the subject as a whole and on individual sections without additional efforts.Note also that this is gentler tool for assessing the level of knowledge in comparison with traditional control work.The student has the opportunity to choose the time of passing the test conveniently (access to the simulator is open during the working hours of the university), and also he sets his own the goals and objectives of the training independently: to learn the minimum permissible level, some intermediate level or to progress to a deeper learning of the material (which is achieved by work with the second block of the simulator), thereby ensuring the study of the main trends and methods used in practice.In our case, we are talking about working with large data sets that reflect a specific practical problem and actualize the appropriate range of knowledge, the part of which are probabilistic estimation, forecasting, etc.
The second block is a dynamic case system based on real data.Case study tasks always involve an ambiguous solution.Consequently, we can say that the creation and protection of the project within the framework of the case study will help, first, to identify those students who are ready for further training in magistracy and graduate school, secondly to create conditions closest to reality and thereby such a graduate will most competitive.Thus, we can conclude that the simulator provides the following stages of mastering the competencies necessary for the training of highly qualified specialists: 1) well acquired fundamental knowledge: it is practiced in solving of various tests ranged and estimated on complexity that allows moving to another level; 2) skills of work with analytical packages and programsis also practiced on a set of various "undergraduate" tasks; 3) at the exit to certain level in the previous conditions, the student is ready to solve the problems which to the maximum are brought closer to reality (working with case study based on real data).
Case study approach allows us to improve the estimation and analytical skills, the skill of finding a rational solution to the existing problem, that is, eliminates the main limitations of the test approach.

Results
In the process of studying any discipline the information environment in which necessary skills are gained is important to the student.Electronic Content now -it's convenient and universal.The information is updated in real-time, public and comprehensive.Such material as a basis for learning is new, and, most importantly, it forces us to completely change the approach to the system of knowledge translation.Accordingly, the knowledge gained because of the course, reflecting most trends and methods used in practice, will not be cut off from reality.An innovative approach is to create a dynamic block of case studies based on real data.Within each work, the basic principle of the case study is observed: each subsequent step is based to one degree or another on the results of the previous stage.The most important point is the content of the case study: these are real constantly updated data.Working with case studies thus generated -is the foundation for the creative activity of the student, which allows identifying among them ready for the further education in today's society.We tried to create a training module most appropriate to this requirement.Thus, the relevance of the theme of this work is due to the fact that modern economic conditions make higher demands on the level of training of qualified personnel, which explains, on the one hand, the need to increase the level of computer literacy of the student, and on the other hand, assumes free knowledge of the analytical apparatus that is acquired only by solving a number of real problems.It is important to note that this is not our first attempt to realize these ideas in practice.We already did the first step.Our previous work was presented at the "8th Annual International Conference on Computer Science Education: Innovation and Technology" in the Singapore (09.10.17-10.10.17) by report (see Mochalina et al. (2017-1)) and more detailed in article (see Mochalina et al. (2017-2)).The developed estimation methodology has found its application partially in the development of working training program on "Financial calculations" (Finance Department).To implement the training module in practice, the computer program "Case-transmitter for the training module" Financial calculations" was developed and implemented.This program doesn't contain test part, only chain of clinging case studies: portfolio theory, forecasting share prices, estimating the option prices and hedging.Many people working with financial instruments know how difficult to teach the graduate to work: theoretical base (see, for example Hull ( 2014)) and reality are weakly connected.And this training module allows to improve significant the situation.It can be easily seen by the diagrams below (Fig. 3 & Fig. 4).The first represents the marks on the course "Financial calculation" in 2016 (before implementing case study method of teaching and in 2017 (the system is implemented) and the second gives the results for graduates in searching job.We also note that in 2016 testing group consist of 260 students, and in 2017 the total number of student was equal to 440.A further strategy connecting to these results involves shifting the developed methodology of translation and estimation of knowledge to virtually any mathematical discipline.And we did it and presented simulator above.For clarity, we show here one of our case studies.

Case study: Modeling of the stages of development of the company based on real data. Task: based on empirical data construct the model allowing to determine the place of the company on the market.
Solution.To determine the configuration of the stages of the companies and to construct pre-classification student have selected 73 companies from the food industry (the minimum allowed sample volume is forty companies).To collect data our students, use Thomson Reuters informational terminals.We proud to say that our university (one of two in Russian Federation) has the agreement with company Thomson Reuters according to which the university got a special class equipped with Thomson Reuters informational terminals (see Fig. 5 below).So, our students have a unique opportunity to work directly with them and they used that during solving this case study.They can collect the data directly from the terminal (Fig. 6) or download it (in the format of pdf-file).For the analysis student constructed two types of variables: the cluster and descriptive.It's important to check that variables were picked up correctly: they don't depend from each other, selections have normal distribution.And then it is possible to use the apparatus of the cluster analysis.The results of hierarchical clustering are represented as a horizontal dendrogram (see Fig. 7).
Mochalina, E. P.; Ivankova, G. V.; Tatarnikov, O. V.Then, next step in solution this case study for student is using the apparatus of the discriminant analysis to find out which of the taken variables had provided the separation of one cluster from another and further to be able to describe the situation within each cluster.Each of the constructed by student discriminant functions allows to calculate unit weights of classification (tags) by the formula: Here the index -is the number of cluster, -the constants for -th cluster, the quantity of variables, the coefficients of -th variable during the calculation of the tag of the classification for th cluster,the observed value for the corresponding object of the selection by -th variable.So, one need to consider the functions of classification.The object (company) gets into the cluster to which the greatest weight (the maximum tag) corresponds.-67,9566 -81,2768 -101,494 -116,256 -177,362 scale of the work done is easy to understand by looking at, for example, the work Hanks at al. (1994) or book Shirokov (2010), whose, in turn, are based on ideas of Adizies (see Adizies (1989Adizies ( ,2013))).
Based on the Faculty of Finance, it is planned to test the developed training and practical simulator, which allows not only to obtain an estimation of the level of knowledge of trainees, but also to identify the most capable and ready for further education students (at the following levels: master's and post-graduate courses).The result of the introduction of such an approach to learning will be the formation and development of students' skills in applying methods of qualitative and quantitative analysis using information technologies that corresponds the requirements of a modern employer.

Figure
Figure 1.Testing

Figure 3 .
Figure 3.The marks Figure 4.The results on searching the job

Figure 5 .
Figure 5. Thomson Reuters class Figure 6.Collecting data from the terminal directly

Table 1 . Functions of classification; grouped by number of cluster
It is a very difficult taskto construct the model like in this case study.To do it student should not only be able to work with informational terminals, but also to possess the apparatus of discriminant and correlation analysis, the skills of working with data sets, books (important!) and the skills