Why can a 4+ score turn out to be bad and a 3 grade good?
It so happened that my activity is related to the creation of training complexes for personnel in hazardous industries. Since imitators develop in the context of multidirectional scientific vectors - computer graphics, engineering psychology (studying the physiological and psychological characteristics of a person, generalized independent characteristics, personality and profession psychograms), ergonomics, cognitive science, computer science, etc.
I often hear the question of why I prefer to work not with the “classical” grade (1-5) or (0-100), but with the whole set (knowledge-skills-skills), and I also necessarily use the degree of transfer of skills to staff working conditions.
Why? There will be two answers - one simple, the other detailed.
Imagine a situation where 2 people undergo training - for example, an operator who is responsible for the operation of a whole complex of equipment and, for example, clearing personnel.
According to the result of the training, the operator receives “4+”, and the cleaner receives “4”. We can conclude that the operator was trained better. If we speak only in terms of the "pedagogical scale" 1-5, then so it is.
Why is it bad?
It turns out that both were mistaken somewhere? Moreover, the operator was mistaken by 0.5, and the cleaner by 1. And now we ask the question - “what can the errors that are behind this unit and 0.5 in real production lead to?” The cleaning lady will forget to put a sign “carefully, wet floor”, someone with some probability will slip and with some probability will get injured... Suppose.
And what about the operator, with some probability he will not be able (for example) to perform the correct actions in the event of an emergency and with some probability the entire object, for example, will fly into the air. What is hiding his “missing” half unit?
It turns out 4+ from the operator “look” worse than 4 or even 3 from the cleaning lady.
That's why I try to never use the classic rating scale when creating and operating simulators.
I have already written about risk management , and now I’ll try to assess and control the necessary level of personnel characteristics...
In other words, I mean that any single assessment, at least 0.5, at least 0.100, cannot adequately reflect the willingness of staff to work. And I show how it is possible to express “readiness” through residual risk (in monetary terms, in the number of deaths, etc.).
Because instead of “the student received 4” say “the current staff training is at the level of probable losses of 240,000 rubles per year, which is at the level of“ acceptable risk “, i.e. staff may be allowed to work »
"the probability of each personnel error is equal to the probability of error on the simulator (simulator), which is completely identical to the real system (a system that faithfully reproduces the real one)"
1. Work execution procedure
An algorithm is a finite set of rules that defines a sequence of operations for solving a specific set of tasks and has five important features: finiteness, certainty, input, output, efficiency. (D.E. Knut)For any mastered profession, it is possible to distinguish the goals of training, for example, personnel should be able to adjust and adjust the equipment characteristic of the mastered profession.
An algorithm is an exact prescription that defines a computational process going from variable source data to the desired result. (A. Markov)
Achieving the goal involves the successful solution of a number of tasks (steps). Thus, the work execution algorithm (regulations) can be represented as a set of ordered tasks, while the algorithm can be linear, but may have a more complex structure (figure).
Picture.Linear and nonlinear algorithm (personnel flow chart)
Evaluation, formation and correction of ZUN (knowledge-abilities-skills) of directly performing work implies, therefore, the formation of ZUN for each task (element) that is part of the algorithm.
Each task, in turn, should be determined:
- input - variable input data set;
- A finite set of rules that defines the sequence of operations;
- used equipment, tools and devices;
- desired result (settable output);
- performance evaluation methodology .
Next, we’ll try to “uncover” point number 5
Figure. Algorithm Composite Element - Tasks
- Device, purpose and principle of operation.
- Key Features and Performance
- Parameter values (make-up torque, inrush current, etc.)
- Safety Rules
- Arrangement of platforms, stairs for convenient and safe maintenance.
- Lighting for facilities, crossings, and service locations.
- Installation and commissioning, wiring diagrams, etc.
- Instrument and Instrument Operating Rules
- sequence of actions (order, regulation)
- Decommissioning, preparation for work, installation.
- Launch and start-up condition.
- Control of technological parameters
- Stop and exit from the flow chart
- Organization and conduct of technical examination
- Control of the main parameters of work
- failure criteria and limit states
- Regulation of an operating mode according to indications of devices.
- Possible causes and emergency stop procedures
- Actions of personnel in cases of emergencies (actions of personnel in case of emergency and emergency situations)
- possible malfunctions and solutions.
- calculation methods (formulas, transfer of quantities from one system to another)
- The procedure for the preparation and conduct of maintenance (technical inspection, repair and overhaul).
- Acceptance of equipment from repair, running-in.
- Filling out sample forms for charts, magazines and reports
- practically (use knowledge) to complete the task (with the necessary accuracy at a given time);
- practically complete the task (with the necessary accuracy at a given time - throughout the entire shift);
For example, the formation of ZUN algorithms for performing work - testing centrifugal pumps (GOST 6134-2007. Dynamic pumps. Test methods) can be divided into the following tasks:
- running-in of the pump (unit)
- removal of pressure and energy characteristics
- pump flow;
- pressure at the inlet and outlet of the pump or the difference in the indicated pressures,
- temperature of the pumped liquid.
- dependence of the power consumption of the pump and its efficiency on the supply
- removal of cavitation characteristic
- self priming pump tests
- processing test results
To implement these tasks, the following knowledge must be formed among the students:
1. Terms, definitions:
- destination indicators (feed, head, speed);
- performance indicators and design (cavitation margin Δh (NPSH), coefficient of performance (COP), pump power, self-priming height, external leakage, mass)
- ergonomic indicators (vibration, noise)
- reliability indicators (mean time to failure, resource)
- characteristics (pressure, energy, cavitation, vibration, noise, self-priming)
2.Test conditions and principles:
- Conditions for determining indicators and characteristics
- Test conditions
- Testing on liquids other than pure cold water
- Tolerances for mass-produced pumps with typical catalog curves
- Test setup diagrams (stands)
- Error Definition
- Volume measurement method
- Principle of measuring the pump head
3. Testing sequence, presentation and presentation of results.
4. Definition of safety indicators:
- Electrical Safety
- Thermal safety
- Mechanical safety
- Other harmful production factors (indicators)
For the formation of skills, practice is required, including the fulfillment of all necessary tasks, that is, the student must have experience in performing the indicated actions. As a rule, each task is divided into a finite set of elementary operations - subtasks (open the suction valve, close the discharge valve, close the valves on the manometers, check...., press the "START" button to turn on the pump, check for vibration and noise, by., etc.)
At the same time, for the formation of knowledge, you can use both textual material, videos, 3D synthesized animation, and simulators. For the formation of skills, and especially skills, the use of imitators or real equipment is necessary. It is also possible to share them (figure).
To evaluate ZUN it is necessary to use the method of assessing the effectiveness of training
Picture. Simulator Screen for “Centrifugal Pump Testing”
Picture. Photo of a real installation
Method (mechanism) for assessing the formation and transfer of knowledge, skills
Assessment of knowledge, skills and abilities formed as a result of training
Assessment and control of the required level of characteristics - knowledge can be estimated based on how much the student remembered (this can be easily measured, for example, using tests).
In the work of Novikov A.M. "Analysis of the quantitative patterns of the exercise process.
Methodical recommendations ”the following data are given:“ When teaching real systems, the following characteristics can serve as a criterion for the level of learning :. ”
- temporary (time taken to complete the action, operations, reaction time, time taken to fix the error, etc.);
- high-speed (labor productivity, speed of reaction, movement, etc., values, inverse of time);
- accuracy (error in measures of physical quantities (millimeters, angles, etc.), number of errors, probability of error, probability of an exact reaction, action, etc.);
- informational (the amount of memorized material, processed information, the amount of perception, etc.).
Figure. Assessment (measurement) of staff knowledge (y=const)
Figure. Assessment (measurement) of staff knowledge (y=f (t))
If the function (percentage of recalled information) within the operating range is above the acceptable level, we can assume that the probability of personnel error for this reason is 0. Otherwise, that is, when part of the function or function is entirely below the acceptable level, within the operating range, the probability of personnel error due to “knowledge” can be calculated as the ratio of the areas of functions above and below the acceptable level, within the operating range.
The areas of functions are above and below the permissible level (the difference or ratio of these areas actually determines the probability of personnel error due to “knowledge”)
it is assumed, as mentioned above, that “the probability of each personnel error is equal to the probability of error on the simulator, which is completely identical to the real system (a system that reliably reproduces the real one)”, i.e. (P=Pf). If we take such a relationship between the level of the characteristic (mismatch, error rate) and the probability of personnel error (P=Pf), then P=1- means 100% probability of an error, P=0 means there is no possibility of error (0%), P=0.5 corresponds to a 50% probability of personnel error. Otherwise (when knowledge, skills and abilities are not completely transferred to the real object, due to differences between the simulator and the real system), the dependence can be given by the expression P=f (Pf).
Evaluation and control of the required level of characteristics - skills can be assessed based on how accurately (correctly) the staff performs actions depending on the available time. Such a check can be performed using simulators, by presenting various events to the trained/tested personnel and measuring the time required to complete an action or reaction to an event. Another approach is also possible - to present various situations and limit the permissible time for actions/reactions. Skills will result in a graph similar to the “knowledge” graph.
Figure. A graphical representation of the characteristics for skills as a function of correctness (above) or error ("inconsistency") of the performed actions from the time spent
The relationship between personnel skills and the probability of personnel error due to “skills” can be determined using function areas above and below the acceptable level within the operating range (the difference or ratio of these areas actually determines the probability of personnel error due to “skills”, see figure).
For example, when balancing a rocking machine (USGN) in the amount of 5 pieces, you can measure how accurately (correctly) the staff performs actions (quality of balancing) depending on the time spent. In this case, the Y axis represents the% compliance value of the current level of “balancing” with the accepted norm.
Evaluation and control of the required level of characteristics - skills can be assessed using an approach similar to the assessment of skills, with additional consideration of the ability to maintain the required level of skill over time in various conditions.
The algorithm for assessing the level of skills is performed as follows: the employee’s shift time interval is divided into several intervals, for example 10. Using a simulator, the accuracy of personnel actions is measured depending on the time spent and for each interval it is calculated (probability of personnel error). Then, the data obtained are presented in the form of a graph of the magnitude - the ability to maintain the level of characteristics over time.
When assessing skills, it is also necessary to take into account the ability of a trained or certified personnel to maintain the level of performance over time (for example, during a shift, with increased fatigue, decreased attention, etc.) under moderate stress (normal conditions), low stress (relaxed state) and high load. In the process of labor activity, the staff goes through three main states replacing each other: the phase of development, or increasing working capacity; high stability phase; health phase (fatigue).
Assessing the ability of personnel to maintain a level of performance over time is necessary, becausethe effectiveness of a person’s work largely depends on the current load and to a large extent on the developed “automatism”, that is, skills. For example, the following graph shows the characteristic levels over time for normal operating conditions (green line) and in the event of an emergency simulation/simulation (stress) (blue line).
Change in the probability of personnel error during the work shift under different conditions. (skills values during the work shift)
During the day (according to the experience of Tallinn taxi drivers), the most dangerous period is 11-15 hours. I agree with the data of Swedish scientists who studied the relationship between workers' erroneous actions and circadian rhythms, it is during these hours that workers have the most errors. And the Slovak scientist Yu. Kuruts, who cites this data, notes that drivers have the greatest number of cases of falling asleep while driving during these daytime hours. At night, the hours that are more dangerous from this point of view are from midnight to 5 a.m.
Such graphs can be used to assess a person’s “readiness” for a given activity, as well as to obtain strengths and weaknesses of personnel.
The relationship of personnel skills and the probability of personnel error through the fault of “skills” can be defined as the maximum value of probability during the entire time of a work shift.
Assessment of the transfer of skills achieved during training to real working conditions
Knowledge of the general principles of transferring the stereotype to one degree or another is necessary both in the development of training programs and in assessing their effectiveness.
Having learned how to track with your right hand, try to do the same with your left hand - here is one example of a stereotype transfer. To a certain extent, any training is associated with the transfer of a stereotype, since the acquisition of a new skill is never completely independent of other activities preceding it. For most training programs, the question of transferring a stereotype is very important, since, except when training is conducted directly at the workplace, the value of the training program will depend on how much of the vaccinated skills will be transferred to the actual working conditions. For example, it was shown that for piloting a helicopter at low altitudes, only 15 hours of specialized navigation training give as much as about 2 thousand hours of general flight practice, a result that fully justifies the time spent on training. Knowledge of the general principles of transferring the stereotype to one degree or another is necessary both in the development of training programs and in assessing their effectiveness.
If, when mastering task A, the grades obtained from another task B improve compared with the grades of the control group that studied only task B, then the transfer from A to B is positive. Task A may consist, for example, in tracking the rotation of the disk with one hand, and task B in tracking with the other hand. Sometimes it happens that the development of task A makes it difficult to master task B, and in this case they talk about a negative transfer. In more complex cases, the so-called retroactive effects can be observed, which occur when A is first mastered, then B, after which a second test is carried out according to A. If such insertion of task B improved the performance of A, then there is a retroactive amplification; if insert B worsens A, then there is retroactive interference (or retroactive inhibition).
The more similar tasks A and B are, the more they affect each other. Whether the transfer will be positive or negative in this case depends on how the characteristics of both tasks, such as display and control or stimulus and reaction, are related. Osgood's three-dimensional surface  is an attempt to summarize the results of early work on the study of the relationship between transference and retroaction. If both the presented stimuli and the required reaction in both tasks are so similar that they are practically indistinguishable, then the transfer will obviously be maximum.In all respects, assignments A and B are variants of the same assignment, so learning A is equivalent to learning B.
Other cases contained in the table. 9.5 can be illustrated by an example taken from the shoe industry. Let task A consist in making a seam on the boot, consisting of separate stitches (stimulus), by repeating pressing the pedal with the necessary force (reaction), and task B, in order to light a series of neon lamps (stimulus) by repeating taps on the telegraph key (reaction). Under these conditions, both the stimulus and the reaction are different for both tasks, and therefore there is no transfer of the stereotype . However, if you ask a previously trained group of shoe makers to light a series of neon lamps by pressing a regular foot pedal, then the reaction in both tasks will be the same, although the incentives are different.
Therefore, a positive transference will occur; Experienced shoe makers perform task B better in this option than untrained people.
The last of the relationship options presented in the table is more difficult to analyze, and the surface of Osgood describes it inaccurately. In our example, to require a different reaction to the same stimuli means to ask the shoe makers to sew stitches by pressing the telegraph key. Such a procedure can give a negative transfer. In the new situation, ceteris paribus, a person usually tends to do the same as in the old one. If the conditions have changed, but this change is not completely obvious, then an old reaction that does not meet the new conditions may occur. In this example, experienced operators of the sewing machine, instead of a series of light taps on the telegraph key, can sometimes try to put pressure on it for a long time and with great effort. But it is also possible that, in spite of individual errors, experienced operators as a whole will show better successes than an untrained group, due to the common similarity of both tasks. The result will partly depend on how points are awarded. In any case, it may turn out that with further development of task B, initially the negative transfer will change to positive, as errors will become less and less. It is important to prevent a negative transfer from a training device, such as a simulator, to real working conditions, but, unfortunately It is not easy to predict when a negative transference will occur. However, from , where an attempt was made to predict obsessive errors using a three-dimensional surface, linking the similarity of stimuli and reactions with the expected transfer characteristics, it follows that as the degree of similarity of reactions increases, the interference between the two tasks increases. Whether such occasional errors are significant depends on the nature of the assignment. When sewing shoes, a random error due to a negative carry may not be important, but when landing an aircraft, such an error can lead to disaster. Such errors are most likely to occur in those cases where the reactions required in tasks A and B are easily confused. Trainees are unlikely to confuse cycling with a cup of coffee, even if the stimulus in both cases is a green light. However, reactions such as raising and lowering the lever, turning the steering wheel clockwise and counterclockwise are very easy to confuse.
The depth of transfer of the stereotype to real conditions tends to increase with increasing training time. In some cases, the amount of training may be more important than the accepted method of training. A study of car driver training programs showed that with 6 hours of practice, the depth of transfer of skills to real driving was higher than with 3 hours of practice, regardless of whether a movie or a trainer was used for training. However, the transfer depth is not a linear function of the training time. With a further increase in this time, usually a decrease in returns occurs, so to determine the effectiveness of training it is necessary to constantly measure the depth of transfer.
In the traditional measurement method, the initial transfer to a new task is estimated by calculating the degree of improvement of indicators for those who have mastered task A, compared with those who have mastered only B.The difference between the indicators of the transfer group and the control group (transfer minus control — for indicators characterizing accuracy; control minus transfer — for indicators characterizing speed or error), usually related to the first attempt to complete task B, is presented as a percentage (percentage) of the total the amount of potential learning. A typical formula has the form
However, the transfer may not remain constant as you study task B, so more flexible assessment methods may be required to monitor the effectiveness of the training.
Some sensitive measure of the value delivered at different volumes of training is especially needed when using simulators, when the cost of training and the cost of full-scale practice are usually high, but known and amenable to regulation. The simulators themselves, the degree of their proximity to real conditions and the corresponding characteristics of the transfer, we do not discuss here. However, the flight simulator can serve as a good example of mastering task A, the results of which should be transferred to a real flight - task B. A typical task is to determine how much training should be on the ground simulator before allowing beginners to fly.
The most useful are indicators that characterize the "savings", or "replacement rate". The transfer efficiency can be estimated by the number of hours of summer time saved due to ground preparation in one or another volume. In , differential and cumulative meters of such efficiency were proposed. If you need 10 flight hours to achieve the required skills, and in the case of pilot training on the simulator within 1 hour, only 8.6 flight hours are needed, then the saving is 1.4 hours. Another hour on the simulator may give a slightly lower saving - say, 1.2 hours, so that the accumulated savings after 2 hours on the simulator will be 2.6 hours. Dividing this value by 2 (the number of hours of training), we obtain a cumulative transfer efficiency coefficient (CECP) of 1.3 per hour ground preparation. The corresponding formula can be written as follows:
As pic. transfer efficiency, estimated by this indicator, as a rule, monotonously decreases. With 5 hours of training, the simulator will still require 5 hours of flight time, and the transfer coefficient will drop to 1.0. It is clear that from the point of view of the total time required to train the pilot, a further increase in training time does not make sense. If the criterion is cost, which is quite possible in the example under consideration, then it may be appropriate to extend ground training. For example, if 1 hour of flight time is three times more expensive than 1 hour on the simulator, then it is beneficial to continue ground training until the efficiency coefficient drops to 0.33. 15 hours of training time plus 5 hours of flight will cost the same as 10 hours of flight time. Obviously, in this case, you just need to express the coefficient of transfer efficiency in units of cost, not training time. There are more complex ways to maximize cost-effectiveness, based on differential calculus methods.
A simpler way of evaluating the effectiveness of training is also shown in Figure In the “L + B” method, the hours of study time (or the number of practical exercises, or the cost of training) for assignment A are added up with the same indicators characterizing the amount of training required after assignment for assignment B after transfer, and this amount is calculated for each of the possible values of the practice volume on task A. As soon as the total value of the indicator for both tasks exceeds its value for task A alone, we can conclude that training has become uneconomical. On the graph, this happens at the point where 5 hours are spent on each task, since 5 hours of training time and 5 hours of flight obviously do not give any savings compared to the standard 10 hours of flight time.It is clear that this boundary point coincides with that defined using the transfer efficiency coefficient.
A properly constructed training program should be oriented towards maximizing the transfer of the stereotype to the task for which it is intended. When achieving a high degree of transfer, you should try to optimize the total training time. The use of quantitative indicators of transfer, as in the above methods for assessing verbal, visual and practical skills, should help ensure the effectiveness of training.