Training Tips for Ships
Tip #1. Immediate exam retakes should never be allowed.
It is very common practice, especially in eLearning environments, to allow trainees to immediately retake an exam they just failed. Worse, sometimes training is set up to allow the trainee to take the exam repeatedly until they pass.
Never do this. It will ensure that your learners are not learning what you want them to learn.
For most people, this raises two immediate questions. First, why not allow immediate retakes? And second, what shall we do instead?
To answer these two questions, it is important to understand that for any non-trivial training program, exams can never be comprehensive assessments of what a trainee knows. Instead, we test only a subset of items to avoid an impossibly long exam. If the trainee does well on those items, then it is likely the case that they have a similar level of knowledge for those questions we did not ask. This is completely sound logic only if (and this is a really important “only if”) trainees don’t have prior knowledge of what is going to be on the exam. If the trainee knows what will be covered on the exam, they will only study enough to learn the answers to those specific questions and ignore the rest. Without prior knowledge of the exam questions, the trainee is forced to study all the material equally thoroughly in order to be able to pass the exam, regardless of which material it covers. Assessments exist not only to assess knowledge, but also, perhaps even more importantly, to motivate comprehensive learning.
So, why must we never allow immediate exam retakes? If an assessment is configured to allow learners to take an exam over and over until they pass, we have allowed them to gain knowledge of what is on the exam through repeated retakes. After four or five failed exam attempts the trainee will have learned enough about the exam to pass it without having to comprehensively study the material, even if the questions are randomized.
Analysis of existing systems set up this way shows that it is not uncommon for the average trainee to write an online exam 5 or more times before they pass. Some of the most enterprising (though misguided) trainees write exams 30 or more times before passing. The ability to repeatedly retake exams disincentivizes studying. While they are eventually passing the exam, these people are absolutely not learning the course material well.
Now onto “what do we do instead”? The answer is to return to an assessment configuration that motivates learning. There are many ways to do this but one of the simplest is to enforce a waiting period between exam retakes. That is, configure your LMS such that when someone fails an exam, they cannot take that exam again for a day or two. This solves the problem by turning exam attempts into a scarce resource that must be used wisely by the trainee. It incentivizes studying and greatly reduces the opportunity to “learn to the exam”. Knowing there is a waiting period encourages them to study the material to some degree before their first attempt. Then, if they do fail, the wait will be best spent doing further studying so as not to squander another scarce exam-writing opportunity. Simply said, it will make comprehensive studying their preferable option and the path of least resistance, which is exactly the goal of a sound assessment methodology.
About the Author: Murray Goldberg is CEO of Marine Learning Systems which provides software and services to optimize knowledge, skills and behavior in maritime operators. In his former life he was a Computer Science faculty member at the University of BC researching on-line learning and assessment delivery models and their effectiveness. This led to him develop WebCT, a learning management system that was used by 14 million students in 80 countries. Murray's "Training Tips for Ships" is a monthly training and education column in the pages of Maritime Reporter & Engineering News.
