| JOURNAL/NEWS ARTICLES |
From "Innovative technologies improve industrial learning"
By Dr. Charles C. Reith
In Business & Industry Connection,
"One of the most important questions in training is how to assure that employees really understand what they need to know to perform quickly, confidently and reliably. OSHA addresses this in its rules on Process Safety Management which has prompted most trainers - and virtually all computerized training programs - to rely on multiple choice tests to determine whether trainees are ready to move from the classroom to the workplace.
However, multiple-choice tests fail to measure the degree of confidence that an employee has in the knowledge provided, or the degree of retainability that may be expected. This is important, because the safety of the public and environment should not be entrusted to operators who may have guessed their way through a multiple choice test or may have learned just enough to answer but not enough to retain. A visionary educator has developed a system he calls Self Assessment Computer Analyzed Testing (SACAT). When you take a multiple choice test, you not only provide your best answer, but you also indicate whether you are 'very sure', ... or ... 'not sure at all'.
The scoring of the test evaluates not only your degree of correctness, but also your level of confidence in gauging what you know and what you don't. At the conclusion of the testing process, decision makers are provided with profiles of their employees that reflect not only what the employee knows, but how usable and retainable the knowledge is. SACAT identifies misinformation that might cause employees to make serious errors. Nothing is more dangerous than an employee who is highly confident and is prepared to act quickly and decisively on a premise that is totally incorrect. SACAT identifies this risk. The approach has much to offer the manufacturing and petrochemical industries, where competence and confidence can be equally important in averting the risk of spills, fires, explosions."
From "Las Crucen adds new twist to multiple choice tests"
By Mike Mrkvicka
The Bulletin (Las Cruces), Feb 1, 1996
"A professor emeritus and former head of the psychology department at New Mexico State University, has spent nearly 30 years perfecting a modified multiple choice answer format that he calls SACAT- an acronym for Self Assessment Computer Analyzed Testing. Students taking the exam, after the answer is marked, must say how confident he or she is about the answer. When a number of students in a class mark the wrong answer, but are extremely sure they are correct, red flags rise. Are the students misinformed? Or did the test question mislead the students?
The SACAT system (is marketed) through his firm, Human Performance Enhancement, Inc. His raison d'etre for self-assessed testing is summarized in a paper he wrote last March for a training conference:
-
'The current methods for testing knowledge, in which it is determined whether a person can answer questions correctly, allow misinformation to be concealed. A wrong answer is taken simply to mean that the person is uninformed and has not yet learned the correct knowledge. However, the situation may be worse than this. The person may be misinformed and strongly believe that the wrong answer which he or she selected in correct. Misinformation not only leads to bad decisions and errors in performance, but also is counterproductive as a foundation for advanced learning.'
On the other hand, more than 90 percent of the students who were "extremely sure" about the correct answer retained the knowledge a week later.
The British Ministry of Defense has purchased the SACAT system and, with the participation of the U.S. Army Research Institute, is now conducting a project to determine the cost effectiveness of widespread use of SACAT."
From "Focus On HAZMAT: To Reduce Liability"
Dr. John S. Townsend
By permission of Industrial Fire World Magazine
"We need to know that our attendees (in training) actually learn what we intend to teach and that they understand what they have learned. When the training program fails, the trainee fails. Failure to properly monitor the liquid level in a tank being filled could result in a release of hazardous material and in reality there are very likely to be injuries and perhaps even fatalities. Training is vulnerable due to the difficulty encountered when one attempts to ascertain exactly what the trainee has learned and his degree of understanding.
A group of psychologists and educators began to investigate this problem in the early 1960s. Their work has resulted in a modification of the traditional multiple choice test called SACAT, an acronym for Self Assessment Computer Analyzed Testing. The technique alerts the astute instructor to the fact that the students have gotten misinformation on certain points. In some instances they can give the correct answer, but probably don't know why it is correct.
Because of the calculations involved, SACAT tests must be scored on an electronic scanner attached to a computer which utilizes (special) software. This scoring service is provided by Human Performance Enhancement, Inc., the publishers of the testing program. This testing regimen enables trainers and instructors to:
1. Detect and identify misinformation and to rectify misconceptions, misunderstandings or errors before they result in actions detrimental, counterproductive or generative of increased hazards.
2. Measure the student's usable knowledge of the training topic.
3. Measure the degree to which the learned material will be retained and available for recall as needed."
The utilization of the SACAT program allows those involved in industrial training to place a greater degree of confidence ... that a trainee will respond to a given situation in a correct and predictable way every time. It also gives us additional leverage in the courtroom. Now we can say with some certainty, 'Yes, this student did understand what he was doing (or was supposed to do) and we can prove it.' Finally, we will have at our disposal a means of measuring understanding. We can now prove that we not only taught, but that the student in fact learned, and also that what was learned was indeed understood."
SACAT test-takers comments:
"Helpful, it makes one more confident."
"I think it makes you study more to prepare for the test because you can't get away with just guessing the answer."
"Helpful to show a person what she actually knows rather than what she thought she knew."
"I feel that it is helpful as you can see if you are really learning or not."
"I think it is helpful. It lets you see the areas in which you didn't study."
"I realize, maybe too late, what idea or concepts I am not totally clear on."
"It helps one know that she actually knows."
"It has made me concentrate more while I'm reading the assigned material."
"It had an affect as it has caused me to put in more time studying the material."
"I think it can be somewhat a motivational tool for the student to take time to study more for the test. But I think it is more of a learning aid than a teaching tool."
"If it is adequately explained it is helpful to the students interest to think about whether or not he truly knows anything."
"It gives the teacher a basis on whether or not the material being taught to the class is being relayed and acknowledged properly."
"The self-assessment method is a useful tool in teaching as it gives the student a chance to see if he really understands the material or not."