Combining Different Components Of An Assessment Procedure

Degryse J, Goedhuys J, Van Puyenbroeck H and Sunaert P
University of Leuven, Antwerp and Gent. Belgium.
In Flanders the different University Departments of General Practice have made a collaborative effort to organise a common final assessment procedure at the end of the vocational training period in General Practice as a certification exam.
This summative assessment procedure is built out of four different components: a well-designed written knowledge test, a structured oral exam, a large scale OSCE type exam and a structured report from the trainers. A pass-fail decision is, based on a combination of the scores obtained from the different modules.
This presentation discusses the different approaches that can be used to combine the different component scores. The practice of combining scores by simple addition of the sub scores seems straightforward. Yet there are convincing reasons for abandoning this approach. From the psychometric perspective a compensatory, conjunctive or a hybrid scoring model can be argued. But there may be good empirical or administrative reasons for using a particular approach. The way different component scores are combined may also influence the behaviour of trainees in an exam.
The discussion is illustrated with data collected from the final assessment procedure in 1996-1998. The impact of using different approaches and computation methods is simulated and discussed.
The choice for a specific combination method should be inspired by the psychometric characteristics of the assessment tools but the final decision is made on an empirical base.
Combining different components of an assessment procedure