Exploring the Gap Between Evidence and Judgement: using video vignettes for practice-based assessment of physiotherapy undergraduates
Abstract
The range of conditions necessary for valid and reliable assessment of clinical competence may result in a gap between amount of evidence available from performance and that required for safe inferences of competence. Two assessment forms were compared for validity and reliability, using six video vignettes of undergraduates on placement. Form A was currently in use on the programme. Form B was developed from a Delphi study involving 108 physiotherapy practitioners' perceptions of competence. Effects of training on assessment decisions were also investigated. Results indicated wide differences in individual ability to assess students. Good students tended to be rated less positively than deserved and poor students better than deserved. Judgements were more valid and reliable on Form B than on Form A (A: ω = 0.496, rho = 0.61; B: ω = 0.647, rho = 0.71) Judgements on both forms were more reliable after training than before (before: ω = 0.524, rho = 0.62; after: ω = 0.620, rho = 0.70). Factor analysis of assessment data from both forms indicated Form B had greater validity amongst clinical assessors. It is concluded that video vignettes are effective in monitoring assessors' judgements and helping to identify the amount of evidence that can reasonably and reliably be collected by clinicians assessing undergraduates in the clinical environment.Citation
Assessment & Evaluation in Higher Education, 26 (3): 189-212Publisher
Taylor & FrancisJournal
Assessment & Evaluation in Higher EducationType
Journal articleLanguage
enISSN
026029381469297X
ae974a485f413a2113503eed53cd6c53
10.1080/02602930120052369