Comparing the importance of clinical competence criteria across specialties: impact on undergraduate assessment

Abstract

Quality measurement in healthcare and higher education indicates the need for a systematic approach to developing undergraduate clinical competence assessment. Validity and reliability may be undermined by differences in assessors' interpretation of what is important. Differing contexts of undergraduates' clinical experience could result in assessors' ratings of activities being deemed less important, omitted or rendered meaningless. This study investigated the level of agreement across and within five clinical specialties in physiotherapy on the relative importance of 89 activities associated with clinical competence. One-way analysis of variance for each activity revealed 12 items differentially rated (p values = 0.05, 0.01 and 0.001). Kendall's coefficient of concordance demonstrated within-group agreement (p = < 0.000). Factor analysis of items upon which there was maximum agreement across specialties, combined with split half reliability analysis (Cronbach's alpha) resulted in eight reliable factors. These included task-specific and generic transferable skills. It was concluded that the factors provided a basis for discussion about clinicians' and academics' contributions to assessment, and a starting point for development of a clinical assessment instrument that could optimise the validity and reliability of clinical assessment decisions.