Evaluating Medical Spanish Proficiency: A Comparison of Physician Assistant Student Self-Assessment to Standardized Patient and Expert Faculty Member Ratings

Christopher Forest, Desiree Lie, Regina Richter-Lagha, Forest/Lie (Co-First Authors)

Research output: Contribution to journalArticlepeer-review

Abstract

Purpose  
Whether physician assistant (PA) students' self-assessment or standardized patient (SP) evaluations of students' medical Spanish proficiency accurately reflect their language proficiency is unclear. This study compares PA student and SP ratings with an expert faculty member's rating to determine whether student or SP ratings can be used to evaluate language proficiency.
Methods  
Fifty-eight students participated in a single-station Spanish Objective Structured Clinical Examination (OSCE) at the midpoint of a medical Spanish curriculum. Using the Interagency Language Roundtable (ILR)—a 6-point, single-item language proficiency scale previously validated among physicians—PA students and SPs evaluated students' medical Spanish proficiency. Their scores were then compared with the scores derived by an expert faculty rater who had viewed a video of each student–SP encounter. The faculty's score was considered the gold standard. Correlation between scores was calculated using Spearman's rank correlation coefficient.
Results  
Mean student scores were highest when rated by SPs (M = 3.8, SD = 0.9), followed by self (M = 3.0, SD = 0.9), and then faculty (M = 2.5, SD = 1.2). Spearman's rank correlation coefficient showed a strong positive correlation between students and the expert faculty rater ( r s  = 0.67,  P  < .001) and between SPs and the expert faculty rater ( r s  = 0.72,  P  < .001). The correlation was stronger for high- than for low-proficiency students. Students' self-rated scores showed significant improvement from baseline to the OSCE.
Conclusions  
PA students participating in a medical Spanish curriculum and SPs show good correlation with an expert faculty rater in assessing Spanish proficiency during an OSCE. Standardized patients demonstrate scoring leniency. The ILR has potential for tracking aggregate student progress and curriculum effectiveness. With training, student self-rating could be used for interval assessment of medical Spanish communication.
Original languageAmerican English
JournalThe Journal of Physician Assistant Education
Volume29
DOIs
StatePublished - Sep 2018

Keywords

  • Spanish instruction
  • Spanish language proficiency
  • physician assistant students
  • OSCE

Disciplines

  • Bilingual, Multilingual, and Multicultural Education
  • Curriculum and Instruction
  • Educational Assessment, Evaluation, and Research
  • Medical Education

Cite this