Reliability in patient-centered observations of family physicians

Abstract

BACKGROUND AND OBJECTIVES: Patient-centered communication is an important component of primary care and related to improved patient health outcomes and satisfaction. The Patient-Centered Observation Form (PCOF) was developed as an educational assessment tool to improve resident physician-patient communication. However, reliability of the tool has not been tested.

METHODS: Residents and patients were observed in routine medical encounters in a Midwestern family medicine residency center as part of a prospective, quasi-experimental study. Four independent observers (two faculty clinicians and two social scientists) used the PCOF to rate videorecorded patient encounters in the areas of establishing rapport, maintaining relationships, agenda setting, efficiency, information gathering, assessing patient perspectives, effective and open use of the electronic medical record (EMR), sharing information, discussion of behavior changes, co-creating a plan, and shared decision making.

RESULTS: A total of 13 physician-patient encounters were observed. Mean overall reliability for the PCOF was 0.67 using four raters, 0.45 for clinicians only, and 0.62 for social scientists. Adequate reliability (>0.7) was found for behavior change discussion (0.89) in clinician ratings but not with social scientists (0.62). Social scientists had adequate reliability in assessing patients' perspectives on health (0.86) and shared decision making (0.78), but these were not considered reliable among clinicians (0.46 and 0.00, respectively).

CONCLUSIONS: Reliability of the PCOF for assessing patient-centered competence is dependent on the content of communication being scored and the training history of the evaluator. These results challenge researchers and physicians to develop more reliable scoring instructions and tools for assessing patient-centered competence.