TY - JOUR
T1 - A SIMPL application to assess oral medicine residents' performance and level of autonomy.
AU - Villa, Alessandro
AU - Shekar, R.
AU - George, B. C.
AU - Bohnen, J. D.
N1 - 1 Division of Oral Medicine and Dentistry, Brigham and Women's Hospital, Department of Oral Medicine, Infection and Immunity, Harvard School of Dental Medicine, Boston, MA, USA. Electronic address: [email protected]. 2 Department of Surgery, University of Michigan, Ann Arbor, MI, USA. 3 Department of Surgery, Massachusetts General Hospital, Boston, MA, USA.
PY - 2018/9
Y1 - 2018/9
N2 - Objectives: The aim of this study was to evaluate a smartphone application, SIMPL, to optimize faculty guidance and oral medicine residents' performance, and to measure resident-faculty agreement for performance and supervision levels. Study design: Raters used the 5-level "Performance" scale to assess resident's readiness for independent practice and the 4-level Zwisch scale to assess faculty guidance, between June 2016 and June 2017. Junior residents were trainees with less than 18 months of training, and senior residents had more than 18 months of training. Descriptive statistics were used to analyze supervision and autonomy. Performance and supervision agreement was estimated as percentage-agreement and measured using κ and 95% confidence intervals. Results: A total of 660 evaluations were performed by 6 residents and 6 faculty members. Senior residents received higher performance scores compared with junior residents (P for trend < .01). In terms of complexity of cases and level of supervision, there were no significant differences between juniors and seniors (P = .69 and P = .39, respectively). Residents were "practice ready" or greater than for 80% cases. Residents received meaningful autonomy for 64.8% cases. Faculty-resident concordance was 86.1% for performance (with κ of 0.77 [95% CI 66.1%-87.6%]) and 92.4% for supervision (with κ of 0.84 [95% CI 80%-88%]). Conclusions: SIMPL can feasibAplly be used for real-time assessment of residents' performance and autonomy.
AB - Objectives: The aim of this study was to evaluate a smartphone application, SIMPL, to optimize faculty guidance and oral medicine residents' performance, and to measure resident-faculty agreement for performance and supervision levels. Study design: Raters used the 5-level "Performance" scale to assess resident's readiness for independent practice and the 4-level Zwisch scale to assess faculty guidance, between June 2016 and June 2017. Junior residents were trainees with less than 18 months of training, and senior residents had more than 18 months of training. Descriptive statistics were used to analyze supervision and autonomy. Performance and supervision agreement was estimated as percentage-agreement and measured using κ and 95% confidence intervals. Results: A total of 660 evaluations were performed by 6 residents and 6 faculty members. Senior residents received higher performance scores compared with junior residents (P for trend < .01). In terms of complexity of cases and level of supervision, there were no significant differences between juniors and seniors (P = .69 and P = .39, respectively). Residents were "practice ready" or greater than for 80% cases. Residents received meaningful autonomy for 64.8% cases. Faculty-resident concordance was 86.1% for performance (with κ of 0.77 [95% CI 66.1%-87.6%]) and 92.4% for supervision (with κ of 0.84 [95% CI 80%-88%]). Conclusions: SIMPL can feasibAplly be used for real-time assessment of residents' performance and autonomy.
KW - Application
KW - Assess
KW - Autonomy
KW - Oral Medicine
KW - Performance
UR - https://pubmed.ncbi.nlm.nih.gov/29937360/
U2 - 10.1016/j.oooo.2018.05.017
DO - 10.1016/j.oooo.2018.05.017
M3 - Article
VL - 126
JO - Oral Surgery, Oral Medicine, Oral Pathology, and Oral Radiology
JF - Oral Surgery, Oral Medicine, Oral Pathology, and Oral Radiology
ER -