The night float rotation had been in place for a number of years in the Paediatric Residency Program at the University of Alberta. Its purpose was to free up residents on their CTU (clinical teaching unit) rotations from doing overnight call (and thus from being post-call) and to provide continuity for inpatients. It consists of five, 15-hour shifts per week not including handover time, with in-house coverage for all inpatient care, as well as consultations for all inpatient admissions provided by one senior and one junior paediatric resident, as well as an off-service resident and anywhere between one and five medical students. While there is considerable potential for education through this work, with an opportunity to function with more autonomy and with greater responsibility for a larger number of diverse patients than that seen during regular hours, residents were not previously receiving structured feedback on the rotation.

Dr. Karen Forbes had led a faculty-proposed night float assessment system to try and address the educational and assessment needs of this rotation. However, this proposal was unsatisfactory to residents for several reasons. I became involved in this project in order to provide a voice to the residents by analyzing and articulating concerns with the proposal. I had concluded that for one, during these 75-hour work weeks, this demanding multi-source feedback proposal called for residents to assess one another weekly and every medical student on a nightly basis, while simultaneously making it the resident’s responsibility to have staff evaluate them on each shift and receive an evaluation by phone or face to face with attending physicians. I also took issue with this evaluation system allowing for input from staff for 1-2 out of the 3 teams, given the fact that the third team had a completely different patient population and multiple covering physicians. Further, because attending staff almost invariably remained at home throughout the night, they had no opportunity to directly observe resident practices in a variety of CanMEDs competency domains, only receiving a snapshot of a few of the varied tasks being completed by residents in the care of the team through brief communications. I had also feared that asking staff to assess too many competencies based on indirect communication could lead to them completing assessments based on pre-conceived notions of residents formed through previous daytime CTU observations, rather than on the current particular stretch of nights. Finally, while I saw value in peer evaluation, I felt that it would be challenging for residents to assign specific scores to one another in an identifiable fashion.

From these concerns, I met with Dr. Forbes and two of the paediatric chief residents over the 2011 year, and proposed solutions that were integrated into the assessments that took place. Specifically, I had requested that:

• Medical students be administered evaluation forms for senior paediatric residents, with optional evaluations of any junior residents that they had worked with in some capacity.

• Pediatric residents fill out evaluation forms for one another based on a rubric, to provide clear feedback that could be justified based on the various statements within that rubric. Residents would be asked to evaluate one another within each domain as needing improvement, satisfactory, or well-developed. Prior proposals had called for reciprocal resident evaluations of junior and senior residents involving a 5-point Likert scale.

• On-call attending physicians evaluate residents based on specific case presentations discussed by phone, which could assume two forms: reviewing a consultation for admission (including discussion of history, physical examination, investigations, impression, problem list, and plans), or reviewing acute clinical problems for inpatients that require re-assessment and a change in plans. Residents would request that the on-call attending complete the assessments prior to presentation, so that the on-call attending physician is primed to provide assessment and feedback. This would be formalized, with physicians able to request and complete evaluations online (through One45) immediately after the case discussion. Evaluations would take the form of encounter cards.

• Time management and teaching be removed as domains in which staff would evaluate residents, because of the absence of direct observation. Additionally, “not able to assess” should be added as a potential choice on evaluation forms, given the variability of service and collaboration between residents and staff.

Dr. Forbes and other collaborators agreed with these recommendations, which were all ultimately incorporated into the night float evaluation proposal that was approved by the residency training committee in January 2012 to be implemented as a six-month pilot project. I am happy to state that both residents and staff were generally happy with this evaluation system, and it has been continued to present.

Examples of evaluations generated:

• Reciprocal evaluations of junior and senior paediatric residents

• Night float assessment for single encounter by on-call attending physician