Guidebook for Clerkship Directors
A curriculum is a plan, a path for students to follow, with a beginning, middle, and end. The word curriculum derives from the Latin word for “running.” Three elements describe the course that the student will run: a set of objectives, learning activities, and assessment of student competency. The objectives set expectations at the beginning of the course; that is, describing what students who “run the course” will be able to do. The learning activities are essentially the motor for getting from the start to the finish. At the end, the assessment system verifies how well students achieved these objectives.
A new clerkship director usually takes over an existing curriculum. The novice director may find that the original design met the needs for the department and the medical school at some point in the past, but the logic supporting that design is now lost. In addition, curricula often grow in piecemeal fashion, with new objectives, activities, and assessment pieces accreting over time. The design of a curriculum given to this new director can be confusing and cumbersome. Though occasionally the director may have the mandate to reconstruct the course from scratch, s/he often needs only to adjust the curriculum by repairing only the least effective parts.
Parts of any clerkship curriculum are fairly standard, and may not need much improvement. The core learning activity for any clerkship is the clinical care of actual patients, under supervision. The two standard assessment tools are a written exam, often the National Board of Medical Examiners’ subject exam for the specialty, and a preceptor rating of the student’s performance. These curricular elements may not need attention. The following steps can help the new clerkship director understand how to create any new curriculum piece, or perhaps even to create a completely new course. (return to top)
The process of curriculum creation or revision is an iterative process with several key steps.1 The steps include (1) analysis and synthesis of local needs and national trends; (2) creation of student learning objectives, learning activities, and a student assessment system; (3) securing of resources; and (4) curriculum evaluation before and after any change, as part of an overall quality improvement process.
All steps do NOT have to be present or in sequence. These steps do not have to and often cannot flow in a predetermined order. The department chair might mandate, for instance, a geriatric focus that fits in with an initiative by the residency program and the research division. There may be a reduction in funding that requires limiting the costs of clerkship administration, and thus there may be a need for creative revision of the learning activities and assessment of student learning. Perhaps a grant was awarded and funded and, as part of this funding, the curriculum must be changed to meet the grant’s objectives.
Analysis of local needs is politically one of the most potent steps. If there is a need within your medical school curriculum, the administration should support any course addressing that need. If the medical school’s curriculum committee, other departments’ clerkship faculty, one’s own faculty, and students can appreciate the local need that this curriculum will fill, these stakeholders should provide at least verbal support. Occasionally, this effort to meet local needs helps to gain financial support from the Dean’s Office. Similarly, clarification of local needs can also lead to collaboration with other departments and courses.
Synthesis of national trends and targeting the curriculum to address those needs may also lead to verbal and financial support from local stakeholders. This step can create opportunities for collaboration from within and outside the institution. Addressing national needs can also help in obtaining funding from external sources.
Consider this example: a national group created a women’s health curriculum. The OB/GYN clerkship director presented this curriculum to other core clerkship directors, and the pediatrics and family medicine clerkship leaders expressed interest in collaborating. The associate dean of medical education supported this collaboration because students have noted a lack of health promotion curriculum emphasis on the Association of American Medical College’s (AAMC) Graduation Questionnaire (GQ). The Dean’s Office offered to provide funding for the standardized patients to be used in assessment.
Evaluate current curriculum
A major step in curriculum change is assessment of the existing curriculum. Take the learning objectives, and create a spreadsheet with the objectives along each row, and the learning activities along each column. Similarly, construct a spreadsheet with the objectives along each row, and the assessment of student competence along the columns. Look for gaps, and then choose which ones to fill. Pick the changes that are most obvious and easiest to fill first. Then look for the changes that are most important or strategic. Sometimes, this step can reveal unintentional redundancy, and lead to pruning of unnecessary objectives, learning activities, or student assessment components.
Create tables / grids
Analyze student experience, opinion, and performance
The most powerful way to evaluate the present curriculum is through analysis of data from the students. The types of data include students’ patient care experience, student performance, and student opinion.
Students’ patient care experience can guide curricular change. A patient care log or survey can reveal if students are caring for an appropriate number of patients representing an appropriate demographic mix. The log can also reveal the breadth and depth of patient problems and procedures encountered, and verify the appropriate level of learner involvement. If there are skills or experiences that faculty have deemed essential for students to complete, their ability to fulfill these expectations will provide important feedback to the faculty. If the logbook reveals that students are seeing certain health problems not covered by the curriculum, there may be a missed opportunity to help the students gain competency in caring for patients with this problem.
For a student logbook to be most helpful, the logbook system must provide reports to faculty and students in timely fashion to guide not only clerkship administrative decisions, but also individual student behavior. If the student perceives a benefit from the logbook, then the quality of data recorded is likely to be higher than if there is no, or only a distant, connection between the data and an improvement to student experience.
Student performance can reveal deficiencies in the curriculum. If there is a national reference group for comparison, written exam performance can indicate areas requiring improved training. In-house written exams can provide more specific information about needed areas for improvement, because all of the performance data are available, not just a comparison to a national mean and standard deviation. Objective Structured Clinical Examinations (OSCEs) can provide a different type of information than written examinations, by measuring behavior-based outcomes, and not just knowledge.
Student opinion can guide curriculum improvement. Post-clerkship interviews or written surveys can provide immediate feedback while the memory of the specific experience remains fresh. The AAMC Graduation Questionnaire (GQ) provides student feedback that is compared to national norms, and also comes at the end of the student’s experience of the curriculum – so the student can compare all clerkships to each other, in the context of preparing for residency training.
Do student performance and opinion match the intent of the clerkship?
Based upon this analysis, an important question to ask is: “Is the curriculum achieving what we mean for it to achieve?” Do we need to adjust the objectives, learning activities, or assessment to match what we intend for the clerkship to accomplish? Are we missing an emphasis that is now important? For example, on a surgery clerkship, is there a need to have students learn more about minimally invasive surgery? Or, on a family medicine clerkship, is there now less need to have the students learn in an outpatient setting, and more of a need to have the students learn how to provide care within the context of a continuous relationship in the inpatient and extended care settings?
Take an example: clerkship students required orientation to the computer 5 years ago, and introduction to using MEDLINE. Over the past 2 to 3 years, however, the curriculum has changed in the Introduction to Clinical Medicine series. Now students come to the clerkship year prepared to use MEDLINE. This computer and MEDLINE orientation can now be removed from the clerkship.
Conduct Needs Assessment
Ask opinions of those involved in the local curriculum. What is working well? What could be improved? What would we be embarrassed for our students NOT to know by the end of the clerkship, and what can reasonably be accomplished in the given amount of time? What would be ideal for this clerkship to accomplish? 2
Seek the opinions of recent graduates in primary care (general pediatrics, general internal medicine, and family medicine) and non-primary care residencies. Did first-year residents feel well prepared compared to graduates from other schools? What do they wish the medical curriculum had provided more or less of? Look at the data organized by residents in primary care, non-primary care, and one’s own specialty.
Ask residents and fellows how to improve the curriculum. Most clerkships rely heavily upon these trainees as teachers. How does the curriculum frustrate them? What can be done to help them fulfill their roles as teachers? The clerkship administration and faculty may not fully understand the realities of the clinical practice setting, and residents and fellows can provide valuable insight and guidance. The clerkship directormight think, for instance, that every student should be competent to place a peripheral venous catheter, but a resident can easily clarify that the hospital has a venous access team that limits any opportunity for students to practice this skill.
Ask the opinions of those in leadership positions (department chair, dean, assistant/associate dean/s). Listen to how these decision-makers measure success. Identify the curricular themes that they want emphasized. How well do clerkship goals fit in with the major trends within the college? Work to clarify the vision of the chair, dean, associate or assistant deans, Curriculum Committee, and reconcile any differences of viewpoint. There may be a hidden agenda that needs to be addressed, or, at least, a stakeholder’s viewpoint that needs to be understood. Does the chair believe that a major purpose of the clerkship is to recruit students into residency training, and you disagree with this emphasis? Do you and the dean seek to educate all students, but your department’s faculty members wish to train only students interested in their specific specialty?
Build upon the medical education group’s strengths and interests. A faculty member who is passionate about a topic and a unit or center of excellence recognized for its research presence can be a very powerful force for curriculum change and effective instruction.
Ask third- and fourth-year students about the curriculum. What worked well and what could be improved? A simple survey with both closed- and open-ended responses can be enough for simple levels of information. Using open-ended questions first can help develop a closed-ended question survey. Another method, one that is more time- and resource-intensive, is conducting a focus group. Focus groups typically are good for finding out the underlying beliefs of a group – how students truly view this curriculum.
Find and report opinions of national leaders or other institutions. Review the medical education literature, and attend presentations at national meetings to look for trends and ideas.
Use the LCME accreditation survey data and USMLE exam performance to guide changes. Prepare for the next LCME site visit, or respond to comments from the most recent site visit. Reflect upon students’ USMLE Step 2 and 2B performance, and see where this clerkship might play a helpful role in improving these measures.
Transform All These Data into Information
Some data are background “noise,” and not music. At least 10% of students will have strong negative opinions about any recent curriculum change. What can and should be addressed and/or changed immediately? What changes are like “low-hanging fruit” – steps taken that are obvious, positive, and can be done easily and quickly? What issues are less urgent, but need long-term attention? What are system problems that you either have to endure or address on a grander scale? What creative opportunities exist? Are there curricular topics that your clerkship currently covers that should be addressed either in another clerkship or at a different stage in training, shared with another clerkship, eliminated, or addressed in an entirely different manner?
Consider this example: a faculty member takes over as the internal medicine clerkship director. She is faced with a problem with one of the rotation settings. One hospital has a large population of patients followed by community physicians. The students complain that they do not get much opportunity to manage these patients fully, because the private physicians feel uncomfortable with relinquishing control of their patients’ care to students. Together with course faculty and the community hospital physicians, the director develops a set of objectives that focus the students’ efforts on a specific set of patient presentations and problems. She develops a “passport” that students present to preceptors as each required presentation or problem is encountered. The community physicians feel more in control of the education, and support the curriculum. Students rate the educational experience higher because they document achievement of the clerkship’s required objectives.
Create a Curriculum “Grid”
Look for gaps in the curriculum. There may be learning objectives that you aren’t addressing but should. Consider adding in learning activities that fit in with important objectives. Make sure that assessment of student competence matches learning objectives and activities.
Collate and Report Data
Meet with faculty and discuss findings. Summarize medical education research findings, and report what is being done elsewhere (literature review, conference proceedings, discussion with colleagues elsewhere.) Get buy-in on need for curricular change. Find funding sources. Work towards achieving consensus.
Develop an Action Plan for Change
Create a team that will guide this change. Give a charter to this team that defines both process and outcome measures of success. Ensure that team members own relevant parts of the process – that these people will be involved in delivering this changed curriculum. Are those who say that objectives must be met really the individuals who will see to it that the objectives are met? Successful curriculum implementation is utterly dependent upon buy-in from faculty who need to feel ownership in the process and product.
Remember that among busy clinical faculty, what sounds like a good idea at the beginning can quickly lose its appeal if small successive measures of progress are not marked. Defining how the team meets can improve their performance. Meeting more often than once per month, for example, helps the group to feel more cohesive, and allows for greater momentum. However, regular interruption of a busy schedule in the clinic, office, hospital, and operating room will punish participants, and can limit attendance. Conversely, if the group convenes only once a month, much of the meeting can be spent on recalling previous work. Setting and adhering to a time line will keep efforts focused and efficiently use committee members’ time.
A learning objective is a goal that describes exactly what the student will be able to demonstrate after successful completion of the curriculum. A well-designed objective clearly describes the end result of training. Use verbs that capture the highest appropriate level of competence.3,4
Learning objectives must honor the overall purpose of the clerkship, and effectively make the clerkship’s goals operational. The relationship between student learning objectives and clerkship goals, however, is complex. Clerkship goals direct the students’ learning objectives, and creating the objectives clarifies and may reframe the true purpose of the clerkship.
Well-written objectives will ensure that the learner knows what s/he should be able to do, under what conditions, and how well it must be done. If objectives are to be meaningful and to serve as a map for what the faculty consider important for students to learn, then these objectives need to be clear, succinct, and doable. And students must be accountable for achieving them.
Objectives must be behavior-specific. Use verbs that describe a student behavior that is observable. To the degree possible, detail the setting or condition for each objective, at the level of student behavior. For example, instead of a vague statement like “Learn how to suture,” write:
National curricula offer a useful resource for guidance in creating and/or revising a clerkship curriculum. Many specialties have defined learning objectives, and provided resources and advice for improving core junior clerkships.
What is the best means for students to achieve these objectives? What activities fit in with the curriculum best, what costs the least while achieving reasonable results, and what activities are necessary to provide an opportunity for students to gain special competencies? Remember that if you are going to hold students accountable, the means of achieving the skill and the ends of evaluating skill acquisition need to be congruent. The default learning activity for a typical clerkship is to have the students care for patients within the clerkship’s typical context of care, the hospital or office setting. Within this framework of “seeing patients,” clerkships address many objectives.
Special projects can address more learning objectives that do not occur reliably within the course of usual work with a preceptor. Incorporating Evidence-Based Medicine (EBM) at the point of clinical care, for instance, might occur in the course of usual care of patients. To ensure that every student gets the same opportunity to practice these skills up to a specific level of competence, a separate activity could be required. Each student could be asked to identify a clinical question amenable to primary literature review, and submit a written report that describes the clinical situation, the search strategy for PubMed, and an analysis of the evidence found, with application back to the original situation. As another example, to offer students a perspective of anatomy that is important in the operating room, clinical anatomy sessions with surgeons can be developed to guide students through common operative cases in a non-threatening environment.
Assessment of Student Achievement of Learning Objectives
How are you going to address what knowledge, skills, and attitudes will be taught? How will you assess student achievement of those knowledge, skills, and attitudes? How will you hold students accountable for achieving the objectives?
If you want to assess students’ clinical performance, don’t give them a written test—watch them perform. This can be done on the wards or through an OSCE. If you want to see if students are able to apply their textbook-derived knowledge to a clinical situation, conduct a performance-based assessment linked to salient points in their reading.
Remember that tasks students perform should have utility that will enable you to assess and improve the experience. Don’t merely collect data—use it!
Are the objectives truly attainable given the time and resources available in the clerkship? Can the objectives pass the litmus test of being used as teaching tools and then as evaluation tools? (return to top)
Though any discussion of resources tends to emphasize external funding, internal sources of funding are critical. One of the big issues that may be under a clerkship director's influence is administrative infrastructure. The clerkship director has to align organizational costs with administrative and teaching support from the department and college.
Clerkship structure can affect greatly the need here. A university hospital setting for a ward rotation, where all students can easily come together centrally for orientation, didactic sessions, and assessment, will need much less administrative support than a geographically distributed clerkship. Innovative educational methods such as Web-based bulletin board discussions require extra administrative support. Logistically complex assessment tools such as OSCEs also require more resources. If there isn’t adequate support for the clerkship’s activities, then the clerkship director may have to eliminate the more resource-intensive teaching and assessment activities.
Recruitment and retention of high quality preceptors requires substantial and continuous effort for clerkships that rely on community physicians. Though the preceptors may be willing to teach without compensation, the clerkship director should offer regional or on-site faculty development. University faculty also need faculty development (See also Chapter 8, Faculty Development)
The cost of preceptor time may be hidden from the clerkship director, whether the preceptors are volunteers in the community or full-time faculty on the tenure track at the university hospital. The effort to develop preceptor skills, however, can cost money. For example, the clerkship director may have to convince the department’s chair to free up faculty from clinical obligations for educational activities. (return to top)
Clerkship Directors often institute changes without obtaining a full set of baseline data about the course’s effectiveness. Many curriculum changes appear logical, as they respond to national and local needs, while fitting into the available resources. But, if the curriculum is not evaluated before and after any change, the clerkship director may never be able to prove that a change actually improved educational outcomes.
Curriculum evaluation should include different types of data, from differing viewpoints.
Solicit opinions about the experience from two major sources: teachers and students. Faculty, residents, and fellows can offer their opinions about the experience. Students can offer general opinions about the experience, too. Students can also self-assess, and provide, for example, a measure of comfort with specific knowledge or skills.
Student patient care experience is an important measure of clerkship success. Though opinion can provide important information about the curriculum, objective data about student experience is critical. Student logbooks or surveys of patient care experience can provide essential information about the clerkship. Tracking patient care experience is also important for accreditation. The Liaison Committee on Medical Education (LCME) has established that each school must demonstrate that students are encountering an appropriate mix of patient problems (See also Chapter 15, The Clerkship Director and the Accreditation Process).
Evaluation of student competency is the essential measure for an effective clerkship. Even if all other measures fall short of their intended mark, this measure is critical.
Though preceptor evaluation of student competence has been criticized as overly subjective, there is no substitute for an expert, seasoned clinician’s point of view. There are many published efforts to improve the reproducibility and increase the specificity of preceptor evaluations.5
Preceptor judgment of student’s competence to perform specific tasks, such as lumbar puncture, is a specific indicator of competence. National Board of Medical Examiners’ (NBME) subject exams or other Multiple Choice Question (MCQ) examinations can assess knowledge. Clinical Practice Examinations or OSCEs can test a student’s ability to demonstrate interviewing, physical examination, and negotiation or counseling skills. Other formal examinations, including short answer, matching, and oral exam, can provide helpful information. Project evaluation (e.g., a home visit project) is also important.
Quality Improvement (QI) of the Curriculum
Langley, Nolan, and Nolan created a model for Quality Improvement (QI) that is highly applicable to curriculum creation and revision.6 QI brings together all of the steps above, especially curriculum evaluation before and after any change. This model asks three basic questions: what is our aim; how will we know that any change leads to an improvement; and what changes can we test that might lead to an improvement?
Figure 2 details a Plan-Do-Study-Act (PDSA) cycle for testing change. Planning involves comparing the existing curriculum to local needs and national trends. Doing is constrained by resource availability, and logistics. The context of the curriculum and the personalities of the faculty can affect how the curriculum is implemented. The Study part of the PDSA cycle depends greatly upon having established clear ways of knowing that a change is leading to a true improvement prior to implementing or altering the curriculum. At its simplest level, the Act phase answers the question: should we adopt this change to the curriculum?
At a more complex level, one change leads to another. As Figure 3 demonstrates visually, serial PDSA cycles can take the curriculum from its present state toward the ideal future.
The 3-Question Model for Curriculum Improvement
*Hint: Click Back button on browser to return to previous spot in text
A Plan-Do-Study-Act (PDSA) Cycle for Curriculum Change
*Hint: Click Back button on browser to return to previous spot in text
Serial PDSA cycles for sequential, incremental improvement
*Hint: Click Back button on browser to return to previous spot in text