|
Home | Mission | Working Relationships | Position Papers | Publications | Clerkship Administrator Resources |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Guidebook for Clerkship Directors
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Guidebook for Clerkship Directors | Annual Meetings | Member Organization Links | LCME ED-2 | Contact Us |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Printed version of Guidebook is available. |
|
| Sections on this page: | |
|
|
<Chapter 7: Evaluation of the Clerkship: Clinical Teachers and Program |
|
“Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”
-Abraham Lincoln
Clerkship directors are responsible for overseeing all components of their student program, but even a well-constructed syllabus with clearly articulated learning objectives, the use of state-of-the-art instructional strategies, and sophisticated performance and program evaluation methodologies will be compromised if the faculty, residents, and community preceptors are not effectively prepared for their roles in the clerkship. It has been said that mediocrity is the enemy of excellence.1 Clerkship directors seeking excellence in their student programs must optimize the contributions of the teachers who are involved as educators, administrators, researchers, communicators, and advisors to students. However, facilitating faculty members’ success in these roles has grown increasingly difficult because of competing pressures of patient care and research. Thus, clerkship directors need to frame faculty development initiatives as part of the continuous cycle of life-long learning and professional development. Katz and Henry2 remind us that teachers need to see themselves as more than transmitters of received knowledge. They need to see themselves as professional and accountable educators who comprehend the connection between teaching and learning, a concept that harkens back to the work of John Dewey in the early 1900s. (return to top)
The purpose of this chapter is to advise clerkship directors on how to justify, develop, implement, and evaluate faculty development programs designed to equip full-time and volunteer faculty and residents with the knowledge and skills needed for them to foster excellence in the student clerkship. Faculty development is defined as a process by which teachers in the clinical setting work systematically to improve their skills in (1) education, (2) mentoring, (3) the scholarship of teaching, and (4) designing and implementing a professional development plan as educators.
The chapter’s content is organized to help readers with the:
Faculty development is critical for meeting accreditation requirements, helping teachers succeed as facilitators of effective and efficient learning, enabling faculty to pursue teaching as scholarship, ongoing quality improvement of the clerkship, and establishing a professional development plan for faculty members’ continued growth as teachers and educators.
The Accreditation Council for Graduate Medical Education (ACGME) and the Association of American Medical College’s Licensing Committee on Medical Education (LCME) both include language in their accreditation requirements stipulating the need for faculty and resident development as teachers.3,4
LCME Standard ED-24 states that “residents who supervise or teach medical students, as well as graduate students and postdoctoral fellows in the biomedical sciences who serve as teachers or teaching assistants, must be familiar with the educational objectives of the course or clerkship and be prepared for their roles in teaching and evaluation.” In 2004, the AAMC website stated that 38% of medical schools undergoing an accreditation site visit were cited for violating this standard. A similar requirement relevant to faculty (FA-4) states, “members of the faculty must have the capability and continued commitment to be effective teachers.” (See also Chapter 16, The Clerkship Director and the Accreditation Process).
The ACGME’s Outcome Project describes each of the six competencies that residency programs are responsible for to ensure resident proficiency. One of the competencies entitled “Practice-Based Improvement and Learning” includes in its definition the need for residents to “facilitate the learning of students and other health professionals.” Therefore, both residency program directors and clerkship directors have responsibilities to their education programs to ensure that those individuals involved in teaching the learners are knowledgeable about the clerkship’s learning objectives and have opportunities to advance their teaching skills.
The need for efficient and effective learning increases as the time available to spend with students decreases. This is due to competing responsibilities for faculty and limited work hours for residents. McKeachie makes this argument in his book McKeachie’s Teaching Tips by stating: “Teaching skillfully may be less time consuming than teaching badly. Teaching well is more fun than teaching poorly. Thus some investment of time and attention to developing skill in teaching is likely to have substantial payoff in self-satisfaction and effectiveness in your career.”5
Faculty development programs aimed at teaching skills have been shown to have both a positive impact on participants’ self assessment of these skills, as well as assessments from their learners.6-13 These programs may be a series of workshops or courses, retreats, self study, small group longitudinal coaching, or a combination.
As part of their clerkship evaluation system, clerkship directors should receive reports on the quality of individual faculty and resident teaching skills as well as reports on other components of the clerkship reflecting its strengths and weaknesses. These ratings often originate from students, but can also reflect peer, outside review, or self-assessment ratings. These evaluation data can serve as powerful justification for faculty development. If faculty or residents who receive negative teaching ratings are to improve, they need to be given feedback, the feedback must be viewed as credible, and they need to learn HOW to change their teaching behaviors.14 Faculty development can help bolster their self awareness and provide them with strategies for enhancing their teaching skills.
Evaluation data about the clerkship program can serve as another source for quality improvement. Data from the AAMC questionnaire completed by medical students show how each clerkship is rated by its students and offer comparisons with other medical schools. Mediocre or poor clerkship rotation ratings completed locally at the end of the clerkship can also be used to justify support for faculty development to the chairman. A faculty development plan connected to the noted weaknesses in the clerkship might serve as part of a convincing set of strategies for ongoing clerkship quality improvement (See also Chapter 7, Evaluation of the Clerkship: Clinical Teachers and Program).
There is increasing acceptance of teaching as a basis for scholarly work. In 1990, Ernest Boyer’s book, Scholarship Reconsidered, extended the traditional definition and criteria of scholarship to include teachers who keep up with current developments, who devise better ways to help students learn, or who pursue research on questions relevant to education.15 Faculty development programs can provide opportunities for faculty to learn how to apply their interests in education to produce scientific papers or “products” that are original, peer reviewed, and disseminated for others to use (See also Chapter 14, Educational Scholarship).
These points of justification provide a much stronger argument in support of faculty development at institutions that value, reward, and recognize the broader definition of scholarship in their tenure and promotion policies. Given the growing development of clinical-educator career tracks, clerkship directors are increasingly better poised to argue for faculty development to foster skill sets of these individuals using education and teaching as their basis for promotion.
Faculty development is often viewed as a collection of “Train the Trainer” type workshops. One of the important justifications for faculty development is that it isn't just a “one-shot” intervention, but a continuous series of efforts that help faculty evolve in their knowledge and skills as educators. Effective faculty development stimulates reflection – thinking about what each teacher wants to accomplish and what they and their learners need to do to achieve these goals. Without the faculty development component, faculty members often view teaching and its associated responsibilities as an expectation of others, rather than a privileged responsibility sought by the teacher.
Well-orchestrated faculty development programs can also have a positive effect on recruitment and retention. Most faculty, community preceptors, and residents want to be good teachers, and welcome the opportunity to hone their skills if it is made available in a user-friendly and meaningful way. Retention can be affected if solid, front-line teachers aren't offered opportunities to fine tune their skills as educators, or learn how to translate their efforts as educators into scholarly work.
Faculty development is justified because it
|
Faculty development requires resources. The level of resources available to the clerkship director will determine the scope of faculty development efforts. (return to top)
Resources, which can include money, space, technology, human resources, print materials, and time, can be sought from several funding sources, including the Dean’s Office, Graduate Medical Education office or hospital, Educational Resource units, and departmental leadership. Resources for faculty development should be a line item in any clerkship director’s budget.
It is convenient, but not necessary, for the clerkship director to be an expert in faculty development. However, clerkship directors should be aware of various available sources and resources. Different types of faculty development programs are available through most professional societies (for example, the Association for Surgical Education, Society of Teachers in Family Medicine, and the Clerkship Directors in Internal Medicine).
Many medical schools and hospitals have education specialists who are available to help plan or deliver faculty development programs for their faculty or staff. Some medical schools work with internal professional educators, clinical or basic science faculty in other departments or in affiliated schools. The main responsibility of the clerkship director is to orchestrate the efforts in faculty development, preferably with a team, to help those involved in the clerkship program succeed. (return to top)
Clerkship directors feel the pressure of these diverse preferences. To ensure that their faculty members are effective educators, which includes teaching for learning, providing feedback efficiently, grading reliably, and revising the curriculum and pedagogy in response to program, system, and discipline-specific needs, each clerkship director must establish an explicit agenda for faculty development.
Setting the faculty development agenda is a three-step process. The first step is to review the learning objectives for medical students in your clerkship, as well as those across discipline competencies. The second step is to articulate the various roles and functions of the people involved in your clerkship who will support and facilitate meeting those learning objectives. The third step involves the clerkship director’s careful consideration of what topics and issues are most appropriate in the local context, given the roles and expectations of faculty, residents, and volunteer faculty in their clerkship. Taking those various roles and needs into account, clerkship directors can then select appropriate topics for faculty development initiatives.
First, clerkship directors should review the general competencies (such as those provided in the AAMC Medical Students Objectives Project (MSOP).17 Then they should review the LCME accreditation standards, medical-school-wide curriculum competencies, and their specialty-specific learning objectives associated with the clerkship. It is helpful to also review associated data sets (e.g., AAMC Graduation Questionnaire) to identify potential foci for faculty development.
Bland and colleagues18 identified four types of faculty (preceptors, non-tenure-track faculty, tenure-track community-based, and tenure-track university-based faculty) and five categories of faculty development based on roles (educator, administrator, researcher, written communicator, and professional academic). The faculty development needs for the individuals involved in your clerkship will vary depending upon the individual’s role. By examining the types of individuals in your clerkship and their role(s), you can determine priorities for faculty development. When the same individuals fill multiple roles, a longitudinal faculty development program may be cost effective. A brief description of each role and potential faculty development topics follow.
Setting/Revising Curriculum
Faculty members need to be aware of the clerkship goals and objectives. Helping teachers to stay connected with curricular purpose is one way to encourage curricular growth and change. Individuals in the Educator role need skills in curriculum development for their assigned curriculum components (for example, a rotation or lecture series) for which they are responsible. Being able to align and properly document how their curriculum unit fits with the clerkship goals and objectives, and then matching instructional methods and materials (which may include education technology) to those objectives is key to having a sound and consistent program.
Strategies for Helping Students Learn (Educational Delivery)
Faculty need to expand their repertoire of effective educational delivery strategies specific to the contexts in which they teach. Whether engaged in one-on-one teaching or facilitating large group interactive lectures, good educational delivery requires thoughtful understanding of the learning process and connection between teacher and student.
Topics for faculty development in this area might include:
Learner Assessment and Clerkship Evaluation
Pop quiz: What’s the difference between assessment and evaluation? Where does “grading” fit in? Unfortunately, we have moved away from the original intention of the word assessment, which comes from the Latin assidere and means sit by. In an ideal educational world, assessment would involve teacher and student, side by side, comfortably discussing the learning-to-date and the learning to come. In reality, we struggle with tests, checklists, and other evidence that the learner has somehow transformed because of the experience, and that these changes will last. In aggregate, these sources of evidence are used to derive the student’s grade (See also Chapter 6, Evaluation and Grading of Students). Faculty development in this area needs to be carefully crafted to match the needs of the faculty responsible for assessing the students.
For multi-site clerkships, equivalent grading across clerkship sites is important to the program and the students, and mandated by the LCME accreditation standards. If site directors are expected to submit grades for their students to the clerkship director, they will need assistance in how to assimilate performance assessment data consistent with the agreed-upon standards for the clerkship (See also Chapter 9, Directing a Clerkship Across Geographically Separated Sites).
Learner assessment results provide the clerkship educator with evidence of whether the students have learned. However, to determine the value or worth of the educational experiences, the effectiveness of teachers, the relative contribution of core curriculum to the achievement of objectives, the educator must conduct an evaluation of the program. The specific foci for the evaluation are determined by multiple stakeholders, (e.g., the clerkship director, the department chair, the school-wide curriculum committee, or accreditation standards) and the purpose (e.g., formative data to improve the clerkship) (See also Chapter 7, Evaluation of the Clerkship: Clinical Teachers and Program).
Educator-role related faculty development topics in the area of assessment and evaluation include:
Faculty development topics may relate to the environment, organization, leadership, or management. Examples of each follow.
Environment
Good education administrators (e.g., clerkship director, rotation or site director, chief resident) know about the economic, political, and organizational landscape that impacts academic medical centers and subsequently the clerkship. They also are aware of resource availability and their learners‘ needs. Site directors need to understand the impact of the culture and learning environment on their students, and work to systematically improve their site-specific learning climate.
A faculty development topic to achieve an optimal learning environment might include instruction on resource negotiation for the clerkship program.
Organization
Organizations are composed of inter-related and inter-dependent departments and programs. Each has its own goals and priorities, and yet maintains a strong connection to the larger group. Effective administrators recognize the parts, the whole, and the interrelationships among them. Clerkship Directors need to help other administrators appreciate what is taught in other components of the clerkship (e.g., core curriculum) so that unnecessary redundancy can be avoided, and purposeful repetition planned. All faculty members need to understand the mission and core values of the institution as it relates to education. However, clerkship-related administrators must effectively use their role in furthering the mission through the clerkship.
Sample faculty development topics on organization might include:
Leadership
Providing leadership in a clerkship can be challenging because it requires ability to recognize various leadership and work styles in oneself and others, work with the content and process of group interaction to achieve desirable outcomes, and foster collaboration among team members. For example, residents need to learn how to run a patient service while at the same time attending to the educational needs of the learners.
Faculty development topics related to leadership issues include:
Management
Managing people has never been as complicated as it is in today’s health care environment.
Faculty development topics related to management might include:
Managing learners and educational programs today also requires an astute awareness of legal boundaries and issues, institutional student evaluation policies, and a clear appreciation of due process related to failing students.
Teachers interested in educational research need access to instruction on social science research. Depending on the individual’s background experiences, instruction on research design, statistics, critical evaluation of the literature, accessing literature in education, and how and where to disseminate scholarly work related to education may be needed. Because these knowledge domains often fall outside the clerkship director’s area of expertise, the clerkship director can direct the interested individual to available resources and programs locally and nationally. However, the clerkship director should be apprised of any anticipated research occurring on the clerkship to ensure compatibility with the clerkship goals and relevance to student learning. The clerkship director also may have to play the role of “gatekeeper” if too many individuals are interested in students as research subjects at one time.
Topics that might be included in a faculty development program on educational research might include:
Topics associated with this role include how to:
Full-time faculty seeking excellence as teachers, educators, researchers, communicators, and administrators must translate their activities into evidence consistent with academic promotion. To achieve academic promotion, these individuals need to develop several additional domains of expertise, including defining and revising a career plan with key benchmarks, understanding the written and unwritten values and policies of academia (e.g., academic promotion, committee service) and developing and maintaining a local and national network of professional colleagues. As with the other role types in forming the faculty development plan, the clerkship director should be aware of local and national resources that can be tapped to meet the needs of these individuals.
After determining the teachers’ learning needs by examining their roles, clerkship directors can begin developing a faculty development plan and agenda. There is certainly no shortage of fodder for faculty development initiatives, and so the trickiest step in setting the agenda is this last step: Determining what is most appropriate based on your local context, highest priority needs, and resources available to help frame the faculty development plan.
First, determine priorities by examining the assessment and evaluation data regarding your clerkship objectives outlined in Step 1. Second, determine who has the roles related to those identified priorities and the associated faculty development topics in Step 2. Third, select those topics that are most common across groups as a set of topics with which to start.
For example, if evidence from Step 1 indicates that the quality of learner assessment varies by rotation site, the clerkship director will want to focus on the topic of learner assessment.
Table 1 reflects a listing of major topic areas by faculty role, with the assessment category expanded to draw on the above example. (return to top)
Once the clerkship director has identified the focus for faculty development, s/he must decide on the most effective methods and strategies for delivering it. As with the design of any instructional activity, the learner (in this case the clerkship faculty and residents), the environment, and the topic must be finalized.
Many faculty members enter the world of academic medicine with an avid interest in teaching. However, as their clinical, research and administrative responsibilities increase, their commitment to teaching (and learning how to be better educators) often wanes. How, then, can the clerkship director successfully provide faculty development to colleagues who may feel they don’t have the time and incentives to devote to teaching, to say nothing of faculty development?
Just as motivating your medical students and residents to learn is a critical component of effective clinical teaching, motivating your colleagues to excel as teachers is an essential component of faculty development. Ideally, the clerkship director can connect to faculty members’ desire to be excellent teachers and identify a gap or opportunity to enhance their skills.
Highlighting Incongruities
Use clinical teaching evaluation ratings or comparative clerkship ratings to point out opportunities as individuals and/or the department to enhance teaching and improve programs.
Addressing Obstacles
Clinical educators increasingly perceive that time to teach is in conflict with expectations for clinical productivity. Teaching time is further stressed by resident duty hour restrictions, and/or a lack of understanding about specific ACGME competencies or other objectives. Identifying such obstacles and then selecting a faculty development strategy to address the obstacle explicitly can help you get started. Providing faculty with compliance strategies (e.g., how students can chart notes consistent with regulatory guidelines, or presenting easy-to-remember “mnemonics for systems-based practice competence” is a critical element in selecting the right faculty development strategy to help faculty improve teaching efficiency and effectiveness.
Providing Incentives
Clerkship directors must actively work with their department chairs and institutional leadership to provide meaningful incentives to faculty who excel as teachers. It may be as easy as increasing the public recognition of the work of teachers or as challenging as providing a relative value units system to provide incentives for educational work.
Once the appropriate motivational trigger(s) for your faculty have been identified, then the clerkship directors can choose from a large variety of programs to present information and develop or hone skills specific to the clerkship and faculty needs. These offerings vary in cost and time commitment and can be used in various combinations to provide a broad exposure of topics and strategies. Table 2, building on a categorization developed by Ullian,20 lists faculty development strategies and examples. The examples are illustrative and not meant to be exhaustive. The examples were found by searching the literature, the Web, and the authors’ knowledge. A brief description of each category and sample resources follows. (return to top)
Educators, administrators, researchers, and professional academics often review the literature in their specialty to keep up-to-date. Keeping up-to-date in education is frequently complicated by the relative naiveté of faculty regarding the appropriate journals to read and how to evaluate the articles. Fortunately, the increase in electronically available journals makes disseminating key articles relatively easy.
Disseminate Key Education Articles to Faculty
As clerkship director, you can review the contents of journals (electronic or paper versions) such as Academic Medicine, Medical Education, and Teaching and Learning in Medicine as well as specialty journals that frequently print articles pertaining to medical education (e.g., Journal of General Internal Medicine, Ambulatory Pediatrics, American Journal of Surgery, and the Journal of the American Medical Association). Review of relevant journal increases your knowledge and allows you to send important articles to all/many of your faculty. Several costs should be considered, including the subscription costs and duplication costs for dissemination of paper copies. Some of these costs may be limited if the institutional library offers electronic access to these journals. Regardless of how you access them, make sure you appropriately address copyright issues.
Start a Medical Education Journal Club
Education journal articles can also be used to establish a Medical Education Journal Club. Several examples, including journal clubs that focus on evidence-based medical education, are described in the literature.43,44 Suggestions for how to succeed with an evidence-based journal club can be found in Phillips and Glasziou.45
Review Published Proceedings and Reviews of Education Articles Outside Your Specialty
The Alliance on Clinical Education (ACE), an organization of representatives from the seven clerkship director organizations, fosters across-discipline, education-related collaboration. This guidebook was developed by members of each of the ACE constituent organizations. ACE has begun an initiative with Teaching and Learning in Medicine (TLM) to publish proceedings from annual meetings of ACE specialty societies. TLM is also now publishing the ACE Journal Watch, a review of educational publications in internal medicine journals. These can serve as a helpful repository of pertinent articles in medical education regardless of discipline.
Creating a newsletter will allow you to summarize some of the articles and evidence you read in the literature and disseminate it to your faculty. This may increase the likelihood that the faculty will become aware of pertinent information because it decreases the time they would need to spend to “read” the literature. An “evidence-based journal club” approach has been adapted for use in a newsletter, The Pediatric Educator, created by COMSEP (the pediatric clerkship directors’ national group). You can further adapt this method for your own use, asking others in your department to scan one or two specific journals, summarize pertinent articles, and write a review for the newsletter.
Newsletters can also highlight educational activities that are occurring or are planned for your department. Finally, a newsletter is a great way to recognize accomplishments and activities of those you teach. Recognition of this kind is well received and can promote the importance of maintaining educational skills. On-line newsletters will minimize the cost, but paper versions are often relatively inexpensive. Each version requires the time you and your colleagues spend on creating the publication. (return to top)
Highlights the Importance of Education
Creating a regular (annual/bi-annual) Grand Rounds focused on education and teaching highlights the importance of education. Invited experts from outside your department or institution often draw a larger audience than the experts in your own department, but add to the cost. Using internal experts allows others in your department to learn of innovations and expertise that are readily available and may emphasize the value of educational scholarship in your school.
Opportunity to Model Teaching Strategies
Interactive strategies may be used to demonstrate the very skills you want your faculty to use in their teaching.
Opportunity to Recognize Outstanding Educators
You may want to recognize full-time and volunteer faculty who have contributed to your educational program during Grand Rounds, enhancing their peer recognition.
Measuring the impact often stops at satisfaction with a grand rounds presentation. At least one example of how to measure changes following a clinical Grand Rounds is documented.46 This study could be adapted for use following an appropriate educational topic, measuring group changes.
Seminars, defined as brief, often didactic sessions, are a frequently used as a faculty development strategy. One-hour sessions can often be incorporated in a regularly scheduled series of presentations for your department. Frequently presented topics include providing feedback, principles of adult learning, clinical teaching skills, and evaluation skills. Role modeling, even in a 60- to 90-minute program, allows participants to observe and practice specific skills. Using standardized students and standardized teachers enhances your ability to demonstrate specific skills in a more controlled manner.
Seminars are often easy to provide and require few resources. One of the key issues to consider is the environment – having a room that is the appropriate size and with appropriate audio/visual capabilities. It is crucial to make the first few seminars outstanding. This will enhance the reputation of the program and encourage future attendance. Time limits for most seminars (60 minutes) often affect what can be adequately presented. Variable attendance, especially if a series is used, makes it difficult to present in a fluid manner. Finally, impact may be difficult to measure. However, you may be able to track the effect on teacher evaluations (if those data are available to you) based on frequency of attendance.
Workshops are frequently presented as ½-day programs, but may be as short as 2 hours. The goal of workshops should include the opportunity for the participants to practice a skill or activity. Workshops should model the principles of adult learning! Optimally, participants receive feedback on their performance. Because of the longer duration, you may have time to collect pre-intervention data, which will enhance your ability to measure the impact of your program.
The major problem with workshops is also their strength – the longer time commitment by faculty. Given the demands for increased clinical productivity, many clinician-educators find it difficult to attend a local workshop. Workshops on campus are often interrupted because some participants must leave during the presentation to address other demands in their office, clinic or hospital. Interruptions can be diminished by conducting the workshop off-campus or by asking faculty not to bring beepers into the room. If beepers are not allowed, someone must attend to the beeps or beepers must be given to a colleague who is not participating in the session.
Comprehensive programs may be developed within your department, your school/university or through national programs and fellowships. Well-developed curricula can address important educational principles and skills, building from one session to the next. Each program can be designed for specific groups (e.g., junior faculty, senior faculty, subspecialty educators, or administrators). Impact of these programs can be measured over time and across several competency areas. Excellent examples of such programs are listed in the references found in Table 2. Do not hesitate to call the creators of these programs, as they are often willing to provide more input about the design and problems they encountered than have been delineated in the publication. Remember, they are educators and mentors!
Departmentally based programs allow the directors to create programs based on local needs assessment and can be changed relatively easily as needs change (e.g., Internal or external mandate for competency-based teaching and assessment). These programs also allow for enhanced networking among members of multiple divisions/sections and may result in new programs that minimize resource use as collaborators from several divisions create new initiatives.
University-wide programs can also be designed with local needs in mind. Including participants from other departments and from other professional schools (e.g. nursing, pharmacy) further enhances networking. This networking may increase the faculty’s awareness of educational issues on campus. Being exposed to others who think differently may stimulate educators to try new strategies. (return to top)
External faculty development programs are becoming increasingly popular, especially as strategies evolve to maximize resource use and expand the impact through national dissemination. The Association of American Medical Colleges (AAMC) provides both national and regional meetings focused on medical education. Educational tracks (lectures and workshops) are also offered during national specialty meetings, such as the Society for Teachers of Family Medicine, Society for General Internal Medicine, the American College of Surgeons, and the Pediatric Academic Societies. Longitudinal programs have also been developed to provide more in-depth training. “Train the trainer models” further expand the potential impact of the program when graduates return to their own setting and provide programs locally, regionally or nationally. Tracking dissemination of strategies learned at “train the trainer” program is a program outcome that can be measured. An advantage of external programs is the expectation of a time commitment, from the participant and, often, his/her chair. The extended programs often result in a relative decrease in distractions when the faculty member is away from the routine workday. However, the cost of these programs may exceed the faculty member’s and/or department’s resources and may limit the numbers of faculty who participate.
Fellowships and advanced degree training are increasingly available for faculty members who are able to commit more time to pursuing a career as an educator. Primary care fellowships often combine clinical training with an educational track. Degrees in medical education are being offered as full-time obligations, weekend programs, and on-line programs and combination programs (on-line with some face-to-face sessions required). There is little literature that discusses the outcomes of these programs.
Results of the AAMC graduation survey, administered each year to the 16,000 graduating medical students, reveal that residents play a significant role in teaching medical students. The Liaison Committee for Medical Education (LCME), the accrediting body for medical schools, recognizes the importance of residents as teachers and has established Standard ED-24 requiring formal programs to teach residents to teach.
Numerous resources are available to assist in creating such programs.
Books
Books, such as those by Schwenk and Whitman,47 and by Dunnington and DaRosa (www.surgicaleducation.com) have been used to develop a curriculum for residents.
Online
The Internet is a rich source of information and presentations.
The Society of General Internal Medicine and the family medicine residency directors provide programs: (www.sgim.org/Handouts/am04/Workshops/WB08.pdf and www.afprd.org/objective4.php). A bibliography on teaching residents to teach was developed at the University of New Mexico and is available at http://hsc.unm.edu/som/ted/ResidentTeachers/bibliography.htm .
Many programs focused on resident teaching use the same strategies outlined in Table 1. Seminars and workshops are common and focus on skills such as clinical teaching,48,49,50 providing feedback and evaluation,51,52 and teaching communication skills.53 Often these sessions are incorporated into a routinely scheduled educational series or as part of a retreat. This allows for transmitting information but, as with faculty, limits the opportunities for practicing the skills presented. A few residencies have initiated block rotations to address teaching skills. This allows for more explicit instruction, practice time for residents and often provides multiple opportunities for observing the resident in teaching situations. Scheduling these rotations depends on the needs of the program, with some residents participating very late in their residency. (return to top)
Faculty development programs offered for your own department or school can use many of the strategies discussed in the previous section. When developing these programs, you will need to address the issues outlined below.
Identifying the needs of your faculty is crucial to developing sustainable programs. Helpful resources include Bland’s18 delineation of the skills needed for successful academic faculty as well as Skeff’s54 and Irby’s55 descriptions of clinical teaching competencies. After completing a needs assessment, you can tailor your program (and strategies) to focus on key issues in your department. Collaborating with others who need to develop faculty skills (e.g., residency program directors) enables optimal resource use.
Getting buy-in to prioritize these sessions is often the major barrier to developing sustainable programs. Your chairman will often set the tone in the department. Obtaining his/her visible support is crucial to ensuring that the programs will be valued. Even with the chairman’s support, other direct supervisors may make it difficult for faculty to take the time needed to participate in a more longitudinal program. Address these issues with the appropriate division/section chiefs first to avoid developing great programs that no one is able to attend.
Another option is to collaborate with your program director, other clerkship directors, or the office of medical education in your institution. By collaborating with others, you can build on the importance of faculty development across groups and may be able to identify resources that aren’t usually available for clerkship programs.
Administrative tasks for any program include identifying a speaker, distributing notification of the session, scheduling a room (and refreshments if possible), reminders as the date draw nears, tracking probable attendees, making handouts and evaluation forms, and follow-up after the course. You may be able to use available departmental resources for seminars and workshops. As the programs expand and become more longitudinal, you are likely to need staff dedicated to this responsibility. Make sure you know who can assist you in these administrative tasks or you will find yourself doing it all, creating a system that quickly becomes difficult to sustain.
Finding the appropriate presenters for faculty development programs can be a challenge. External experts often lend credibility to a program but may add expenses that exceed the budget. Experts in your own institution can usually be identified, perhaps through the office of medical education or through educators in other departments. As faculty development continues, you can create your own internal experts, asking graduates to assist with presentations and leadership activities.
Evaluating faculty development programs is a key component to demonstrating their value and benefits (See also Chapter 7, Evaluation of the Clerkship: Clinical Teachers and Program). Program design must include plans for evaluating participants’ satisfaction and measurable outcomes such as scholarly productivity, improvement of skills, career advancement, and leadership activities.30,38,56
The cost of internal programs is often under-reported, and few published studies address the issue.57 The cost of a single didactic presentation for faculty development is measured primarily in preparation time (approximately 8 hours for a 1-hour new presentation). However, more extensive programs require support staff and the participants’ time away from revenue-generating activities, in addition to faculty time for preparation and presentation.
Negotiating for sufficient support prior to initiating an in-depth program will help you define the limits of activities you can provide. Options for support may include internal funds from the dean, academies or faculty scholars’ programs and/or alumni endowment for education. You may need to initiate and set up your own account. (return to top)
If you are just getting started, make sure you build on the expertise of the many dedicated faculty developers who work in the field. Build on previous work by adopting or adapting programs that:
Perform a literature search to identify programs that have been disseminated in the peer-reviewed literature. Manuscripts may provide sufficient description of the program, but you can avoid pitfalls by also contacting the author to discuss what worked, what didn’t, and how your setting may require some adaptation of their program.
Excellent resources are also available through the Internet. Recently, a search using “faculty development” and “medical education” as the search strategy resulted in 229,000 hits, including sites that describe programs and provide some resources (e.g., PowerPoint presentations). Many of these lead to offices of medical education or faculty development at various medical schools.
The faculty and staff in other medical schools can be of great assistance, and may even identify people you were not aware of in your own institution. If you use a program readily available on the Web, even if you modify it, be sure to give appropriate attribution for the basis of your program (i.e., explicitly identify the person who developed the original program during the presentation). It is also professionally polite to contact the original author to let him/her know you have adopted or adapted his/her materials.
The AAMC, with guidance from the Group on Educational Affairs (GEA, a professional development group open to all individuals at AAMC-affiliated institutions), has created MedEdPORTAL (Providing Online Resources To Advance Learning), a resource databank to house education materials. MedEdPORTAL includes education materials that have been developed, submitted, peer reviewed, and made available for public dissemination. The plan is for MedEdPORTAL to include resources relevant for teaching and assessing the Accreditation Council on Graduate Medical Education (ACGME) competencies. Materials may be appropriate for students (undergraduate medical education), residents (graduate medical education), faculty (faculty development or continuing medical education). View a conceptual model of the repository at the AAMC’s Web site www.aamc.org/members/gea/cube.htm).
Literature Search
A literature search (through Ovid or other databases) can provide background. If you are not sure what to search for, the Stanford Faculty Development Center Web site (http://sfdc.stanford.edu/) provides access to its literature databases, including one on clinical teaching. Once you identify a potential strategy, make sure you meet with people who have expertise in designing such programs. Make sure that your program will address what you want it to address and incorporate evaluation methods that will allow you to determine the program impact. Evaluation is a crucial step in sustaining faculty development and will help you disseminate your work.
You can network with colleagues at the national level by:
Some schools have recently formed academies or societies for medical education. They are typically composed of individuals who have been recognized for excellence in education and these individuals are a vital resource for both teaching your faculty development initiatives and/or guiding you to appropriate resources.
Contact your medical education office or curriculum dean to see if they have access to resources (e.g., funds, individuals, books, and journals) and/or can assist you in designing, implementing, or evaluating a faculty development effort. A listing of offices that support medical education by schools is available through the Society of Directors of Research in Medical Education at www.sdrme.org.
(return to top)
Clerkship directors are responsible for the development of faculty so they meet their teaching responsibilities to the student education program and clerkship directors and seek to excel as educators. We have described a wide variety of strategies for faculty development that will help you to match your local needs and resources with the goals of your programs.
The most important step is to just get started. Follow Steps 1 – 3, described above. Use data already available (e.g., comparative evaluation of your clerkship and teaching evaluations) and the results of a needs assessment to select the topics by education roles that are priorities for faculty development in your clerkship. Design a faculty development curriculum targeted to improve your educational program. Many resources are available to help you get started and there are opportunities to advance your own scholarship by adding to the knowledge about what works in faculty development.
But, how will you know if your program works? The next section focuses on how to evaluate the success of your faculty development initiative. Evaluation is a critical element in the design of any instructional program and to advance faculty development as a field of scholarly inquiry.
Evaluation focuses on determining the value or worth of a specific program or activity. Evaluation of faculty development programs should: (1) determine the degree to which the individual faculty obtained the knowledge, skills, or attitudes taught in the program and (2) create a management system that promotes program accountability, efficiency, and cost-effectiveness. Outline the objectives for your faculty development program so your program can monitor participants’ progress toward your objectives (e.g., excellence as teachers and educators).
An evaluation plan should be designed to provide formative and summative data to assess the degree to which targeted objectives are achieved. Formative data are used during the program to mold participant performance, enhance the outcomes by addressing barriers, overcome challenges, and build on identified strengths. Summative data are used to judge the overall success and outcomes of the program, highlighting the products and impact the program has on participants and on your institution.
Progress Evaluation monitors the timely implementation of the plan.
Process Evaluation identifies the strengths and weaknesses of day-to-day program operations, thereby guiding program revision to ensure that you are on-task toward meeting program objectives. Feedback from participants can highlight strategies that are working well and those that need revision. Attendance problems can be identified early, allowing you to determine whether changes in scheduling or content are needed or whether there are issues with prioritizing the program in the life of the faculty member or department.
Product and Results Evaluation assesses the degree to which participants have achieved required objectives. Available data from clerkship and residency evaluation, clinical teaching ratings, and learner outcomes (e.g., residency choice) are often used for product and results evaluation.
An implementation timeline facilitates tracking the progress of your program and identifying problems proactively.
Progress Evaluation
Be sure to answer questions like: Are the tasks needed to prepare and present the program organized so as to occur on time? Are participants attending the sessions and completing their assignments?
Process Evaluation
Process data evaluate the effectiveness of the methods to achieve your objectives so that on-going program revisions can be made to keep the program targeted to its objectives. Strategies for monitoring progress and obtaining process evaluation data vary by program, but often contain a basic set of tasks (see Table 3). Using the timeline will keep you on track and identify areas that become barriers for success.
Faculty development programs should be designed to advance the legislated and department-specific priorities (e.g., improve teaching, implement competency-based education and assessment). Product evaluation provides data to assess the degree to which the program achieves its objectives.
Product evaluation should be designed to assess four levels of impact, as defined by Kirkpatrick.58 These are reaction, learning, behaviors, and results.
Evaluation of participant reaction measures the satisfaction with the program, often through surveys. Evaluation of learning may use self-assessment or tests of cognitive skills. These are strengthened by assessing the knowledge, skills, or abilities before and after the program (pre-post assessment). Measurements that document the use of the newly gained knowledge or skills provide the evidence of the next level – changing behaviors. Finally, the results of the program may be measured by quantifying its impact. Examples of results measurements include improved clinical teaching ratings, increased productivity, improved quality of service, or changes in recruitment and promotion rates.
The data used to determine success in achieving these objectives may include:
The use of existing data demonstrates progress towards the development of an efficient, cost-effective program evaluation system. To assure that you are evaluating the key components of your faculty development program, you may want to create a table, similar to the example shown in Table 4, that matches the evaluation data source you will use for each objective and key performance indicators. This will vary for each project. (return to top)
There is a need for faculty development in all levels of higher and professional education. A clerkship director’s responsibilities include equipping faculty and residents with the knowledge and skills necessary to help them succeed in their various roles in the student education program. The clerkship director is not expected to directly deliver all faculty development programs, but he or she should help lead the development of faculty development initiatives needed to meet faculty and resident learning needs relevant to student education.
This chapter was written to help guide those efforts by addressing who, why, what, how, and so what questions that must be answered before rolling out faculty development activities. Following through will require an investment of time, resources, and energy, but the benefits are that you may help faculty discover, or some cases, rediscover the joy of teaching, learning, and the world of education.
Faculty Development Topics |
Educator |
Administrator |
Researcher |
Communicator |
Professional Academic |
Curriculum |
X |
X |
X |
X |
X |
Teaching Methods |
X |
X |
|
|
|
Learner Assessment |
X |
X |
X |
X |
|
Sampling of performance |
X |
X |
X |
X |
|
Frame of reference training |
X |
X |
|
X |
|
Performance benchmarks |
X |
X |
|
X |
|
Standard setting |
|
X |
|
|
|
Program Evaluation |
|
X |
|
|
|
Leadership |
X |
X |
X |
X |
X |
Career Planning |
|
X |
|
|
X |
*Hint: Click Back button on browser to return to previous spot in text
OR (return to top)
Publications & Resources |
Examples |
Journal Articles & Medical Education Books |
Journals: Academic Medicine; Medical Teacher; Clinical Teacher; Medical Education, Teaching and Learning in Medicine; Journal of General Internal Medicine; Family Medicine; Ambulatory Pediatrics; Academic Psychiatry; Focus on Surgical Education |
Newsletters and electronic “tip” sites |
The Pediatric Educator (www.comsep.org) |
Educational Repositories Invited/peer reviewed submissions to educational resource repositories. |
EPERC www.eperc.mcw.edu ; |
Continuing Medical Education |
Examples |
Grand Rounds |
Consider working with other clerkship directors to co-sponsor around topics of shared interest. |
Seminars & Webinars |
Local Seminars: Webinars: www.iamse.org |
Workshops (There are numerous examples found in search engines such as Ovid) |
Examples include: Clinical Teaching 24 |
Comprehensive Faculty Development Programs |
Examples |
Departmental programs |
Clinical Teaching:27 Peer support28 |
Organization-wide programs |
Administrative Colloquium 29; |
National programs |
Faculty Development Scholars Program33; Undergraduate Medical Education in the 21st Century34; |
Certificates, Fellowships |
Certificates: AAMC/GEA Medical Education Research Certificate (MERC) www.aamc.org/members/gea/rimebrochure.htm Specialty Specific Fellowships: SERF (Surgical Education Research Fellowship); APGO/Solvay Educational Scholars Program |
Individual Activities |
Examples |
Self Assessment and Reflection |
Reflection:36,37 |
Direct observation |
Peer coaching 40,41 ; Videoanalysis:42 |
Degree programs |
Online Master’s Program for Healthcare Professionals (Cincinnati); |
*Hint: Click Back button on browser to return to previous spot in text
OR (return to top)
Program Activities for Clinical Teaching Workshops |
Time Period |
|||
1 |
2 |
3 |
4 |
|
Planning Meeting with core team (eg, other faculty, administrative support) |
X |
X |
X |
X |
Reserve room and parking + order food |
X |
|
|
|
Define workshop objectives |
X |
|
|
|
Develop invitation list/contact list |
X |
|
|
|
Create documents (eg, contact info, schedule) |
X |
X |
|
|
Finalize schedule dates, topics, speakers |
X |
X |
X |
X |
Obtain CME credit |
|
X |
|
|
Send out invitations/announcements |
|
X |
|
|
Obtain pre-program assessments |
|
X |
|
|
Implement workshop |
|
|
X |
|
Obtain program session evaluation |
|
|
|
X |
Obtain evaluation data from external sources |
|
|
|
X |
Collect, analyze, and report product/outcomes data |
|
|
|
X |
Meet with chair/supervisor to report on results, Review faculty development plan and realign |
X |
|
|
X |
*Hint: Click Back button on browser to return to previous spot in text
OR (return to top)
Objectives: Clinical Teaching |
Learner Teaching Evaluations |
Evaluation Comments about Preceptors |
Downloads of EBM resources from library |
Preceptor Completed Rating Forms |
Increase use of EBM in teaching interactions |
X |
|
X |
X |
Use strategies to increase efficiency as teachers |
X |
|
X |
X |
Accurately rate learners using required clerkship form and submit |
X |
X |
|
X |
Improve overall effectiveness as a clinical teacher |
X |
X |
X |
X |
*Hint: Click Back button on browser to return to previous spot in text
OR (return to top
References
- Zigler Z. Top performance: How to develop excellence in yourself and others. New York: Berkley, 1987.
- Katz J and Henry M. Turning professors into teachers. American Council on Education. Phoenix: Oryx, 1993.
- www.aamc.org
- www.acgme.org
- McKeachie WJ. McKeachie’s teaching tips (11th Ed). Boston: Houghton Mifflin Co., 2002.
- Pololi LH, Knight SM, Dennis K, Frankel, RM. Helping medical school faculty realize their dreams: an innovative, collaborative mentoring program. Acad Med. 2002;77:377-84.
- Elliott DL, Skeff KM, Stratos GA. How do you get to the improvement of teaching? A longitudinal faculty development program for medical educators. Teach Learn Med. 1999;11:52-7.
- Hewson MG, Copeland HL. Outcomes assessment of a faculty development program in medicine and pediatrics. Acad Med. 2001;76 (suppl 10):S68-S76.
- Hewson MG, Copeland HL, Fishleder AJ. What’s the use of faculty development? Program evaluation using retrospective self-assessments and independent performance ratings. Teach Learn Med. 2001;12:153-60.
- Skeff KM, Campbell M, Stratos G, Jones HW III, Cooke M. Assessment of attending physicians of a seminar method to improve clinical teaching. J Med Educ. 1984;59:944-50.
- Skeff KM, Stratos G, Bergen MR. Evaluation of a medical faculty development program: A comparison of traditional pre-post and retrospective pre-post self assessment ratings. Eval Health Prof. 1992;15:350-366.
- Litzelman DK, Stratos G, Skeff KM. The effect of clinical teaching retreat on residents’ teaching skills. Acad Med. 1994;5:433-434.
- Cole KA, Barker LR, Kolodner K, Williamson P, Wright SM, and Kern DE. Faculty development in teaching skills: An intensive longitudinal model. Acad Med. 2004;79:469-480.
- Centra J. Reflective faculty evaluation. San Francisco: Jossey-Bass, , 1993.
- Boyer EL. Scholarship reconsidered: Priorities of the professoriate. Princeton: Princeton University Press, 1990.
- Pololi LH, Dennis K, Winn GM, Mitchell J. A needs assessment of medical school faculty: caring for the caretakers. J Contin Educ HealthProf. Winter 2003;23(1):21-29.
- Association of American Medical Colleges. Report 1: Learning Objectives for Medical Education Students: Guidelines for Medical Schools. 1998.
- Bland CJ Schmitz CC Stritter FT Henry RC Aluise JJ. Successful Faculty in Academic Medicine: Essential Skills and How to Acquire Them. New York: Springer Publication, 1990.
- Pangaro L. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med.1999;74(11):1203-1207.
- Ullian JA. Types of faculty development programs. Fam Med 1997; 29:237-241.
- Salerno SM O’Malley PG, Pangaro LN, Wheeler GA, Moores LK, Jackson JL. Faculty development seminars based on the one-minute preceptor improve feedback in the ambulatory setting. J Gen Intern Med. 2002:17(10):779-87.
- Lang F, Everett K, McGowen R, Bennard B. Faculty development in communication skills instruction: insights from a longitudinal program with "real-time feedback". Acad Med. 2000;75(12):1222-8.
- Hemmer PA Pangaro L. Using formal evaluation sessions for case-based faculty development during clinical clerkships. Acad Med. 2000; 75(12):1216-21.
- Steinert Y, Lawn N, Handfield-Jones R, Nasmith L, Lussier D, Levitt C. Orientation for new teachers. Workshop on clinical teaching skills. Can Fam Phys.1995; 41:79-85.
- Gelula MH, Yudkowsky R. Microteaching and standardized students support faculty development for clinical teaching. Acad Med. 2002;77(9):941.
- Mandel LP, Stenchever MA, Vontver LA. A model clinical faculty workshop program for a multisite clerkship. Obstet Gynecol. 1997;90(6):1011-4.
- Lye PS, Simpson DE, Wendelberger KJ, Bragg DS. Clinical teaching rounds. A case-oriented faculty development program. Arch Pediatr Adolesc Med. 1998;152(3):293-5.
- Edwards K. "Short stops": peer support of scholarly activity. Acad Med. 2002;77(9):939.
- McCurdy FA., Beck G, Maroon A, Gomes H, Lane PH. The administrative colloquium: developing management and leadership skills for faculty. Ambul Pediatr. 2004;4(1 Suppl):124-8.
- Gruppen LD, Frohna AZ, Anderson RM, Lowe KD. Faculty development for educational leadership and scholarship. Acad Med. 2003;78(2):137-41.3
- Pololi LH, Knight SM, Dennis K, Frankel RM. Helping medical school faculty realize their dreams: an innovative collaborative mentoring program. Acad Med. 2002;77(5):377-84.
- Hewson MG. A theory-based faculty development program for clinician-educators. Acad Med. 2002;75(5):498-501.
- Osborn LM, Roberts KB, Greenberg L, DeWitt T, Devries JM, Wilson M, Simpson DE. The APA/HRSA Faculty Development Scholars Program: introduction to the supplement. Ambul Pediatr. 2004;4(1 Suppl):83-7.
- Peters AS, Ladden MD, Kotch JB, Fletcher RH. Evaluation of a faculty development program in managing care. Acad Med. 2002; 77(11):1121-7.
- Skeff KM, Stratos GA, Berman J, Bergen MR. Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med. 1992;152(6):1156-61.
- Pinsky LE, Irby DM. "If at first you don't succeed": using failure to improve teaching. Acad Med. 1997;72(11):973-6.
- Branch WT Jr, Paranjape A. Feedback and reflection: teaching methods for clinical settings. Acad Med. 2002;77(12 Pt 1):1185-8.
- Lye P, Heidenreich C, Wang-Cheng Bragg D, Simpson D, and the Advanced Faculty Development Group. Experienced clinical educators improve their clinical teaching effectiveness. Ambul Pediatr. 2003;3(2):93-7.
- Beckman TJ, Lee MC, Mandrekar JN. A comparison of clinical teaching evaluations by resident and peer physicians. Med Teach. 2004;26(4):321-5.
- Flynn SP, Bedinghaus J, Snyder C, Hekelman F. Peer coaching in clinical teaching: a case report. Fam Med. 1994;26(9):569-70.
- Orlander JD, Gupta M, Fincke BG, Manning ME, Hershamn W. Co-teaching: a faculty development strategy. Med Educ. 2000;34(4):257-65.
- Bazuin CH, Yonke AM. Improvement of teaching skills in a clinical setting. J Med Educ. 1978;53(5):377-82.
- Centeno AM, Blanco A, Arce M. Journal club devoted to educational issues. Acad Med. 1999;74(5):464.
- Simpson D, Flynn C, Wendelberger K. An evidence-based education journal club. Acad Med. 1997;72(5):464.
- Phillips RS, Glasziou P. What makes evidence-based journal clubs succeed? ACP Journal Club. 2004;140(3):A11-2.
- Neumayer L, Wako E, Fergestaad J, Dayton M. Impact of journal articles and grand rounds on practice: CT scanning in appendicitis. J Gastrointest Surg. 2002;6(3):338-41.
- Schwenk, T.L., Whitman, L.. Residents as Teachers: A Guide to Educational Practice, 2nd Ed. Salt Lake City, UT: Department of Family and Preventive Medicine, University of Utah School of Medicine, 1993.
- Morrison EH, Rucker L, Boker JR, Gabbert CC, Hubbell FA, Hitchcock MA, Prislin MD. The effect of a 13-hour curriculum to improve residents' teaching skills: a randomized trial. Ann Intern Med. 2004;141(4):257-63.
- Zabar S, Hanley K, Stevens DL, Kalet A, Schwartz MD, Pearlman E, Brenner J, Kachur EK, Lipkin M. Measuring the competence of residents as teachers. J Gen Intern Med. 2004;19(5 Pt 2):530-3.
- Dunnington GL, DaRosa D. A prospective randomized trial of a residents-as-teachers training program. Acad Med. 1998;73(6):696-700.
- White CB, Bassali RW, Heery LB. Teaching residents to teach. An instructional program for training pediatric residents to precept third-year medical students in the ambulatory clinic. Arch Pediatr Adolesc Med. 1997;151(7):730-5.
- Furney SL, Orsini AN, Orsetti KE, Stern DT, Gruppen LD, Irby DM. Teaching the one-minute preceptor. A randomized controlled trial. J Gen Intern Med. 2001;16(9):620-4.
- Smith RC, Marshall-Dorsey AA, Osborn GG, Shebroe V, Lyles JS, Stoffelmayr BE, Van Egeren LF, Mettler J, Maduschke K, Stanley JM, Gardiner JC. Evidence-based guidelines for teaching patient-centered interviewing. Patient Education & Counseling. 2000;39(1):27-36.
- Skeff KM. Clinical teaching improvement: past and future for faculty development. Fam Med. 1997;29:252-257.
- Irby DM. Clinical teaching and the clinical teacher. J Med Educ. 1986;61(9 Pt 2):35-45.
- Morzinski JA, Simpson DE. Outcomes of a comprehensive faculty development program for local full-time faculty. Fam Med. 2003;35(6):434-9.
- Simpson DE, Bragg D, Biernat K, Treat R. Outcomes results from the evaluation of the APA/HRSA Faculty Scholars Program. Ambulatory Pediatr. 2004;4(1 Suppl):103-12.
- Kirkpatrick DL. Evaluating training programs: the four levels. San Francisco: Berrett-Koehler, 1994.
<Chapter 7: Evaluation of the Clerkship: Clinical Teachers and Program