Robert Karp, MD
Avrahom Gurwitz, MPH
Department of Pediatrics
SUNY-Downstate Medical Center
Brooklyn, NY
Darwin Deen
Department of Family Practice
Albert Einstein College of Medicine
Pretest
Q1. Pick the best answer: A part of the evaluation essential for providing corrective action for the learner occurs when:
A. Establishing the structure of an evaluation of teaching program
B. Examining successes and failures while the teaching is going on.
C. When giving a summation
D. Correction for the learner is equally important at all levels
Q2. Least likely answer. The least likely way to provide an interactive teaching program likely to engage the learner in learning about diet and diabetes in pregnancy.
A. A fact-laded PowerPoint presentation with absolutely AMAZING GRAPHICS enabling you to show 6 perinatal complications of diabetes mellitus with treatments for each.
B. A one-on-one discussion of a pregnant teenager with diabetes after you have had a chance to see the patient for a history and physical examination. The pediatrician spent 5 minutes during the interview observing.
C. A chalkboard presentation of a case study of a child with diabetes. There are 3 residents, two students an experienced pediatrician and a dietician who engage in a discussion with an objective of teaching preventive strategies in diabetes of pregnancy
D. A presentation to a larger group, using a point focuses PowerPoint presentation on diet strategies where there are breakout groups of 4 or 5, each with an experienced resident that engage in one of several questions. At conclusion the groups give short summaries.
Objectives:
On completion of Part V, residents will be able to:
1. Put elements in place for effective evaluation of nutrition (or other) teaching programs.
2. Develop interactive teaching and evaluation tools for presentation of a nutrition (or other) topic.
Facilitator preparation:
The American Council on Graduate Medical Education (ACGME) website provides entry-level material suitable for general use:
The Nutrition Curriculum Guide for Training Physicians developed by the Nutrition Academic Award Program Curriculum Committee for the National Institutes of Health can be found at:
In 1984, New York/New Jersey Regional Center for Nutrition Education put together a Conference on Nutrition Teaching in Medical Schools at the New York Academy of Medicine.
See Levy M, et al. Goals delineation for nutrition teaching programs Bull New York Academy of Medicine. 1984;60#6:558-563
Karp RJ. A methodology for nutrition curriculum evaluation Bull New York Academy of Medicine. 1984;60#6:585-590
Levy provides an evaluation grid on p 599 that is modified for use here. A similar though more complex grid is provided by the ACGME.
Silberman M. (1996) Active Learning: 101 ways to teach any subject. Needham Heights, MA. Allyn & Bacon.
Hirsch ED Jr. (2006) The knowledge Deficit: Closing the Shocking Education Gap for American Children New York.
Silberman provides many ways to enhance participatory and experiential learning. Hirsch is among the most eloquent critic of contemporary American education.
Introduction:
It is difficult to develop evaluation tools that get beyond "True/ False" or "Best Choice" knowledge testing. Do we really want recitation of soon to be forgotten facts from our students and residents? "There are few productive, modern or rewarding jobs or careers," writes Rick Morgan, "that can be done without access to outside information on a regular basis." Specific facts learned in school are quickly forgotten. What are retained are the skills "… to write clearly, to analyze objectively, and to think creatively." (New York Times, 5/22/06 pA20) This is what we seek in developing evaluation tools for our Teacher's Guide to Pediatric Nutrition.
This commentary on evaluation contains three sections. The first provides an overview of evaluation theory: What structure to put in place, what processes to use, and finally, how to measure outcomes. These are the necessary components for assessing changes in knowledge, skills and attitudes. The assessment is not only of resident achievement; what we measure is our effectiveness. Simply stated, if they fail; we fail.
In the second section, we apply these theories to the specific needs for feedback and evaluation of residents in clinical practice. We emphasize the integration of teaching and evaluation using an "Active Learning" model as suggested by John Dewy, Jerome Brunner, Mel Silberman, and other contemporary educators.
In the third section we address the importance of stimulating "working" memory, a concept that includes retention of both knowledge and the skills necessary to make use of it. The three sections ties together in that effective evaluation requires proper structure. That structure should be dynamic and not static -- active rather than passive learning. Active learning is an essential element in retention and use of information and skills; moreover, it is experiential. As John Dewey, Jerome Brunner and a host of other early investigators of effective education point out, when students do something that that touches their consciousness, that something "sticks." By contrast, that which is learned without experience is quickly lost. (see Karp, 1993)
Annotated Answers
A1. The answer is B. The three parts of evaluation are A. 1) Creating structures likely to succeed, 2) developing a formative evaluation during which "feed-back" (e.g., corrective action) is given, and 3) a summative evaluation is given. Elements of corrective action are the heart of a formative evaluation. Evaluation programs are heavily burdened by jargon - "educanationalese." Like it or not, learning the jargon is a part of being an effective educator.
A2. The answer is A. You know the advertisement for PowerPoint. It's the one where a fellow makes a fool of himself making hand gestures while presenting data. It's more likely that the learners will remember and use material from that supposed foolishness than the whole lot of "stuff" presented as in A, above. PowerPoint is indeed quite powerful and an excellent tool for providing information to a large group so long as there is focus and imagination in the presentation. Essential points need experiential reinforcement. All the other answers give ways to stimulate learning in an experiential and affective way.
