A continuous quality improvement model for developing innovative instructional strategies
Draugalis, JoLaine R
A Continuous Quality Improvement Model for Developing Innovative Instructional Strategies1
We used the continuous quality improvement (CQI) process to implement changes in a course in pharmacy research methods. The goals were to: (i) increase integration between topics within the course and between it and other courses; (ii) enable students to become self-learners; and (iii) decrease class time devoted to lecture in anticipation of putting the class notes on the Internet. To increase integration between courses, the CQI process was applied to homework assignments by identifying the disease states covered in the pharmacology and medicinal chemistry case discussions. Relevant studies were identified and used as content for problems. For the research design section, topics were organized within a single conceptual framework. To enable students to become self-learners in pharmaceutical care practice, strong skills in reading a research report are needed. We developed a new session at the beginning of the semester on how to read a research report then provided multiple practice opportunities during classes, on assignments, and on exams. To decrease lecture time, short, in-class, small group activities were developed. Evaluation data were primarily student comments and scores on selected exam items. The CQl process provided an explicit approach to introducing innovative methods into the classroom and changed our focus from covering material to assuring that students have learned basic skills and concepts. Our experience indicates that the CQI process should be applicable to a wide range of course material.
INTRODUCTION
Faculty members in higher education must strive to continually improve both course content and instructional methods to enable student learning(1). Specific to pharmaceutical education is the call that how we teach (and students learn) is as important as what we teach(2). Therefore there is a need for pharmacy educators to change from a method based exclusively on lectures to incorporate student-centered and interactive methods. The purpose of this article is to describe how a continuous quality improvement (CQI) framework can be used to introduce innovative instructional methods in the classroom.
GOALS-AND OBJECTIVES
The goal of this innovation was to use the tools of CQI to address learning issues in a course on pharmacy research methods. CQI, as we have adapted it, uses insights about student learning, variation in student performance, and the psychology of learning to improve the learning process(3). The FOCUSPDCA cycle is shown in Figure 1.
The goal of changing the course was to respond to three objectives:
1. Modify the course content to increase integration and reduce the amount of material that appears isolated or with little relationship to other modules in the course and curriculum.
2. Modify the course to enable students to become self-learners. Continuous learning is a requirement for providing effective pharmaceutical care. After students leave the College environment, they will be responsible for their own learning and one of the primary methods of self-learning is reading reports of research. Hence, students must be able to assess the results of research as reported in the literature.
3. Increase class time spent on something other than the transmission of information in anticipation that class notes will become available on the Internet so students will not need to come to class to simply obtain information.
DESCRIPTION OF THE TEACHING INNOVATION
Content. The catalog course description of PhPr 461 Methodology in Pharmacy Research. 3 semester credits. Application of research design, statistical methods, evaluation techniques, and ethical dimensions to critically evaluate published literature, research reports, and proposals. The course is arranged in five parts: (i) introduction; (ii)statistical techniques/data analysis; (iii) aspects of research design; (iv) fundamentals of clinical studies; and (v) survey research and epidemiologic methods. An extensive set of course notes (360 pages) is disseminated the first day of class. The course purpose is to provide the student with the skills and principles of clinical research design and biostatistics needed for evaluation of the medical literature and assessment of research reports and proposals. The course has eight objectives. Each instructional topic has specific learning objectives. The specific course content and mechanics have been described previously(4,5).
Student Audience/Level of Student. Novice; course has no prerequisites.
Point in the Curriculum Where Used. required course in the first semester of the second professional year of four-years in an entry-level Phan-nD program.
The teaching innovation described is the application of a CQI process to on-going course evaluation and revisions in PhPr 461. The first learning process that we wanted to improve was that of integration. For quite some time, we have been integrating the statistical analysis and study design components. However, we wanted to increase students’ ability to integrate the material from various units in the class and to increase their ability to integrate learning from this course with material learned in other courses. The changes in the course were based on the assumption that if the instructors can’t or don’t integrate material in the classroom, students will be very unlikely to integrate it outside the classroom.
Process. Using the FOCUS-PDCA approach we will describe major changes we made in the summer of 1998 and throughout the fall semester to improve the PhPr 461 experience. The first example is portrayed in the FOCUS-PDCA framework and others will follow using a narrative form.
Find process to improve – homework assignments.
Organize team that knows the process – two course instructors and course teaching assistant (first semester graduate student who took the course as a Pharm.D. student)
Clarify current knowledge of the learning process
P – Modify approach, content, and construction of homework assignments_
D – Gather data from 1997 course evaluation form and item analysis of 1997 homework sets.
C – Use data to improve process. Students use homework assignments to prepare for exams as specific learning objectives guide the development of both.
A – use of contemporary literature and actual abstracts/articles are fruitful endeavors – yet improvements are possible.
Understand causes of learning process variation
P – students find it challenging to deal with all the new concepts presented within the class; students find it difficult to integrate isolated information presented in the class, given all the other content-specific information in other courses; students working alone do not reap as much benefit from the homework assignments as those students working in study groups.
D – find out what disease states are being covered in Medicinal Chemistry and Pharmacology and accompanying Case Discussion via course syllabi and discussion with course coordinators
C – Students appreciate real-world practice examples and relate to inquiries from practicing pharmacists.
A – Construct assignments to align with other course content and when possible interject contemporary examples.
Select the learning process improvement
Continually reinforce important concepts in class as well as on assignments. Content congruence with concurrent coursework – seek articles and abstracts that match those topics. For example, Assignments 1, 2, and 5 were directed at GERD, angina, and asthma that were constructed to match content from other courses. As well, Assignment I was based on one abstract, Assignment 2 on two abstracts, and by the final assignment, students were required to read an abstract and a complete article in order to complete the assignment. Encourage formation of study groups to enhance learning opportunities. Relevance in terms of social and professional concems-subscribe to Journal Watch and scan for useful literature citations/abstracts. Incorporate items based on topics such as sexually transmitted diseases, consequences of binge drinking, and safer sex. Use up-to-date applications via contemporary literature as well as real-world inquiries from practicing pharmacists. See Appendix A for example homework item based on a 1996 inquiry from a pharmacist at the Prescott VAMC.
NARRATIVE EXAMPLES
Three units in the research design section were identified for major modification: (i) randomization processes, (ii) effects of subject withdrawals, and (iii) methods of prevention, and compliance. Previously, each was a stand-alone topic treated in a single lecture that followed the section of types of research designs that are taught using the framework of Campbell and Stanley(6). Salient features of several research designs are presented with example research reports. Students learn to differentiate the types of designs and to identify the primary threat to internal validity associated with each design based on previously presented threats to internal and external validity. For example, they learn that an experimental research design is characterized by use of a random assignment process, and prospective, parallel comparison groups. The primary threat to internal validity is experimental mortality (subject withdrawal) but compliance and randomization are not presented as part of the cognitive map (a cognitive map is a procedural tool for guiding analysis) for assessing internal validity. Hence the topics are isolated from each other and from previous classes on research design and statistical analysis.
The strategy we chose for improvement was to extend the threats of internal validity inherent to the design as identified by Campbell and Stanley(6) to include threats resulting from the operational aspects of conducting research as identified by Cook and Campbell(7). An introductory section was developed to define the two types of threats to internal validity and to review the research designs and the associated threats listed by Campbell and Stanley.
The approach is used again in the following semester when students are given example research reports to assess in the Drug Information course. Students have three 75-minute class sessions in which they analyze research reports. The initial analysis is the same analysis that was used with example articles in the Methods course. However, as we analyze the articles, we expand the discussion to include operational threats to internal validity. Students are provided with a chart (see Appendix B) to guide their analysis.
The second learning process that we wanted to improve was students’ self-learning ability. Our understanding of the selflearning process of practitioners indicates that practitioners must be able to read and assess reports of research to continue to learn after leaving the classroom. This is especially crucial in light of the pharmaceutical care practice philosophy To enable students to be change agents and implement innovative practices, they need to be able to use the literature, One only needs to consider the APhA practice principles of data collection, information evaluation, formulating a plan, implementing the plan, and monitoring and modifying the plan/assuring positive outcomes, to see this obvious connection(8).
We decided that one strategy for increasing self-learning abilities was to enable students to read research reports at the beginning of the class then provide them with practice throughout the Methods course through a variety of means that is reinforced the following semester in the Drug Information course.
The change also was to respond to observations during an in-class exercise. Students were asked to read a table of results from a report and respond to questions about the p-values presented. Listening in on the students’ discussions revealed that students did not know how to read a table. They did not recognize that the table was structured to provide information on the independent and dependent variables.
In addition, the modification responds to a realization that self-learning is facilitated if students have mastered basic skills. Self-learning from research reports will be greatly hindered if students have not learned to identify the independent and dependent variables in a study. Observations over several years have convinced us that this is a difficult skill to learn despite its basic nature and despite the fact that it was being addressed frequently throughout the course. Differentiating between types of variables is based on concept learning and concepts are learned with repeated practice with examples(9). Therefore, a unit on reading a research report was developed and put into the second class session so that students could begin practice with these concepts early.
The basic skill of reading a research report requires that students be able to identify the components of the report (i.e., abstract, introduction, methods, results, discussion, and conclusions) and that they understand the logic behind the information presented. Therefore a unit was developed to teach students the components of a research report and how to follow the logic of a report. Students begin by identifying the section of a report in which excerpts from a research report appear. Then they are provided with a cognitive map (skeleton) of a research report and we work through an example report in class.
The independent and dependent variables are identified and tracked through the report. Students are shown how to identify the variables in the purpose statement, how the independent variable (treatment) and the dependent variable are described in the methods section, and how the measures on the dependent variable are reported according to level of treatment in the results section. Specifically, we look at tables and discuss how they are set up to show both the independent and dependent variables and the results of statistical testing. Students are sometimes amazed that each section of the report is related!
Skills related to reading and interpreting a research report are reinforced throughout the semester in class materials, assignments, and on examinations. For example, items related to reading tables appear on two exams. At the end of the research design section we again review the logical process for evaluating statistical conclusion validity, internal validity, and external validity.
Development of in-class activities was the strategy chosen to make class time more than an opportunity to transmit information. The purpose of the course is for students to learn the basic concepts of statistics and research design so that they can understand and evaluate research literature. Again, concepts are learned through examples and application(9). Therefore, the most effective instructional strategy will provide students with an opportunity to differentiate examples and apply concepts. In-class activities provide that opportunity.
Short in-class activities have been developed for the first two classes and for several of the research design classes. The first class provides information on the scientific method, the role of statistics, and inference of cause and effect. The first in-class activity asks students to “write an example hypothesis and identify the independent and dependent variables.” Students are allowed 5 to 10 minutes for the task and then it is discussed in class. Example hypotheses with their independent and dependent variables are written on the board and questions answered.
The in-class activity during the second class is designed to help students learn to identify the sections of a research report. Statements from a report are excerpted and students are asked to identify the section of the research report where the statement would be located. After students complete the activity, their responses are discussed as a class.
In-class activities in the research design section include a fill-in the blank vocabulary review and study questions for quasiexperimental designs. Two activities are provided for the unit on randomization. During most in-class activities students are encouraged to work in pairs or small groups of three or four and discuss the activity At the end of the task, a raised hand is used to bring the group together again to discuss responses on the questions.
In-class activities provide numerous teachable moments. For example, in the exercise asking students to state a hypothesis with its independent and dependent variable, one group of students hypothesized that tight shoes cause pain. Their independent variable was shoe fit (good versus tight) which led to a discussion of how the investigator would operationalize the concept of “tight shoes.” One suggestion was to provide subjects with shoes two sizes too small.
EVIDENCE OF STUDENT LEARNING
We assess application and other higher levels of learning by constructing items using examples that require students to apply concept skills rather than simply recognize or recall information. Additional specifics such as reliability assessment and item analysis techniques are provided in previous articles(4,5).
Students appreciate the homework assignments, both because they constitute 25 percent of the course grade and because they are useful study guides for examinations. We encourage students to indicate why some choices are wrong which can be quite illuminating for instructors and students alike.
The in-class exercises provide some of the most useful and current information on learning. Particularly informative are small group or pair discussions of how to do an exercise or the meaning of the question. For instance, while students were completing a sorting task, two students were having a discussion on whether a research report was written in present or past tense. The instructor can intervene and clarify the issue with the individual students then discuss the topic again with the entire class.
Both instructors are frequently approached by third and fourth year students to discuss using cognitive maps and skills developed in the course when the students need to evaluate literature reports in other courses and on their clerkship rotations. In our curriculum students are required to carry out an investigative senior project. Application of concepts and skills acquired in the PhPr 461 course is evidenced in students’ proposals, conduct of study, and final presentations, We believe the CQI-generated innovative strategies will engender even increased application and use of course skills and materials.
EVALUATIVE DATA
Two previous papers provide evidence of peer-review of previous course planning: delivery(4), and revisions(5). One of the methods we use to evaluate the course is to have a discussion in class about specific sections of the course. Students are asked for comments and are also asked specific questions. After adding the inclass exercises this past year, students were asked about that aspect of the course. In general, the students seemed to like the exercises and several students were vocal about preferring inclass exercises to lectures.
At the conclusion of the fall 1998 semester, students consistently rated both instructors as being highly effective and both are thought to clearly communicate and explain concepts and ideas by providing examples and relevant applications as rated by students on instructor evaluation forms. At the end of the semester students were positive in the evaluation of handouts/supplementary course materials, the utility of assignments, and course organization (all items averaged 4.3 or higher on a five-point scale).
The more telling story comes from students after completing the course – both in their remaining time in the school as well as from alumni. Some examples were provided in the previous section about how evidence of student learning is manifested later in the curriculum. The more senior students have been heard telling their more junior colleagues, to make sure and save the “461 Book” as they will be sure to use it. Numerous alumni have told us that they frequently consult the course notes as they evaluate proposals and the professional literature – and many are surprised that these course materials are the ones they have used the most post-graduation!
PERSONAL REFLECTIONS – LESSONS LEARNED
The CQI approach represents an explicit approach to improving the quality of teaching and subsequent learning as well as an ongoing and incremental approach to innovation. Using this method over a period of years can have a substantial cumulative effect. The traditional method of altering courses is primarily reactive. Problems are brought to the attention of the instructor or the college administration and the instructor responds by changing lectures, The CQI process is a proactive approach; that is, student outcomes are evaluated issues identified, the learning process is examined, and based on the type of learning required, changes are made. It is a change in focus from that of covering material to assuring that students have learned basic concepts and skills. The goal is for students to be able to perform basic cognitive skills rather than to recall a large quantity of information for an exam.
Frustration with process and some outcomes led us to try new approaches. In the past, we have tried having the students divide into small groups to respond to questions then the instructor led a discussion of responses to the questions as a class. The classrooms are not designed in a way that is conducive to group discussions and it seems to create a lot of chaos trying to divide students into specific groups. Therefore, when we wanted to add in-class exercises, we used pairs or small groups (up to four students) without having to physically move around the room. It seems to create less chaos while allowing students to discuss the items in the activity.
Another lesson learned is related to the selection of articles for examples in class and for assignment purposes. The articles need to be well written as the students are novices and it is difficult for them to identify clear examples – to sort through a poorly written article is beyond their level of skill as well as extremely time consuming. It is difficult to put oneself in the students’ position and recognize what tyros they really are. The articles need to be short and sometimes abstracts will suffice. Again students are beginners and it usually takes them a considerable amount of time to read even a short report. The study also needs to be relatively simple, preferably with one independent variable and not more than two or three dependent variables with simple statistics – many times a tall order. The topic also needs to be relevant to pharmaceutical care. Given such strict criteria, appropriate articles are not easy to locate. Given the attitude of most students toward statistics and research design, the introductory example article concerned with the treatment of anal fissures offers an opportunity for a little levity.
CONCLUSIONS
Eventually we hope to hang all materials on the college web page, so it will be even more unnecessary to come to class for only information dissemination, rather we’ll be doing even more in-class exercises and discussion and application of principles and concepts. We plan to incorporate some of the homework assignment activities as in-class activities so more interaction and discussion are stimulated. A recent modification of our main lecture hall including an ergonomic-designed podium, digital switching system (among PC, Elmo digital presenter; and VCR) three external sources; and wireless keyboard will also provide us with additional opportunities. We plan to return to a competency-based final exam where previously we had an open-book final and the students answered inquiries about two articles and/or abstracts they were given.
We recognize and agree with Susan Johnston’s(IO) seven deadly assumptions:
Students will apply the content on their own after class.
Students don’t need instruction or tasks to be structured.
Students learn best by hearing the expert version first.
Students can integrate new information by just listening well.
Students should do their own work during class time.
Students don’t need much guidance from the instructor.
Students overcome complexity gaps between class work and tests.
We will keep these assumptions in mind as we continually strive to improve the PhPr 461 learning experience. Faculty members of all academic disciplines could apply the CQI Model to any learning process in an effort to develop innovative instructional strategies.
1Manuscript submitted based on portfolio submited to the 1999 Council of Faculties Innovations in Teaching competition
References
(1) Boyer, EL., College – The Undergraduate Experience in America. Harper and Row Publishers, New York NY (1987) p. 159.
(2) Commission to Implement Change in Pharmaceutical Education, “Background Paper II: Entry-level curricular outcomes, curricular content and educational process,” Am. J. Pharm. Educ., 57, 377-385(1993).
(3) McLaughlin, C.P and Kaluzny, A.D., Continuous Quality Improvement (1994). HealthCare: Theoq, Implementation, and Applications. Aspen Publishers, lnc.,Gaithersburg MD(1994).
(4) Draugalis, J.R. and Slack, M.K., “A course model for teaching research evaluation in colleges of pharmacy,”Am. J. Pharm. Educ., 56,48-52(1992).
(5) Draugalis. J.R., Carter, J.1 and Slack, M.K., “Survey course on research methods: Integrating statistical analysis and study design,” ibid. 62, 1723(1998),
(6) Campbell, D.T. and Stanley, J.C., Experimental and Quasi-Experimental Designs for Research. Houghton Mifflin Company, Boston MA (1963).
(7) Cook, TD. and Campbell, D.T, Quasi-Experimentation-Design & Analysis Issues.for Field Settings. Houghton Mifflin Company, Boston MA (1979).
(8) Rovers, J.P., Currie, J.D., Hagel, H.P, McDonough, R.P. and Sobotka. U, A Practical Guide to Pharmaceutical Care. American Pharmaceutical Association, Washington DC. (1998) pp. 222-224.
(9) Merrill, M.D. and Tennyson, R.D., Teaching Concepts: An Instructional Design Guide. Educational Technology Publications, Englewood Cliffs NJ (1977).
(10) Johnston, S., “Supporting student success in the classroom,” Cooperative Learning and College Teaching,. 8(3), 11-14(1998).
JoLaine R. Draugalis and Marion K. Slack
College of Pharmacy, The University of Arizona, PO. Box 210207, Tucson AZ 85721-0207
Copyright American Association of Colleges of Pharmacy Fall 1999
Provided by ProQuest Information and Learning Company. All rights Reserved
