Value of Practice Quizzes for Developmental Students, The

Brothen, Thomas

Abstract

This study compares the behavior of developmental students with non developmental students in their use of Internet practice quizzes. These quizzes proved to be beneficial to all students, but as a group, the developmental students used them less effectively. On the other hand, the quizzes proved to be particularly effective for those developmental students who increased their use of them during the semester. We discuss the importance of not leaving the use of such learning aids to chance and suggest that instructors find ways to induce developmental students to use them.

The National Association of Developmental Education defines developmental education, in part, as “development of general and discipline-specific learning strategies” (“Definition of,” 2001). To be ultimately useful, the learning strategies taught to developmental students need to generalize to their subsequent courses. However, getting students to transfer their newly acquired skills is not an easy task even though most educators identify it as an important objective (Haskell, 2001). For many years, we have facilitated the development of student learning strategies by creating Internet practice quizzes for students in our mastery learning model general psychology course (Bloom, 1976; Brothen & Wambach, 2000). Our primary objective with this technique has been to teach students the value of acquiring feedback on their learning progress (Wambach, Brothen, & Dikel, 2000). Our evaluations of this course have consistently demonstrated that students who use practice tests earn better grades in the course (Brothen & Wambach, 2000) but we have little evidence that feedback seeking such as that facilitated by practice tests transfers to other courses.

Brothen (1998) reviewed research demonstrating that mastery learning courses are learning environments in which developmental students can be as successful as more highly academically qualified students. In the context of a mastery approach, Internet practice quizzes have important effects beyond preparing students for specific tests. Because they give students immediate feedback on their studying and refer them to their textbook or other source of information to restudy, two important things become more probable. First, students learn the information better and come to appreciate the value of feedback and review. Second, they get feedback about the effectiveness of their learning strategies and how to improve them. This approach echoes Thomas and Rohwer’s (1986) executive monitoring method of studying: students appraise their need for further study, deploy strategies to meet those needs, and assess their learning progress. Reading, taking a computerized quiz that gives informational feedback, restudying, and retaking the quiz nicely fits the executive monitoring model.

In two previous studies (Brothen & Wambach, 2001a, 2004) we described how students used Internet practice quizzes to assess their knowledge of course material and how important it is for instructors to set time limits on such quizzes to make them more effective as learning tools. We completed these studies in a psychology of human life-span development course for prospective nursing students-experienced students with high college entrance scores and high grade point averages. Students could take chapter practice quizzes over the Internet that prepared them for unit tests. They also completed a comprehensive research paper in a writing-intensive environment that required multiple drafts and redrafts under our close supervision.

Our first study showed that without time limits, some students tried to use the quizzes as a shortcut to learn the material. They would start a quiz before reading the text and skim it to find correct answers. The typical pattern was for these students to spend as long as it took to find the correct answers to as few of our 10 item quizzes as possible. Students taking this approach did poorly on the exams. We countered this strategy in our second study by placing time limits on the quizzes that made it highly unlikely that students who did not read the text could find the answers before the quiz had to be submitted. We found this procedure induced students to read before attempting the practice quizzes.

Because of a change in Nursing School policies, we were presented with the opportunity to see if our developmental students transferred their use of practice quizzes to a new situation and to compare their behavior with that of non-developmental students. The typical past student in our psychology of human lifespan development course had been a sophomore or junior student who had a grade point average sufficient to enter the highly competitive nursing program at the University of Minnesota. The policy change was that students were advised to take human development in the second semester of their freshman year. Because students who have more credits register before students with fewer credits, we reserved seats in the course for freshmen students in our developmental program. Thus, in addition to three sections of the course which enrolled students from the usual, highly qualified population, we opened an additional section for developmental students so that they would have an opportunity to get into the course before it filled with the usual cohort of upperclassmen. This study reports our experience with developmental and non-developmental students’ Internet practice quiz taking behavior.

In our previous research (Brothen & Wambach, 2000, 2001a, 2004) practice quizzes were worth points towards students’ grades. In the study reported here, students received no points for doing the practice quizzes-they existed only as a study aid. Because we did not lecture on the textbook material or otherwise systematically help students prepare for tests, the practice quizzes were a central feature of our course. Because they had completed general psychology and were pursuing transfer to the highly competitive nursing program, we expected that our developmental students would make use of study aids provided in the course. Thus, we hypothesized that developmental students would utilize the quizzes as effectively as other students.

Method

Participants

We began the semester with 96 students enrolled in 4 sections of our psychology of human life-span development course-all of whom had completed the prerequisite general psychology and college composition courses. Seven of the original 96 enrollees either formally withdrew early or failed to finish the course and were dropped from the study. This left 89 participants, 24 developmental (Dv) students and 65 non-developmental (NDv) students. One section contained 19 Dv students and 2 NDv students while 5 additional Dv students were enrolled in 1 of the other 3 sections containing nearly all NDv students. Two sections met simultaneously in a 48 station computer classroom one night each week while the other 2 met together the next night each week. Both instructors (the present authors) met with their respective sections together during the 2 class periods each week.

Procedure

All sections were taught the same with identical textbook assignments, quizzes, tests, and paper assignments. The class operated in a combined mastery learning (Brothen & Wambach, 2000), writing laboratory format. Students studied the textbook with the aid of chapter practice quizzes and came to class to take 8 unit tests. We delivered the quizzes and tests over the Internet via the WebCT course management system. Students worked on 6 writing assignments that required them to do library research on a topic of their choosing, write reports on their research, consult with us about strategies for improving their writing, revise the assignments in class with our help, and turn in their assignments for grading and additional feedback. The writing project culminated in a 10-15 page formal research paper.

Students in both sections took 10 item multiple-choice practice quizzes for each of the 26 textbook (Berger, 2001) chapters on a self-paced schedule as many times as they liked at any computer with access to the Internet. The WebCT quiz feature drew items randomly from item pools of approximately 90 to 100 for each chapter. Each quiz had a 10 minute time limit to discourage students from looking up answers and after the students finished a quiz they saw all the questions from that quiz with their answers highlighted, whether each was correct, and the total number correct.

The course syllabus and our first day course introduction stressed that the best way for students to approach practice quizzes was to use them for feedback on the effectiveness of their studying and to complete them before the respective unit exams which would only include material also tested on the quizzes. We told students that the best approach was to study each chapter thoroughly, take the quiz without looking at their books, use quiz feedback to find the sections in the textbook the questions they missed came from, restudy those sections, and then try again. However, they were free to use the quizzes or not use them as they chose.

The 8 computerized unit tests were closed book, 30 minute, 20 item multiple-choice exams covering 3 to 4 chapters. Students could take each test 3 times and upon finishing saw their scores and a textbook reference for each missed question. They could restudy and take the test again during the testing period and their highest score counted toward their grade. We drew the items for unit tests directly from the chapter practice quiz item banks. Students could very well have seen all of them on the practice quizzes if they took enough of them. Students could keep track of all their test scores and paper grades with the WebCT point-checking tool.

At term’s end, we obtained students’ number of completed credits prior to the term in which they took our course, gradepoint averages (GPA) in those prior credits, high school percentile ranks, and ACT scores from the university records office (consistent with human subjects guidelines). WebCT keeps quiz records that we also accessed for this study. For chapter practice quizzes, we recorded their highest scores, number of quizzes taken, and the number of perfect scores on each quiz. We also recorded their unit exam scores.

Results

We hypothesized that Dv and NDv students would not differ in their use of the practice quizzes. We tested this hypothesis in several ways. First, our previous research on Internet practice quizzes demonstrated that their proper use is beneficial to student learning (Brothen & Wambach, 2001a, 2004). We replicated that finding in this study. For each of the 8 unit tests, we correlated the total number of practice quizzes each student took on that unit with his or her test score. The resulting 8 correlations ranged from +.26 to +.52. All were significant with p

Differences Between Developmental and Nondevelopmental Students

First, we examined with t-tests for independent samples the academic predictor differences between our 24 Dv and 65 NDv students. As expected, the NDv students’ mean high school ranks were higher (Ms = 67.73 vs. 48.63, t = 2.57, p

Second, we examined with t-tests for independent samples the course performance differences between NDv and Dv students. All 8 unit test scores were higher for NDv students and all were significantly different with p

Third, we examined with t-tests for independent samples the difference in number of practice quizzes taken by NDv and Dv students and how their performance differed on them. Over the semester, NDv students took more practice quizzes but the differences were not significant (M = 125.25 vs. 105.71, t = .75, p =.456). However, the NDv students attained higher scores on each of the 26 chapter practice quizzes. The mean high score across chapters ranged from 6.36 to 8.89 for Dv students and from 8.67 to 9.54 for NDv students. For the 26 differences, two were non significant (p > .05) four were marginally significant (ps from .051 to .095) and 20 were significant by conventional probability levels (p

Taking of practice quizzes also was associated with grade differences for the Dv and NDv students. We divided students into the top quartile for taking practice quizzes (> 157 for the semester) or the bottom quartile (

Approach to Practice Quizzes

As we noted above, taking practice quizzes correlated with test performance. The correlation between total number of practice quizzes taken was significantly related to total unit test points (r = .42, p

To explore the high score differences, we examined the number of practice quiz scores of 10 attained by students-an indication of chapter mastery. NDv students achieved twice as many 10s over all takings of practice quizzes over all chapters than Dv students but this difference did not achieve statistical significance, likely due to the large standard deviations (M= 20.63 vs. 10.50, SD = 29.31, vs. 34.16, t = 1.29, p = .201). However, for all students, the number of 10s achieved did correlate with total test points (r =.44, p

We explored these differences more carefully by examining the relationships between achieving 10s and other variables for all students in the course. Total number of 10s did not correlate with high school rank (r = .20, ns), ACT Comprehensive score (r = .07, ns), or prior grade point average (r =.20, ns). However, it did correlate with total number of practice quizzes taken (r = .82, p

Developmental Students ‘ Changes in Practice Quiz Taking

Across the entire course, students increased practice quiz taking after the first unit test. The Dv students took an average of 2.46 (SD = 3.00) quizzes for each chapter on test 1 and increased to a high of 5.49 (SD = 5.67) on test 5 while the NDv students took an average of 3.28 (SD = 2.72) on test 1 and increased to a high of 5.53 (SD = 5.21) on test 5. The repeated measures linear trend for Dv students was significant (F = 5.63, p

As noted above, taking more practice quizzes was associated with better unit test scores and this was particularly true for Dv students-but this was not true of all of them. A median split on the Dv students’ GPA (at 3.0) divided them into 2 groups of 12 and revealed a telling difference. As might be expected, the higher GPA students had more total test points than the low GPA students (M = 133.25 vs. 116.08, t = 2.79, p

What caused this difference? Our data suggests that it was because many high GPA Dv students “figured out” how to use the practice quizzes as the semester progressed. This is indicated by the fact that there was no correlation between the change in number of practice quizzes taken in the second half of the semester and test score increases in the second half for low GPA Dv students (r = .20, ns) while there was a strong relationship for high GPA Dv students (r = .67, p

We then examined the Dv students’ prior performance in our personalized system of instruction general psychology course (Brothen & Wambach, 2000) in which practice quizzes were required and high scores resulted in course points. Better grades in our general course were associated with quiz behavior in the present course. There was no relationship between grade in our general course and high school rank (r = -.08, p = .74) or ACT Comprehensive score (r = .22, p = .37). However, the more successful students were in our general course, the more practice quizzes they took in the development course (r = .46, p

Discussion

It is clear from our previous research (Brothen & Wambach, 200 Ia, 2004) and from the results of this study, that doing Internet practice quizzes is positively associated with learning course material and course grade. However, our hypothesis in this study was only partially supported. Our previously more successful students were likely to use the quizzes effectively. The data from this study reinforces the conclusion that students must approach such quizzes correctly to benefit from them. This study revealed two things about developmental students’ approach to such quizzes. First, as a group, they did not utilize them as much and as effectively as the non-developmental students. The extensive use of Internet practice quizzes in our general psychology course apparently did not readily transfer to the subsequent course. The previous course’s quizzes were required and high scores resulted in course points. Apparently, not all of our general psychology students-particularly the ones who did not do as well in that course, had internalized the learning advantages of using practice quizzes as well as we had hoped. But some apparently recognized their value, transferred their use, and did better in the subsequent course.

Second, as a group, our developmental students had increased their practice quiz taking behavior by the fifth unit test to the level of the non-developmental students. However, the higher GPA students accounted for this increase. Apparently, the lower GPA students who needed the practice most did them the least while our higher achieving developmental students did practice quizzes at a higher rate to better effect.

Conclusion

We believe that Internet practice quizzes are good for developmental students but instructors need to find ways to convince students of their inherent value. In our general psychology course, we rely on points to get students to do them and to persist until they get high scores. In subsequent courses using such quizzes, instructors should strive to strike a balance between giving students points and trying other ways to induce students to use them. Deci and Ryan’s (1985) position on

reinforcement procedures in education suggests that giving points leads students to an instrumental approach whereby they only seek out the immediate reward rather than using practice quizzes as part of a good study plan. But our data also showed that if we depend on our developmental students’ intrinsic motivation to use such quizzes effectively, we may well be disappointed. The data suggested that the more successful developmental students realized early in the semester that the quizzes would be helpful to them. We think that is one reason why they became better students at some point after entering college-they began taking advantage of opportunities to improve their performance. For example, they saw our general psychology course practice quizzes as study tools rather than simply a way to gain points. Other developmental students needed more convincing and simply making the quizzes available to them as a study tool was not enough.

The challenge for developmental educators is to find the right “mix” of inducements that stimulate effective behaviors in their students. We outlined an approach to this in two theoretical papers (Brothen & Wambach, 2001b; Wambach et al., 2000). In brief, our position is that students need to be challenged, supported, and given feedback on their performance. In the course described in this study, students needed to recognize that the feedback and support that Internet practice quizzes provided was useful in meeting the challenging course requirements. In the future, we need to develop and explore techniques to induce students to use practice quizzes, as well as other methods, to improve their performance as students.

References

Berger, K. S. (2001). The developing person through the lifespan, (5th ed.). New York: Worth.

Bloom, B. (1976). Human characteristics and school learning. New York: McGraw-Hill.

Brothen, T. (1998). Transforming instruction with technology for developmental students. Journal of Developmental Education, 21 (3), 2-4, 6, 8.

Brothen, T., & Wambach, C. (2000). A research based approach to developing a computer-assisted course for developmental students. In J. L. Higbee, & P. L. Dwinell (Eds.), The many faces of developmental education (pp. 59-72). Warrensberg, MO.: National Association for Developmental Education.

Brothen, T., & Wambach, C. (2001a). Effective student use of computerized quizzes. Teaching of Psychology, 28, 292-294.

Brothen, T., & Wambach, C. (2001b). A selectionist approach to developmental education. In D. B. Lundell & J. L. Higbee (Eds.), Theoretical perspectives for developmental education (pp. 165-172). Minneapolis, MN: Center for Research in Developmental Education and Urban Literacy, General College, University of Minnesota.

Brothen, T., & Wambach, C. (2004). The value of time limits on Internet quizzes. Teaching of Psychology, 31, 62-64.

Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press.

Haskell, R. E. (2001). Transfer of learning: Cognition, instruction, and reasoning. San Diego, CA: Academic Press.

National Association for Developmental Education. Definition of Developmental Education. (2001). Retrieved February 8, 2005 from http://www.nade.net/A1.%20de_definition.htm

Thomas, J. W., & Rohwer, W. D. (1986). Academic studying: The role of learning strategies. Educational Psychologist, 21, 19-41.

Wambach, C., Brothen, T., & Dikel, T. N. (2000). Toward a developmental theory for developmental educators. Journal of Developmental Education, 24(1), 2-4, 6, 8, 10, 29.

By Thomas Brothen and Cathrine Wambach, General College, University of Minnesota

Thomas Brothen is Morse-Alumni Distinguished Teaching Professor of Psychology and Social Sciences in the General College of the University of Minnesota. Cathrine Wambach is an Associate Professor and Director of the Office of Research and Evaluation in the General College of the University of Minnesota.

Copyright New York College Learning Skills Association, Developmental Studies Department Spring 2006

Provided by ProQuest Information and Learning Company. All rights Reserved