Peer-and-Self Assessment to Reveal the Ranking of Each Individual’s Contribution to a Group Project
Teamwork and virtual teamwork are becoming more and more important in IS professions. Group project assignments play an important role to train students’ skills in teamwork in IS education. To reduce the free-rider problem and treat each group member fairly, the instructor needs to distinguish each individual’s contribution to a group project. In this paper, we analyze one commonly used peer-and-self assessment application and point out its critical drawback: the deduced ranking might be wrong as some members do not tell the truth. Alternatively, we propose an effective mechanism to modify the peer-and-self assessment. Under the revised peer-and-self assessment, truth-telling is each individual’s dominant strategy and the instructor can effectively distinguish each member’s contribution to a group project. A field experiment and the associated survey are used to validate the revised self-and-peer assessment approach. Generally, the revised peer-and-self assessment is acceptable to students and it is a valid, effective, and useful tool to the instructor.
Keywords: Peer-and-Self Assessment, Ranking of Contributions, Teamwork, Group Project, Game Theory
Teamwork and virtual teamwork are becoming more and more important in IS professions (Pottert et al. 2000). Group project assignments have become an intrinsic part of coursework in information systems education (Steenkamp, 2002). There are numerous benefits for students learning via team or group projects (Lejk and Wyvill, 1997; Lopez-Real and Chan, 1999). Some key research on the use of teamwork in information systems education includes Wojtkowski (1987), Keeler and Anson (1995), Alavi et al. (1995), Fellers (1996), Mennecke and Bradley (1998), Van Slyke et al. (1999), and Reif and Kruck (2001). In information systems education, many courses such as introduction to information systems, system analysis and design, software engineering, and management information systems require group projects as an important part of the course (Turban et al. 2004). A cooperative group project assignment provides a good experience for students to understand the relevant principles in the courses. For example, in an introductory course of information systems, a group project assignment such as assigning each team to a college collaborative tool (e.g. Blackboard, WebCT, etc.) gives students a good chance to learn how to collaborate to perform a virtual group work. McCloskey (2004) gives a teaching tip for how to form effective teams for group projects in IS courses and Martz, Shepherd and Hickey (2001) discuss how to use groupware in a classroom environment. As the important step for a group project, the instructor needs to evaluate it and assign final grades to each member in the group. The fair assessment of a group project is a very important issue because it helps to stimulate students to work hard in a group work (Leach et al. 2001). The usual practice is that the instructor reviews and evaluates a project, and assigns the same grade to all the members in a group. Although it is an easy way for the instructor to implement such an evaluation, there are some underlying drawbacks. First of all, there might be a free-rider problem in a group project (Bartlett 1995). Since all the students in a group will be assigned the same grade, the marginal efforts of one student will benefit all the people in the group. For the contributor, his marginal gain is only part of the total gains derived from his efforts. On the other hand, the people without any effort can obtain gains from other contributor’s efforts. If the students try to maximize their utility in terms of their efforts and gains, some of team members have an incentive to be a free-rider, which leads to a low quality team work. secondly, this evaluation method eliminates the difference among the people in a group. Admittedly, even though everyone in the team does his or her best, the contribution to the group project is still different from others’ because of his or her different background, motivation and intelligence. Conway and Kember (1993) points out that students complain that awarding the same mark to all group members is often not a fair evaluation of individual effort. Thirdly, the instructor usually expects a “normal” distribution of grades in any given class. Groups of students with the same grade can easily lead to grade clustering and might bring a “nonsmooth” distribution.
Due to the above mentioned drawbacks, during evaluation of a group project, the instructor needs to identify each individual’s effort and get to know the different contributions of group members. However, there is an asymmetric information problem. Although each member knows each other’s efforts in the group, for the instructor, each individual’s effort is private information. Generally, there are two approaches for the instructor to distinguish each individual’s contribution in a team. The first approach is for the instructor to conduct an investigation, such as reading logbooks designed to show the sequential progresses of a group project and the detailed descriptions of each member’s activities towards the project. However, the instructor needs to spend extra time and energy for such investigation. The second approach is to ask the students to report the efforts of all people in their group. This is called a peer-and-self assessment. There are a number of studies about the peer assessment practices (Dochy et al. 1999; Sluijsmans et al. 1999; Falchikov and Goldfinch, 2001; Sindre et al. 2003). Leach (2001) states that adult learners have a legitimate role in an assessment partnership and describes how it is used in practice. Keaten and Richardson (1992) also affirms that peer assessment fosters an appreciation for internal awards and interpersonal relationships in the classroom. So, if the peer-and-self assessment is valid, it is appealing to the instructor. One critical problem with this method is the question as to whether everyone tells the truth or not (Sindre et al., 2003). It is obvious that some group members have an incentive to exaggerate their own contribution during a peer-and-self assessment. Besides, does everyone have an incentive to tell the truth about other member’s contribution? Students feel that criticizing their friends is not easy. Hanrahan and Isaacs (2001) mentions that students show hostility towards peer assessment in their university courses. Some evidence actually demonstrates increasing opposition by students to peer assessment (Rushton et al. 1993). Kwan and Leung (1996) suggests that the role of evaluating each other upsets some students and it is “risky and unfair”. The instructor is concerned about the biasness of peer-and-self assessment and students also doubt its objectivity and claim no training in such assessment practices (Cheng and Warren, 1997; Sluijsmans and Moerkerke, 2001).
Therefore, we need a simple and reliable mechanism which allows the students to easily complete the peer-and-self assessment and also encourages them to tell the truth. If we take the peer-and-self assessment as a game played by the people in a team, we need a simple game so that every player has “tell-the-truth” as his or her dominant strategy. Rafiq and Fullerton (1996) traces one university’s approach to peer assessment by developing fair and reliable systems for group projects in the field of civil engineering. They show the relevance and drawbacks of the method of peer assessment devised by Goldfinch and Raeside (1990) and propose some new methods. Conway and Kember (1993) examines ways in which students may be awarded individual marks, reflecting personal effort, for a group project. They also criticize the method by Goldfinch and Raeside (1990), and outline a simplified scheme for assessing the contribution of an individual to a group project. Reif and Kruck (2001) takes advantage of IT and use a Web-based form to conduct student peer assessments for group works. This paper tries to investigate the same topic, but it is strikingly different from the existing general education and information systems education literature in three aspects. First, unlike the existing literature which assumes that everyone always tells the truth, this paper admits that the members in a group have an incentive to lie. second, unlike the existing literature using traditional research methodologies, this paper uses game theory to analyze the peer-and-self assessment and an associated field experiment is conducted to validate the analytical framework. Third, while the existing literature emphasizes the numerical assessment and training tools that students use for the peer-and-self assessment, this paper focuses on the ordinal ranking assessment and the mechanism design. The paper is organized as follows. In section two it reviews one commonly used peer-and-self assessment application for a group project, analyzes the assessment method’s drawbacks and proposes a modified mechanism for the assessment approach. In section three the paper discusses an extended application of the revised mechanism in a peer presentation evaluation. In section four a field experiment is presented to validate the modified self-and-peer assessment, and concluding remarks are shown in section five.
2. PEER-AND-SELF ASSESSMENT
One application of the peer-and-self assessment for a group project is conducted as follows. Each individual is asked to give a ranking evaluation for the contributions of all the members in a group. The instructor collects the evaluation reports to deduce each individual’s contribution to the group project. This application can be illustrated in detail by the following example. Suppose three students, A, B and C, form a team to conduct a project. We assume that after finishing the project, these three students know very well the ranking of their contributions as A>B>C. The instructor asks them to give a peer-and-self assessment for the teamwork. Suppose each one exaggerates his or her contribution and always ranks themselves as the highest contributor. After collecting the assessment reports, the instructor ignores the evaluator’s rank for himself, and deduces the implicit ranking for all the members in the group. For example, from Report 1 in Table 1, A tells the true ranking; B and C exaggerate their rank respectively, but they tell the truth about the ranking of the others. Now let us look at how the instructor deduces the group ranking. He eliminates A’s assessment for A, B’s assessment for B and C’s assessment for C. The following information remains: from A’s assessment, B>C, from B’s assessment, A>C and from C’s assessment, A>B. The instructor can immediately construct the ranking for the group: A>B>C, which is the correct ranking. If the instructor uses this deduced ranking to evaluate each member’s contribution, every member is treated fairly.
However, the story from Report 2 in Table 1 is different. In Report 2, A lies to tell others’ ranking, B and C exaggerate their rank respectively, but they tell the truth about others’ ranking. Now, let us check the ranking deduced by the instructor. As usual, the instructor eliminates A’s assessment for A, B’s assessment for B and C’s assessment for C. The following information remains: from A’s assessment, OB, from B’s assessment, A>C, and from C’s assessment, A>B. This results in a ranking for the group: A>OB, which is incorrect. It is easy to see that A is indifferent, but C benefits (gets promoted), and costs B (gets demoted) from Report 2. Note the instructor will not know who lied in Report 2 unless B complains about the wrong ranking.
Recall we assume that each member’s contribution to the project is strictly different, as A>B>C, which means no ties exist. If the deduced ranking from any report is a cycle, i.e., no sensible ranking can be gained from the report, the instructor will realize someone must have lied in the report. Report 3 gives us such an example. After eliminating A’s assessment for A, B’s assessment for B and C’s assessment for C, the following information remains: from A’s assessment, B>C, from B’s assessment, OA, and from C’s assessment, A>B, which lead to the cycle A>B>OA, which is invalid. In this case, the instructor realizes someone has lied and investigates the logbooks to find and punish the liar (in this example the liar is B). We outline the eight possible peer-and-self assessment reports and the associated liar(s), deduced rankings, people benefit or cost from the reports in Table 1.
Now, let us look at A’s strategy under different assessment reports. That is, which strategy is dominant for A: tell the truth or lie? We extract A’s payoffs under the eight reports in Table 1 and create Table 2 as the payoff matrix for A. From the table, we can see that “to lie” is weakly dominated by “tell the truth”. Therefore, we can say that A will choose to tell the truth in the assessment process. This choice is reasonable to A because he or she is the top contributor in the group and has no incentive to tell a lie about himself/herself or about others as he cannot get any extra benefits.
Therefore, we can delete the reports in which A lies, i.e., reports 2, 5, 6 and 8 in Table 1, only reports 1, 3, 4 and 7 remaining in Table 1. Next, let us look at what kind of strategies that B and C will take. In the same token, we focus on B’s and C’s payoffs in the remaining 4 reports and outline the payoff matrix for B and C in Table 3. The first element in parenthesis is B’s payoff and the second element is C’s payoff.
It is obvious that neither B nor C has a strictly dominant strategy. For B, given that C tells the truth, B also chooses to tell the truth, but given that C lies, B is indifferent between truth-telling and lying. For C, given that B tells the truth, C is indifferent between truth-telling and lying, but given that B lies, C will also lie. We can see there are three Nash equilibriums in B’s and C’s strategies. That is, both B and C are either truthful, or lie at the same time, or C lies and B tells the truth. Strikingly, the strategy that both lie leads to the Pareto improvement compared to the other two sets of strategies. As we can see, under the conditions that both B and C lie and A tells the truth, the ranking becomes B>OA, where both B and C are promoted. So, there are some incentives for both B and C to collude with each other to lie. So far, we can see that this simple peer-and-self assessment application has one critical drawback, that is, the deduced ranking of group members’ contributions might be wrong. Under the Nash equilibrium where both B and C are truthful, that is, everyone tells the truth, the instructor can easily obtain the correct contribution ranking of A>B>C. Under the Nash equilibrium where both B and C lie, the instructor obtains the wrong ranking of B>OA. Under the Nash equilibrium where C lies and B tells the truth, the instructor obtains the misleading ranking of B>A>C. We need to modify the game mechanism so that everyone has an incentive to tell the truth. Theoretically, we can reward the truth-teller (called the “optimistic” approach) or punish the liar (called the “pessimistic” approach) allowing a liar to switch to telling the truth, and letting the truth-teller remains truthful. Here, for simplicity, we modify the mechanism by using threat of potential penalty (If we use rewarding the truth-teller instead of punishing the liar to change B’s and C’s payoff matrix, we will get the identical analytical results). Let us put two additional rules to the game. 1) After the peer-and-self assessment, the instructor will declare the final grades to all the members in a group, and allow them to complain about the final grade ranking. That is, everyone knows not only his or her grade but also all the others’ grades in the group. This rule is reasonable because if the instructor does not distinguish each individual’s contribution, everyone gets the same grade, and this also means that everyone in a group knows each other’s grades. It is obvious that if the ranking of final grades for the group members is A>B>C, everyone receives fair treatment. Under this result, there might be no one to complain about the final result, but we cannot exclude that B or C will purposely complain about it. If the ranking of final grades is B>OA, or B>A>C, A will not be satisfied with the result. The student will definitely complain that someone lies or there is collusion between B and C. So, the complaint of A is a signal of a wrong ranking to the instructor. If we set up the penalty rule at advance, and let everyone know it before they submit their assessment reports, we can correct their misbehavior of either be lying or purposely complaining about the correct ranking. Say we add another rule. 2) If someone complains about the final grade ranking, the instructor will investigate the group project. If the instructor finds the ranking from the peer-and-self assessment is wrong, he will punish anyone who lies by demoting his/her final grade. If he finds that the ranking from the peer-and-self assessment is correct, he will punish the complainer in the same way. For example, if both B and C collude and A complains for the final grade, the instructor investigates the teamwork, and will assign the final grade like A[much greater than]B>C, where [much greater than] means that B’s and C’s final grades are largely behind of A’s. This rule also helps to prevent B or C from complaining when they are treated fairly under the correct ranking A>B>C.
Under these two additional rules, we can change the payoff matrix for B and C. We adjust their payoffs so that both B and C deviate from the strategy of “lie”. The new payoff matrix is listed in Table 4.
The new payoff matrix brings the two significant changes. Firstly, both B and C have a strict dominant strategy now. For B, no matter what kind of strategy C will take, he or she will always choose to tell the truth. For C, no matter what kind of strategy B will take, he or she will always choose to tell the truth as well. secondly, there is only one Nash equilibrium, that is, both B and C tell the truth. So, the modified mechanism eliminates the possibility of assessment reports where someone lies. Under the new game, only report 1 in Table 1 is the outcome of the peer-and-self assessment. The instructor does not need to spend time to investigate the group project, but he can distinguish the contribution ranking of group members, because the participants will tell the truth and the deduced ranking is the true ranking, that is, A>B>C.
3. PEER ASSESSMENT FOR PRESENTATIONS
For a big information systems class, it is quite difficult for the instructor to evaluate each student’s project presentation due to time constraints. Alternatively, the instructor sometimes encourages students to conduct peer assessment which can also benefit the students to develop their critical ability and increase their involvement and interests (MacAlpine, 1999; Reif and Kruck, 2002). The instructor can divide the students into several panels and ask the students to evaluate their presentations by themselves in each panel. We assume that the instructor knows that the qualities of projects are similar within one panel. The instructor wants the ranking of their presentations in order to identify the students in the panel. The common method for the instructor to conduct the peer assessment for presentations is as follows. The instructor arranges the order of presentations for students in each panel. The instructor rotates to inspect all the panels, but cannot get all the information about each presentation in each panel during this inspection. The instructor can take advantage of the IT facility to record all the presentations. We assume after the presentations in a panel, the instructor can recover all the information about them, and correctly evaluate the ranking of the members in the panel from different channels such as reviewing the slides and videos of the presentations. The presentation and evaluation in a panel are usually processed as follows. After one student finishes his or her presentation, all the other students give an individual assessment and the instructor collects their assessment reports. When the next person presents, all the other students including the previous presenter will give their individual assessments. The procedure ends when all the students finish their presentations and assessments, and the instructor collects all the assessment reports. This game seems quite fair, but actually it is not. Let us look at the following example. Suppose we have three students A, B and C in a panel. The presentation and evaluation process is described in the Table 5.
The order for presentation is A, B and C. Before B and C give their individual assessments to A’s presentation, they might consider that in the next turn(s), A will give his or her evaluation for their presentations. B and C might be concerned that once A gets to know what kind of assessment is assigned to them, this might affect A’s evaluations of them. So, B and C are inclined to avoid communicating with A including asking A questions about his or her presentation. At the same time, B and C would be wondering that, if A’s presentation is bad, A might have an incentive to give B and C lower evaluations no matter how good or bad their presentations are. Therefore, B and C will consider not only the true quality of A’s presentation, but also the possible action that A will take on them. Therefore, both B and C are more likely to lie. Also, after B’s presentation, both A and C will give evaluations to B. However, you can see the different situations for A and C. Since A has already presented, he or she would not consider B’s ex post assessment, but may consider B’s ex ante evaluation. C, would consider the ex post evaluation of B. After C’s turn for presentation, both A and B give their individual evaluations to C. In the same fashion, both might consider what kinds of ex ante evaluations that C has assigned to them. Obviously, this sequential or dynamic game for peer assessment is not symmetric to each player and its result might be biased. Another drawback for this peer assessment application is that before C finishes his or her presentation, no one can compare and know the true ranking of the presentations in the panel. It is only after C has presented that the true information about the ranking is complete and known to everyone. But under this assessment procedure, they cannot modify their previous evaluations. Therefore, the sequential peer assessment for presentations is not appropriate in this setting. Alternatively, we try to convert the sequential game into a simultaneous game. Then, we can use the revised mechanism described in the previous section. That is, let A, B and C present their works first. Then, the instructor asks them to give the ranking assessments on their presentations at the same time. In this way, this peer assessment for presentations in a panel is identical to the peer-and-self assessment for a group project. By using the same game mechanism, we ensure that everyone in this panel has to tell the truth.
4. VALIDATION: A FIELD EXPERIMENTAL STUDY
To validate the modified self-and-peer assessment, we conducted a field experiment in class for an introductory information systems course. We divided 36 students into 12 teams (from Team #1 to Team #12) where each group was composed of three people. The teamwork was to ask each group to design and prepare a group travel plan to Hawaii during Christmas holidays by exploring relevant information through the Internet. Team members must hold at least three meetings to discuss the joint project, and they were required to record collective activities and individual activities as detailed as possible in their logbook. For individual activities, they must be endorsed by other members. Each group had ten days to finish its teamwork. We announced that the instructor would not only grade their joint project, but also read their logbook to identify each member’s contribution to the group work. We also told the students that different members in the same team might get different points and how much each member got would depend on their joint work and their individual contribution. If they had any question during the group work, the students could contact the instructor for help. On the due date, after collecting the finished joint projects, we asked the students to conduct a self-and-peer assessment. Each student was required to write the ranking of each member’s contribution to the group work in a descending order. If a student thought that the contribution of team members was the same, they explicitly wrote the sentence “All team members contribute equally”. We also had announced the rules of the revised self-and-peer assessment to the students before they started. Instead of using the “pessimistic” rule which applies punishing the liars or purpose complainer in the self-andpeer assessment, we used the “optimistic” rule since we thought the “optimistic” rule was more acceptable and less likely to create confrontational learning environment. We announced the regulations as below: “First, anyone who participates in the self-and-peer assessment will get 10 points extra credit for their project work except if they are accused of lying or purposely complained about the correct ranking (see the third rule below for the details); second, after grading the group work and the peer-and-self assessment, the instructor will declare the final grades to all the people in each team, and allow them to complain about the final grade ranking; Third, if someone complains about the ranking, the instructor will check the logbook. If the ranking from the peer-and-self assessment is wrong, anyone who lies in the peer-and-self assessment will only get 2 points extra credit instead of 10 points. If the instructor finds that the ranking from the peer-and-self assessment is correct, the complainer will only get 2 points extra credit instead of 10 points”. In order to know the attitudes and perceptions of students towards to the self-and-peer assessment, we conducted a survey after the self-and-peer assessment (see the survey in Appendix 1).
From all the student’s assessment reports, we figured out that people in three groups (Team #3, Team #4 and Team #12) had written the ranking in a descending order, which suggested the contribution of each member in each team was not equal. The students in the rest nine groups stated each member had contributed equally in their team. In order to check whether the students in Teams #3, #4 and #12 really told the truth about the ranking of each member’s contribution under the new assessment mechanism, we carefully read the logbooks of each team. We found that the members in each of Team #3, Team #4 and Team #12 did contribute differently and each member’s assessment report correctly reflected the ranking of each member’s contribution. We also checked the logbooks of the other nine teams. We found that people in Team #10 had contributed slightly differently and people in each of the other eight teams had contributed equally. We announced the ranking results from the self-and-peer assessments to each team, and none of students complained about the rankings. For Team #10, we checked whether the students purposely lied in the assessment, or lied for any other reasons. We recalled and talked to the three members in Team #10. We found that for this team, they simply divided their group work into three small parts, and each member only focused on his/her own job, and then, simply combined the three parts together as “one completed project”. For group discussion in Team #10, each member only reported his/her progresses and problems. The reason that the team members in Team #10 stated they had contributed equally was because they had not monitored and known each others’ work. In other words, they simply segmented the group project into three individual pieces. Since they did not know the real contributions of other members, they assumed that each one had contributed equally. This leads to us another problem in the group project, namely can we make a team work as a real team work and let each member collaborate, coordinate and monitor each other? This exception of Team #10 does not imply our proposed self-and-peer assessment is a failure because the subjects in the team did not know the true ranking themselves. It is true that if each team member did not know other members’ contributions, any kind of self-and-peer assessment will not work. Generally speaking, the field experiment validates the success of our proposed self-and-peer assessment approach.
In order to know the attitudes and perceptions of students towards the group project and the self-and-peer assessment, we conducted a survey. The survey results are listed in Table 6. From Table 6, we can summarize the following survey results about the group work and the self-and-peer assessment: First, most of students (72%) realized that the group work was useful for their course study. But, still 28% of students did not realize such importance to their study. This implies there is a need to emphasize the importance of group work in IS courses. second, the difficulty level of the group work used for this field experiment has a normal distribution, which suggests the group project assignment is appropriate. Third, most of student (83%) thought that everyone had worked hard towards the group project. Some portions (17%) observed the negative attitudes of students to the team work. Fourth, a majority of students (70%) were cognizant of that everyone’s contribution was equal, but a substantial portion (25%) reported the free-rider problem in their group work. Fifth, it is significant that a majority (47%) of students did not accept that everyone got the same grade in their team if each team member’s contribution was not equal. This justifies the importance of distinguishing the different contribution of each team member and application of our proposed self-and-peer assessment approach. Sixth, almost all of students could accept the self-and-peer assessment. Only 5% of students felt uncomfortable with the assessment. This result shows the revised self-and-peer assessment approach in the study is applicable and acceptable to students. Seventh, only a small portion of students thought the “optimistic” (rewarding) rule is not acceptable. Most of students thought either the rule was helpful to evaluate the students’ contribution so that everyone was treated fairly, or the rule was similar to other school regulations. Eighth, when we tried to use “pessimistic” (punishing) rule instead of “optimistic” approach, the attitude of some students changed. In the survey, more people thought the pessimistic approach was not acceptable to the students (from 11% to 22%) and 42% of students thought the punishing rule was worse than the previous rewarding rule. From the responses of the sixth, seventh and eighth questions surveyed, we can see the revised self-and-peer assessment approach is acceptable to most of students, and the “optimistic” rule is better than the “pessimistic” rule. If some instructors are worried about the potential confrontational learning environment when students conduct the self-and-peer assessment, they are encouraged to use the “optimistic” rule instead of the “pessimistic” rule.
Teamwork and virtual teamwork are becoming more and more important in IS professions. Therefore, group project assignments play an important role to train students’ teamwork skills in information systems education. To reduce the free-rider problem and treat each group member fairly, the instructor needs to distinguish each individual’s contribution to a group project. However, there is an asymmetric information problem for the instructor. The instructor needs to invest extra efforts to explore the information about the ranking of each member’s contribution in a group; however, there is an associated cost in doing this. In the paper, we analyze one commonly used peer-and-self assessment application and point out its critical drawback: the deduced ranking might be wrong as some members do not tell the truth. Alternatively, we offer an effective mechanism to modify the peer-and-self assessment. The advantage of the revised peer-and-self assessment is that under the new mechanism, truth-telling is each individual’s dominant strategy. Therefore, by using the revised peer-and-self assessment, the instructor can effectively distinguish each member’s contribution to a group work. This paper also points out the drawbacks of the commonly used sequential peer assessment for presentations. It suggests that in order to reduce the assessment asymmetry and evaluation bias, we need to convert the sequential game into a simultaneous game and use the revised peer-and-self assessment. A field experiment was used to validate the revised self-and-peer assessment approach. Generally, the field experiment and the associated survey show that the revised peer-and-self assessment is acceptable to students and it is a valid, effective, and useful tool for the instructor.
Alavi, M., Wheeler, B. and Valacich, J. (1995), “Using IT to Reengineer Business Education: An Exploratory Investigation of Collaborative telelearning”, MIS Quarterly, Vol.19, No.3, 1995, pp.293-312
Bartlett, R. L. (1995), “A flip of the Coin – A Roll of the Die: An Answer to the Free-Rider Problem in Economic Instruction”, Journal of Economic Education. Vol. 26, No. 2, 1995, pp 131-139
Cheng, W. and Warren, M. (1997), “Having second Thoughts: Student Perceptions before and after a Peer Assessment Exercise”, Studies in Higher Education. Vol. 22, No. 2, 1997, pp. 233-239.
Conway, R. and Kember, D. (1993), “Peer Assessment of an Individual’s Contribution to a Group Project”, Assessment & Evaluation in Higher Education. Vol. 18, No. 1, 1993, pp 45-57
Dochy, F., Segers, M. and Sluijsmans, D. (1999), “The Use of Self-, Peer-, and Co-Assessment in Higher Education: a Review”, Studies in Higher Education. Vol. 24, No. 3, 1999, pp. 331-350.
Falchikov, N. and Goldfinch, J. (2001), “Student Peer Assessment in Higher Education: a Meta-analysis Comparing Peer and Teacher Marks”, Review of Educational Research. Vol. 70, No. 3, 2001, pp. 287-322.
Fellers, J. (1996), “Teaching Teamwork: Exploring the Use of Cooperative Learning Teams in Information Systems Education”, The Data Base for Advances in Information Systems. Vol.27, No.2, 1996, pp. 44-59
Goldfinch, J. M. and Raeside, R. (1990), “Development of Peer Assessment Technique for Obtaining Individual Marks on a Group Project”, Assessment and Evaluation in Higher Education. Vol. 15, No. 3, 1990, pp. 210-225.
Hanrahan, S. and Isaacs, G. (2001), “Assessing Self- and Peer Assessment: the Students’ Views”, Higher Education Research and Development. Vol. 20, No. 1, 2001, pp. 53-70.
Keaten, J. A. and Richardson, M. E. (1992), “A Field Investigation of Peer Assessment as Part of the Student Group Grading Process”, Paper presented at the Western Speech Communication Association Convention, Albuquerque, NM, 1992
Keeler, C and Anson, R. (1995), “An Assessment of Cooperative Learning Used for Basic Computer Skills Instruction in the College Classroom”, Journal of Education Computing Research. Vol.12, No.4, 1995, pp.379-393
Kwan, K. and Leung, R. (1996), “Tutor versus Peer Group Assessment of Student Performance in a Simulation Training Exercise”, Assessment and Evaluation in Higher Education. Vol. 21, No. 3, 1996, pp. 205-214.
Leach, L., Neutze, G. and Zepke, N. (2001), “Assessment and Empowerment: Some Critical Questions”, Assessment & Evaluation in Higher Education. Vol. 26, No. 4, 2001, pp 293-305
Lejk, M. and Wyvill, M. (1997), “Group Learning and Group Assessment on Undergraduate Computing Courses in Higher Education in the UK: Results of a Survey”, Assessment & Evaluation in Higher Education. Vol. 22, No. 1, 1997, p81-92
Lopez-Real, F. and Chan, Y.R. (1999), “Peer Assessment of a Group Project in a Primary Mathematics Education Course”, Assessment and Evaluation in Higher Education. Vol. 24, No. 1, 1999, pp 67-79
MacAlpine, J.M.K. (1999), “Improving and Encouraging Peer Assessment of Student Presentations”, Assessment and Evaluation in Higher Education. Vol.24, No.l, 1999, pp 15-25
Martz, B. and Shepherd, M. and Hickey, A. (2001), “Using Groupware in a Classroom Environment”, Journal of Information Systems Education. Vol.12, No.l, 2001, pp. 31-41
McCloskey, D. (2004), “Adding Realism to the Formation, Management and Evaluation of Project Teams”, Journal of Information Systems Education. Vol.15, No.l, 2004, pp. 9-11
Mennecke, B. and Bradley, J. (1998), “Making Project Group Works: The Impact of Structuring Group Role on the Performance and Perception of Information Systems Project Teams”, Journal of Computer Information Systems. Vol.39, No.l, 1998, pp.30-36
Pottert, R.E., Balthazardtt, P. and Lee Elder, K. (2000), “Delivering the Cross cultural Virtual Teamwork Experience: An Example with Mexican and American MBA Team Members”, Journal of Information Systems Education. Vol.11, No.3-4, 2000, pp. 111-220
Rafiq, Y. and Fullerton, H. (1996), “Peer Assessment of Group Projects in Civil Engineering”, Assessment & Evaluation in Higher Education. Vol. 21, No. 1, 1996, pp 83-91
Reif, H. and Kruck, S.E. (2002), “Integrating Student Groupwork Rating into Student Course Grades”, Journal of Information Systems Education. Vol.12, No.2, 2002. pp.57-63
Rushton, C., Ramsey, P. and Rada, R. (1993), “Peer Assessment in a Collaborative Hypermedia Environment: a case Study”, Journal of Computer-Based Instruction. Vol. 20, No. 3, 1993, p. 75.
Sindre, G., Moody, D.L., Brasethvik, T. and Solvberg, A. (2003), “Introducing Peer Review in an IS Analysis Course”, Journal of Information Systems Education. Vol. 14, No.1,2003, pp.101-119
Sluijsmans, D., Dochy, F. and Moerkerke, G. (1999), “Creating a Learning Environment by Using Self-Peer and Co-Assessment”, Learning Environments Research. Vol. 1,No. 3, 1999, pp. 293-319.
Sluijsmans, D., Moerkerke, G., Dochy, F. and Van Merrienboer, J. J. G. (2001), “Peer Assessment in Problem Based Learning”, Studies in Educational Evaluation. Vol. 27, No. 2, 2001, pp. 153-173.
Steenkamp, A.L. (2002) “A standards-based Approach to Team-based Student Projects in an Information Technology Curriculum”, Proceedings of the 17th Annual Conference of the International Academy for Information Management, 2002, pp. 54-62
Turban, E., McLean, E. and Wetherbe, J. (2004), “Information Technology for Management”, 2004, John Wiley & Sons, Inc
Van Slyke, C., Trimmer, K. and Kittner, M. (1999), “Teaching Teamwork in Information Systems Courses”, Journal of Information Systems Education. Vol.10, No.3-4, 1999, pp.36-46
Wojtkowski, W. and Wojtkowski, G. W. (1987), “Utilizing Group Learning in Computer Information Classes”, Journal of Education for Business. Vol.62, No.8, 1987, pp.346-352
Department of Operations and Information Management
School of Business Administration
University of Connecticut
Storrs, CT 06269 USA
Department of Economies
University of British Columbia
Vancouver, BC V6T IZl Canada
Yanbin Tu is a PhD Candidate in the Department of Operations and Information Management, Business School at the University of Connecticut. He holds an M.A. in Economics from Simon Fraser University, Canada and an M.A. in Economics from East China Normal University, China. His research interests include electronic commerce, e-marketing, IT product sales, economics of information systems and information systems education. His work has appeared or will appear in Proceedings of the 2004 Americas Conference on Information Systems, Proceedings of the 2004 Annual Meeting of Decision Sciences Institute, Marketing Theory and Applications, and Proceedings of the 2004 Workshop on E-Business.
Min Lu is a PhD candidate in the Economies Department at the University of British Columbia, Canada. She holds a M.A. in Economics from Simon Fraser University, Canada. Her research interests include economics of information systems, e-marketing, international finance, and monetary macroeconomics. Her work will appear in Marketing Theory and Applications, and Proceedings of the 2004 Workshop on E-Business.
Copyright EDSIG Summer 2005
Provided by ProQuest Information and Learning Company. All rights Reserved