What’s the difference? – college-level distance and classroom-based education

What’s the difference? – college-level distance and classroom-based education

James P. Merisotis

Outcomes of Distance vs. Traditional Classroom-Based Learning

What’s the difference between distance learning and traditional classroom-based instruction? This question has become increasingly prominent as technology has made distance learning much more common. In fact, there is now at least one major Web site, maintained by North Carolina State University’ s Thomas Russell, dedicated to this question. The Russell Web site (and a recently published companion book) is called The No Significant Difference Phenomenon, and compiles various articles, papers, and research studies on distance learning.

With few exceptions, the bulk of these writings suggest that the learning outcomes of students using technology at a distance are similar to those of students who participate in conventional classroom instruction. The “no significant difference” finding has become accepted as fact in the policy community in particular, where at least some public officials have pronounced that the last college campus has been built.

We decided to examine the issue of distance learning’s effectiveness by reviewing the available evidence on the subject. This was accomplished simply through a thorough review of the literature, including everything from original research to how-to articles to policy papers. We were determined to find out the answers to several key questions: What are the findings of the research on the effectiveness of distance education? Are they valid? Are there gaps in the research that require further investigation and information? What does the literature suggest for the future?

Because so much has been written about distance learning in higher education – there are now at least half a dozen journals that deal with college-level distance education as their main theme – we limited the scope of our review to material published during the 1990s. This still left us with the task of reviewing several hundred articles, papers, and dissertations.

It turns out that the vast majority of what is written about distance learning is opinion pieces, how-to articles, and second-hand reports that don’t include original research with subjects (students or faculty) who are being studied. We focused our inquiry on the original research, including experimental, descriptive, correlational, and case studies, since this is the only appropriate way to assess the differences between distance and classroom-based learning. We certainly don’t think we reviewed every study published since 1990, but our analysis clearly captured the most important and salient of these works.

WHAT DOES THE RESEARCH SAY?

One of our most important conclusions from this review is that there is a relative paucity of original research dedicated to explaining or predicting phenomena related to distance learning. Despite the large volume of written material concentrating on distance learning, the amount of original research is quite limited. Our analysis encompassed about 40 of these original works of research – a number far smaller than is often cited as “evidence” that there is no significant difference.

From this more limited group of original research, three broad measures of the effectiveness of distance education are usually examined. These include

* Student outcomes, such as grades and test scores;

* Student attitudes about learning through distance education; and

* Overall student satisfaction toward distance learning.

Most of these studies conclude that, regardless of the technology used, distance-learning courses compare favorably with classroom-based instruction and enjoy high student satisfaction. For example, many experimental studies suggest that the distance-learning students have similar grades or test scores, or have the same attitudes toward the course. The descriptive analyses and case studies focus on student and faculty attitudes and perceptions of distance learning. These studies typically conclude that students and faculty have a positive view toward distance learning.

A closer look at the research, however, suggests that it may not be prudent to accept these findings at face value. Several problems with the conclusions reached through this research are apparent. The most significant problem is that the overall quality of the original research is questionable and thereby renders many of the findings inconclusive.

The findings of the original research must be read with some caution. Assessing the quality of the original research requires a determination that the studies adhered to commonly accepted principles of good research. This kind of analysis is much more than an academic exercise. These principles are essential if the results of the studies are to be considered valid and generalizable. If a study does not abide by these principles, the results can be erroneous or misleading, and thereby lead to conclusions that result in poor public policy.

KEY SHORTCOMINGS OF THE RESEARCH

Below are four shortcomings of the original research:

1. Much of the research does not control for extraneous variables and therefore cannot show cause and effect. Most experimental studies of distance learning are designed to measure how a specific technology (the “cause”) impacts upon some type of learning outcome or influences student attitudes toward a course (the “effect”). For this relationship to be assessed accurately, other potential “causes” must not influence the measured outcomes. But in almost all of the experimental research, there was inadequate control of extraneous variables. As a result, it was often difficult to rule out differences other than the technology as the “causal agents.”

2. Most of the studies do not use randomly selected subjects. The single best way of controlling for extraneous variables is to assign students randomly to both the experimental and control groups. However, many of the published studies reviewed used intact groups for comparison purposes. As a result, these studies run the risk of having a number of variables-such as pedagogy, student characteristics, and time-on-task – affect academic achievement or student satisfaction, not just the technology used to provide the education at a distance. Of course, random selection doesn’t guarantee that the control and experimental groups are similar in all attributes, but it is accepted practice in good research.

3. The validity and reliability of the instruments used to measure student outcomes and attitudes are questionable. An important component of good educational research relates to proper measurement of learning outcomes and/or student attitudes. In short, do the measurement instruments – the final examinations, quizzes, questionnaires, or attitude scales developed by the teacher – measure what they are supposed to measure? A well-conducted study would include evidence of the validity and reliability of the measurement instruments so that the reader could have confidence in the results. But in almost all of the studies reviewed, this information was lacking.

4. Many studies do not adequately control for the feelings and attitudes of the students and faculty – what the educational research refers to as “reactive effects.” Reactive effects are a number of factors associated with the way in which a study is conducted and the feelings and attitudes of the students involved. One reactive effect, the “Novelty Effect,” refers to increased interest, motivation, or participation on the part of students simply because they are doing something different, not necessarily better. Another, called the “John Henry Effect,” refers to control groups or their teachers who feel threatened or challenged by being in competition with a new program or approach and, as a result, outdo themselves and perform well beyond what would normally be expected. In many studies, precautions were not taken in the research to guard against these effects.

GAPS IN THE RESEARCH

Notwithstanding the fact that the overall quality of the research needs improvement, there are several important issues regarding the effectiveness of distance learning that require further investigation and information. These gaps must be filled so that public policy discussions can be based on accurate and adequate information. Specific issues include the following.

1. The research has tended to emphasize student outcomes for individual courses rather than for total academic programs. A major gap in the research is the lack of studies dedicated to measuring the effectiveness of total academic programs taught using distance learning. Virtually all of the comparative or descriptive studies focus upon individual courses. This raises serious questions about whether a total academic program delivered by technology compares favorably with a program provided on campus. In addition to cognitive skills and verbal, quantitative, and subject-matter competence, outcomes with regard to critical-thinking skills, attitudes and values, moral development, and so on need to be addressed. This is especially important since public policy is typically aimed at providing access to degrees or programs of study, not just single courses.

2. The research does not take into account differences among students. A substantial portion of research on distance learning has been conducted to demonstrate no significant difference in achievement levels between groups of distance and traditional learners. However, there is wide variance of achievement and attitudes within the groups, which indicates that learners have a variety of different characteristics. The factors influencing these differences could include gender, age, educational experience, motivation, and others. Gathering samples of students and amalgamating them into averages produces an illusory “typical learner,” which masks the enormous variability of the student population. Further research needs to focus on how individuals learn, rather than on how groups learn.

3. The research does not adequately explain why the course dropout rates of distance learners are higher. A number of studies showed that higher percentages of students who participated in distance-learning courses dropped out before the courses were completed compared to students in conventional classrooms. The issue of student persistence is troubling both because of the negative consequences associated with dropping out and because the research could be excluding these dropouts – thereby tilting the student outcome findings toward those who are “successful.”

4. The research does not take into consideration how the different learning styles of students relate to the use of particular technologies. Our understanding of how the learner, the learning task, and a particular technology interact is limited. Learner characteristics are a major factor in the achievement and satisfaction levels of students participating in distance education. Information regarding a student’s preferred learning style will influence how the course is designed and what type of technology is used. Additional research could result in more information regarding why different technologies might be better suited for specific learning tasks.

5. The research focuses mostly on the impact of individual technologies rather than on the interaction of multiple technologies. Much of the literature on distance learning focuses on one technology and either describes its effectiveness Or compares it to the conventional classroom experience (or does both). Most technologies, however, are multifunctional and can be adapted to address a wide range of learning outcomes. Unfortunately, there are few studies that examine more than one technology – or the synergistic effects of certain technologies-in addressing specific educational outcomes and student groups. The few studies that are available do not provide ample grounds for generalization because of a range of limitations, including small sample sizes and lack of sufficient explanation of the instructional treatment.

6. The research does not include a theoretical or conceptual framework. There is a vital need to develop a more integrated, coherent, and sophisticated program of research on distance learning that is based on theory. Theory allows researchers to build on the work of others and, thereby, to increase the probability of addressing the more significant questions regarding distance learning. Using theory as a guiding framework also allows the research to be replicated and enhances its generalizability, thereby making individual studies more meaningful.

7. The research does not adequately address the effectiveness of digital “libraries.” Students participating in distance learning, particularly those in remote locations, are often introduced to a digital “library” that provides access to bibliographies, as well as to the full text of a variety of resources. The library is at the core of the higher education experience and, particularly at the graduate level, is an integral part of the teaching/learning process. Some digital libraries boast an enormous array of resources, with the implicit notion that they can provide the same service as the traditional library. But do digital libraries provide adequate services for the academic programs they are established to support? Anecdotal evidence seems to suggest that the curricular objectives of some distance-learning courses have been altered because of a limited variety of books, journals, and other resources available online.

IMPLICATIONS

Research on distance learning has been driven by what many are calling the “information revolution.” Advancements in technology offer both the general public, and faculty in particular, a dizzying array of unprecedented challenges. Technology is having, and will continue to have, a profound impact on colleges and universities in America and around the globe.

Distance learning, which was once a poor and often unwelcome stepchild within the academic community, is becoming increasingly more visible as a part of the higher education family. But the research and literature reviewed for this paper indicate that the higher education community has a lot to learn about how, and in what ways, technology can enhance the teaching/learning process, particularly at a distance.

As with other educational innovations that have come before it, there is some danger that the innovations made possible through distance education are advancing more rapidly than our understanding of its practical uses. Princeton historian Robert Darnton makes this point in “The New Age of the Book,” an essay in the March 18, 1999 New York Review of Books about a similar realm: electronic publishing. Darnton observes that, since its inception, electronic publishing has passed through three stages: “an initial phase of utopian enthusiasm, a period of disillusionment, and a new tendency toward pragmatism.”

In the context of the research on distance learning and its effectiveness, more emphasis has been placed on the “utopian” possibilities of the technology and its potential to do as well as classroom-based instruction. But not enough “pragmatism” has been applied to allow for a discussion of distance learning’ s practical implications as a supplement to enhance teaching and learning.

There are at least three broad implications that can be derived from our review of the original research and the other literature. The first is that the notion of “access to college” in the distance-learning context is unclear. Many of the advocates of distance learning tout access to college-level education as a raison d’etre for the proliferation of distance education. Indeed, in some states public policy leaders are recommending using distance education in lieu of “bricks and mortar” learning.

Of particular concern is access as it relates to the efficacy of computer-mediated learning. Unlike two-way interactive video, where students and the instructor can see and talk to each other in a conventional classroom, computer-mediated learning requires special skills of students and more sophisticated technical support if students are to interact fully. Questions that need to be asked include: What is the “quality” of the access? Does the student have the necessary skills to use the technology? What are the best ways to participate in asynchronous communication? Is there adequate technical support? And perhaps most important: Will the cost of purchasing a computer and maintaining software be prohibitive for a substantial number of students?

Second, it seems clear that technology cannot replace the human factor in higher education. Faculty involved in distance education find themselves acting as a combination of content experts, learning process design experts, process implementation managers, motivators, mentors, and interpreters. In short, technology “can leverage faculty time, but it cannot replace most human contact without significant quality losses,” as William Massy has stated.

Third, although the ostensible purpose of much of the research is to ascertain how technology affects student learning and student satisfaction, many of the results seem to indicate that technology is not nearly as important as other factors, such as learning tasks, learner characteristics, student motivation, and the instructor. The irony is that most of the research on technology ends up addressing an activity that is fundamental to the academy, namely pedagogy – the art of teaching. To that extent, the research has had a salutary effect in that a rising tide lifts all boats. Any discussion about enhancing the teaching/learning process through technology also has the beneficial effect of improving how students are taught on campus.

Consider this example. In 1987, the American Association for Higher Education published “Seven Principles for Good Practice in Undergraduate Education,” which distilled findings from the research on the undergraduate experience. The principles were revived in 1996 by Arthur Chickering and Stephen Ehrmann to enable those using new communication and information technologies to enhance the teaching/learning process. In one form or another, the principles have been incorporated in a variety of publications on good practice, and were evident in many of the studies.

AAHE’s principles of good practice include those methods that

* encourage contacts between students and faculty;

* develop reciprocity and cooperation among students;

* use active learning techniques;

* give prompt feedback;

* emphasize time-on-task;

* communicate high expectations; and

* respect diverse talents and ways of learning.

In a sense, the discussion has come full circle. The research on distance learning has a long way to go, and much of it is inconclusive. On the other hand, technology has helped the academy to continue its focus on the essential goals of teaching and learning. As a result, either implicitly or explicitly, the key question that needs to be asked is: What is the best way to teach students?

This article is based on the report What’s the Difference? A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education, which was published in April by The Institute for Higher Education Policy and was sponsored by the American Federation of Teachers and the National Education Association. A list of the studies referred to in this article can be found on page 37 of the report.

Jamie P. Merisotis is President of The Institute for Higher Education Policy in Washington, DC. Ronald A. Phipps is Senior Associate at The Institute.

COPYRIGHT 1999 Heldref Publications

COPYRIGHT 2000 Gale Group