Public Policy and Program Evaluation – Review
Jennifer C. Greene
Evert Vedung. New Brunswick, NJ: Transaction, 1997. 336 pp. $34.95.
In Public Policy and Program Evaluation, Swedish political scientist Evert Vedung positions program evaluation as a highly significant, even essential tool of Western democratic decision making in the public domain. This introductory-level text is intended for students of both evaluation and public administration. It joins social science with political science by linking evaluation concepts and processes to theories and frameworks of public administration, including, for example, systems perspectives and theories of organizational change. Vedung highlights the strong “results orientation” of contemporary public decision making, from local municipalities to federal bureaucracies, and offers a practical, decision-oriented approach to evaluation designed to assist public officials in meeting these contemporary demands for “results.” Vedung equally emphasizes the politicized and democratic contexts of contemporary public decision making in Western societies, and his practical conceptualization of evaluation is tempered by such political realities as interest-group influence and by such democratic ideals as pluralism.
Vedung presents his conceptualization of evaluation in the service of public decision making in 15 chapters, supplemented by a glossary of evaluation concepts. The 15 chapters of the text can be divided into two principal arguments. In the first four chapters, Vedung offers his definition of evaluation and his view of the field. He defines evaluation as the “careful retrospective assessment of the merit, worth, and value of administration, output, and outcome of government interventions, which is intended to play a role in future, practical action situations” (p. 3). With this definition, he focuses his view of evaluation on retrospective assessments of public program implementation and outcomes, leaving critiques of originating policies to other fields. This definition also highlights evaluation as a practical and instrumental activity and underscores the pivotal role of valuing in evaluation. Throughout the text, Vedung discusses the valuing part of evaluation and thoughtfully acknowledges the challenges – both technical and political – of identifying appropriate criteria and standards on which to judge the program under study.
Vedung offers his view of the field of program evaluation in a fairly lengthy chapter 4. He differentiates evaluation models by their organizers or basic questions (p. 35). He offers three clusters of models: effectiveness, efficiency, and professional models. Effectiveness models address the results of interventions and can be organized around stated program goals, side effects, observed results, system components, or client or stakeholder concerns. Economic models focus on costs and benefits and can address productivity or efficiency. The lone professional model presented is peer review.
Chapter 5 introduces the substantive core of Vedung’s own view of evaluation, presented as the “Eight Problems Approach to Public Policy Evaluation.” Vedung appropriately avers that it is the problems or questions addressed, not the designs or methods, that provide identity to evaluation. These eight problems, which frame the remainder of the book, are (1) identifying the evaluation purpose, (2) selecting the evaluator, (3) analyzing the intervention, (4) describing the program as implemented, (5) determining the program’s results, (6) explaining these results, (7) judging the program with identified criteria and standards of performance, and (8) using the evaluation in practical action situations.
In Vedung’s detailed discussions of these problems, he attends especially to the problem of explanation or impact evaluation, which he views as “important, particularly for higher level decision-makers in national bodies” (p. 166). Through four separate chapters (and 90 pages), he discusses both conventional and alternative approaches to impact assessment. His critique of the randomized experiment as the premier model for public program evaluation, especially in Sweden, is balanced and persuasive. His comprehensive discussion of “process evaluation” as an alternative analytic framework for attributing results to interventions is a unique contribution of this book. Vedung’s process evaluation is not an assessment of program implementation (often called process or implementation evaluation in the U.S.) nor a scientific explanation of how intended activities led to observed results (as in theory-oriented or theory-driven evaluation). Rather, Vedung presents process evaluation as a comprehensive analysis of the program in its historical, political, economic, and social context that seeks a “whole pattern of causal interdependencies” (p. 210). To guide process evaluators, Vedung offers six broad factors that include the policy history of the intervention, its implementation at multiple levels (from agency to street-level bureaucrat to beneficiary), and its location in networks of issues and influence. Vedung’s championship of process evaluation exemplifies his beliefs that “evaluation researchers must . . . learn to live with the insight that public policymaking can never be transformed into science in the way desired by outright technocrats and radical experimentalists” (p. 192). More profoundly, “the demand for rational, social science like evaluation must be subordinated to the requirements of a democratic body politic” (p. 288).
Vedung succeeds in this book in presenting evaluation as a service to public democratic decision making. He offers public officials a well-articulated view of what evaluation can do for them. His presentation includes strong and appropriate attention to the place of stated program goals in evaluation, to the pluralism often warranted in judging program merit and worth, and to the essential role of utilization in practical evaluation. His discussion of strengths and weaknesses of stated program goals as the primary organizer for evaluation makes an important argument not often heard amidst the widespread rejection of goal-oriented evaluation models. Vedung maintains that goals are “publicly and officially adopted in political assemblies by the representatives of the people” (p. 41) and thereby hold special status as institutionalized, collective goals of the state. “Citizens and elected officials have legitimate reasons to ascertain whether policy goals have in fact materialized in the field” (p. 90), and accountability to citizens is essential in a representative democracy. The public administration lens integrated into this book also generates several creative ideas, including the suggestion to describe the program being evaluated through the use of policy instruments (chap. 8) and the portrayal of monitoring as not checking only for compliance but rather as a comprehensive assessment of program delivery and coverage, “where the government meets society, as it were” (p. 137; chap. 9).
Vedung presents evaluation in the service of public decision making, but not as a critic of public affairs, policies, or programs. His ideas are thus more closely aligned with those of U.S. evaluation theorists Joseph Wholey and Eleanor Chelimsky than those of U.S. theorist Ernest House. House has long argued that evaluation should serve to democratize public decision making by including a greater plurality and diversity of voices and perspectives and by engendering decisions that move toward greater justice and equity in society. This critical perspective is all but absent from Vedung’s evaluation approach. I would have also been enlightened by more attention to the contrasts between the Swedish and the U.S. democratic contexts for evaluation. I benefited from the detailed Swedish examples in the text and wanted more analysis of how they uniquely, or not, define democratic decision making in Sweden. I further wanted some additional connections between the various frameworks presented in the book, especially between Vedung’s view of extant evaluation models (chap. 4) and his “eight problems approach” (chaps. 5 and 15). Finally, I was occasionally distracted but more often entertained by the unusual English usages sprinkled throughout what is otherwise a clearly written manuscript. Through this text, beginning students, especially of public administration, will gain a clear, well-reasoned understanding of how program evaluation can facilitate their work. Other texts will be needed to ensure a critical voice.
Jennifer C. Greene Department of Policy Analysis and Management College of Human Ecology Cornell University Ithaca, NY 14853
COPYRIGHT 1999 Cornell University, Johnson Graduate School
COPYRIGHT 2000 Gale Group
