Measures of effectiveness in the information environment
David C. Grohoski
Assessing the effectiveness of an information operation is one of the greatest challenges facing a staff. Despite the evolution of information operations (IO) doctrine and the refinement of supporting tactics, techniques, and procedures (TTPs), the Army has not solved the problem of IO assessment methodology. The question remains: lacking physical evidence, how can we quantify the intangible attributes of the information environment to assess the effectiveness of IO?
This article addresses the matter of quantifying the effectiveness of IO. Then it presents a methodology for developing measures of effectiveness (MOEs) to ascertain IO effects on friendly and enemy forces.
Statement of the Problem
Why is assessing the impact of IO so difficult? First, the information environment is an abstract construct, and IO operates within that construct. According to Joint Publication 1-02, Department of Defense Dictionary of Military and Associated Terms, the “information environment” is the aggregate of individuals, organizations, or systems that collect, process, or disseminate information. Also included in the information environment is information itself. Thus, the information environment is a combination of physical assets (e.g., information systems) and nonphysical concepts (e.g., information, information-based processes, and human decisionmaking processes). IO attacks and protects the physical assets of information systems to affect the nonphysical aspects of the information environment. Transitioning from visible effects resulting from destruction of tangible assets such as command posts (CPs) and radar systems to abstract effects such as disrupted information flow and degraded decisionmaking is a challenging task.
Second, not all of IO’s capabilities reside in the physical world. While physical destruction is tangible enough, many capabilities include nonphysical aspects–operations security (OPSEC), electronic warfare (EW), military deception, psychological operations (PSYOPs), etc. For IO purposes, the effects ultimately produced by these elements should occur in the intangible domain of ideas, perceptions, and attitudes. Capturing data or information to measure such nonphysical effects is difficult and often time-consuming, requiring a depth of analysis that seems impossible during high-tempo operations.
Third, an integrated IO campaign achieves a complex, tiered hierarchy of effects (see Figure 1). The attack on or protection of physical assets (information systems) yields what one can call “first-order effects,” such as the destruction, degradation, and disruption of enemy signal nodes and CPs, or perhaps the presentation of false observables for collection by enemy intelligence systems. We direct these first-order activities against the enemy’s information system to achieve second-order effects on the enemy’s information and information-based processes, which in turn, seek a third-order effect on the enemy commander’s decisionmaking (i.e., the ultimate target of IO). Defensively, first-order effects may be the protection of friendly force information system assets; second-order effects may be the maintenance of situational awareness or an uninterrupted information flow; and third-order effects may be the preservation of effective decisionmaking. Each level of effects will likely yield corresponding enemy and friendly reactions, resulting in a complex, tiered set of causes and effects, which require interpretation to determine the overall impact of the IO. This maze of causal relationships requires something more than traditional battle damage assessment (BDA).
[FIGURE 1 OMITTED]
Assessment and the Hierarchy of Effects
Measurement and analysis of effects resulting from the attack of enemy information systems and protection of friendly information systems make an IO campaign assessment possible. However, to do this, it is necessary to understand the hierarchy of effects resulting from IO activities (e.g., first-, second-, and third-order effects).
First-order effects result from those actions directed against the enemy’s information system and those measures taken to protect the friendly information system. Generally, one deduces first-order effects by using information derived from unit reporting and BDA. (1) This assessment determines if planned IO tasks have occurred, and the direct result of these actions and activities. Generation of second- and third-order effects are by the aggregate of actions directed against enemy and friendly information systems. These effects are subtle and less quantifiable than first-order effects. At these levels, assessment seeks to determine if the aggregate of executed IO tasks have achieved the desired result:
 What were the effects on the enemy and friendly information systems (second-order effects)?
 Were the enemy and friendly commanders affected (third-order effects), and if so, how and to what extent?
Determination of second- and third-order effects is usually through inductive analysis of intelligence reporting and assessments.
Establishing Cause and Effect
The only way to assess cause-and-effect linkages is to acknowledge–
 Military conflict consists of interactions among humans and technologies.
 Linkage of physical assets of a military force and the intangible aspects of military operations, such as morale, leadership, will, and cohesion.
Thus, attacking physical assets–CPs, target acquisition systems, intelligence collection and processing systems, and communications systems–will adversely impact a military force’s ability to make and act upon decisions and consequently will have a detrimental affect on those intangibles that provide the military force with the ability to conduct operations.
Correlation exists when the value of an action (e.g., number of occurrences, degree of the effect, etc.) increases (or decreases) while the value of the effect also increases (or decreases). For example, if as the number of PSYOP leaflets dropped on enemy formations increases so do the number of enemy soldiers surrendering, or if as the number of jamming attacks against a command and control (C2) net increases, the traffic on that net decreases, then a probable correlation exists. This deductive reasoning forms the basis of determining first-order effects.
However, the apparent relationship between action (cause) and effects may be coincidental because the occurrence of an effect is accidental, or perhaps caused by the correlation of multiple actions executed to achieve the effect. For example, if friendly forces are successfully engaging enemy formations with fires and maneuver at the same time PSYOP activities are urging enemy soldiers to surrender, then correlating an increase in surrendering soldiers to PSYOP activities alone may not be possible. Furthermore, because an IO will often employ multiple elements to engage the enemy’s information system, the cumulative effect of IO support to friendly combat actions may make the impact of individual IO activities indistinguishable. Since there will rarely be enough time to rule out definitively coincidental relationships, the only possible antidote is an in-depth knowledge of the enemy and information environment that facilitates the development of an informed estimate through inductive reasoning.
What Are Measures of Effectiveness?
Unfortunately, the emerging joint definition of “MOEs” (e.g., tools used to measure results achieved in the overall mission and execution of assigned tasks (2)) provides little clarity as to what MOEs and how we can MOEs or them to assess IO. Therefore, for this discussion, we propose the following definition: measures of effectiveness are standards of reference used as the basis of comparison to evaluate the success or progress of an operation.
MOEs are a means to determine second and third-order effects by establishing a cause-and-effect linkage between the usually observable and quantifiable first-order effects and the abstract and subjective second- and third-order effects. MOEs do not constitute the assessment itself but are an evaluation means to determine if the individual IO tasks are achieving the IO objectives and whether accomplishment of the IO objectives is fulfilling the concept of IO support (see Figure 2). (3)
[FIGURE 2 OMITTED]
The staff develops MOEs as part of the planning process to determine the effects of both offensive and defensive IO. To be meaningful, MOEs must link friendly and enemy actions and activities (cause) to enemy and friendly capabilities to make and act upon decisions (effect). (4) Therefore, MOE development begins with the IO mission statement and objectives.
If the staff does not properly craft the IO mission statement and objectives, then developing corresponding MOEs will be difficult, if not impossible. Therefore, the IO mission statement should focus on specific aspects of the operation and not be so general that it merely identifies standard doctrinal requirements for IO. An example of a simplified corps-level IO mission statement is:
On order, II Corps conducts information
operations in the area
of operations in order to disrupt
Northland 1st Army C2 and influence
local leaders and populace,
allowing the destruction of
Northland 1st Army armed forces.
We develop MOEs to assess the desired effect of each IO objective. (5) Ideally, each objective has a clearly defined, attainable effect, otherwise it is not possible to determine if, or when, we achieved that effect, and hence, whether the IO met the objective. (6) Examples of objectives for the previous IO concept statement are–
 Disrupt Northland 1st Army’s capability to integrate air defense early warning and acquisition systems.
 Neutralize Northland reconnaissance assets’ ability to detect the II Corps’ main offensive effort.
 Disrupt the Northland 1st Army commander’s synchronization of corps- and army-level operations.
 Influence local civilian leaders and population groups to not interfere with II Corps’ forces and operations.
As previously noted, MOEs for second-order effects seek to determine if the aggregate of IO actions and activities are accomplishing the IO objectives. If possible, the MOE should be observable to aid intelligence collection, quantifiable to increase objectivity, precise to ensure accuracy, and correlated to the progress of the operation to attain timeliness. While it is possible for an IO objective to have multiple MOEs, intelligence collection and analysis assets are limited. Possible second-order MOEs could be–
 Ninety percent or more of early warning sites and air defense sector control centers suppressed.
 No II Corps deep attacks effectively engaged by radar-guided surface-to-air missiles.
 No II Corps high-value assets attacked by surface-to-surface fires, air strikes, or special purpose forces’ direct action.
 No synchronized fires and maneuver above division level.
 No reserve divisions committed against II Corps’ main effort.
 No blockage of II Corps’ main supply routes by the civilian populace.
 No instances of local leaders inciting the populace to interfere with II Corps’ operations.
MOEs for third-order effects seek to determine if we affected the enemy and friendly commanders as planned. These MOEs should determine if the decisionmaker has responded as predicted. Thus, in all likelihood, these MOEs will be subjective. A possible MOE for third-order effects may be: No counterattack by the Northland 1st Army against the II Corps’ main effort. (7) Figure 3 provides an example of an IO concept, objective, and supporting MOEs.
Assessment–Putting It All Together
MOEs are but one part of the assessment process. Traditional BDA and other intelligence analyses, as well as friendly unit reporting, are still essential to assessing the information operation’s effectiveness. These sources provide the information on quantifiable effects that we can use as the basis for estimates of whether we are achieving the IO objectives and concept of IO support.
Once the staff defines the MOEs, they then develop a mechanism to obtain the information needed to determine the three orders of effects. The IO staff must determine the–
 Assessments required.
 Specific information needed to make the assessments possible.
 Agencies and assets that will provide the information.
This assessment plan then contributes to the command’s intelligence collection plan and friendly forces’ information requirements.
Timely and accurate reporting of information is essential to assessing effectiveness of the IO. Subordinate units report much of this information to their higher headquarters. Maneuver units, tactical PSYOPs teams, and civil affairs tactical support teams, as well as tactical human intelligence (HUMINT) teams (which traditionally include counter-intelligence personnel), and other intelligence collection assets all provide information with which to gauge IO success. Additionally, on-going intelligence analysis, including analysis of media and other open sources, supports assessing whether an IO campaign is achieving its objectives and if the IO concept of support is successful.
To receive information, the IO staff must actively monitor the operational situation and aggressively pursue information through unit reports and debriefings, IO working group (IOWG) meetings, and other venues. Commanders’ battle update briefings, conference calls, and other meetings also facilitate monitoring IO execution by providing forums from which to receive information for subsequent analysis. Some other actions the IO staff can do include–
 Submit requests for information (RFIs) based upon the assessment plan.
 Develop IO input to the commander’s critical information requirements (CCIRs).
 Coordinate with the deep operations coordination cell (DOCC) and targeting board for BDA reporting.
 Review assessments at each IOWG meeting.
 Monitor G2 and G3 incident databases and analyze trends.
Ultimately, an assessment is successful when it is possible to decide when to proceed with the plan, when to reengage a target, when to execute a branch of the plan, or when to execute a sequel. MOEs fit into this effort by facilitating the organization and assessment of the information needed to support these decisions.
Developing MOEs to assess the effectiveness of the information operation is a difficult task. In many respects, MOE creation is much more art than science. However, through development of proper MOEs and an effective assessment plan as discussed in this article, we can link the science involved in achieving and assessing first-order effects to the more subjective assessments of accomplishing second- and third-order effects. Thus, the staff can make an informed estimate of the effects resulting from execution of a command’s IO tasks and establish the progress of the information operation campaign.
Clearly more work is necessary. However, as IO practitioners continue to work with MOEs and other assessment methodologies, they will continue to refine and validate successful techniques and procedures.
Figure 3. Example IO Objective and MOE.
IO Objective 1st Order MOE (BDA) 2nd Order MOE
Disrupt * Destruction of corps * No synchronized
Northland 1st & army headquarters fires and
Army maneuver above
commander’s * Destroyed or captured division level
synchronization reconnaissance teams
of corps- and * No reserve
army-level * Decreased corps- divisions
operations level and above C2 committed against
communications II Corps’ main
* Increased division C2
IO Objective 3rd Order MOE
Disrupt No counterattack by
Northland 1st the Northland 1st
Army Army against II
commander’s Corps’ main effort
of corps- and
(1.) Unit reporting and BDA address the success or failure of planned IO tasks to attack enemy and defend friendly information system assets. This information helps determine which enemy and friendly assets our actions affected and yields an estimate of the immediate results. The purpose of this first-order assessment is to determine if current IO tasks and the level of effort applied to the IO are adequate.
(2.) Joint Publication 3-60, Joint Doctrine for Targeting, 17 January 2002, page GL-8.
(3.) We may develop MOEs to measure the accomplishment of individual IO tasks. Doing so is largely dependent upon the importance of the task, as well as the availability of resources and time to plan and conduct an assessment to that level of detail.
(4.) Murray, William S., “A Will to Measure,” Parameters, Autumn 2001, page 135.
(5.) A well-crafted IO objective specifies an effect, an object of the effect, and a purpose for the effect. Normally, offensive IO objectives are in terms of causing an adversary to do or not do something. Defensive IO objectives are in terms of protecting and defending friendly force’s information and information systems.
(6.) It is important to note that doctrine does not provide specific effects for IO. Typical effects used in the field are deny, destroy, degrade, disrupt, deceive, exploit, and influence. Some of these effects are taken from targeting and therefore have specific definitions, while we use other effects simply because they seem appropriate to IO. Having well-defined effects will certainly assist the planning and development of IO objectives and MOEs. Another noteworthy aspect of IO doctrine is the lack of terms for describing defensive IO effects.
(7.) This MOE assumes that the enemy commander’s predicted decision (as determined by the G2’s intelligence preparation of the battlespace), was either to block or counterattack the II Corps’ ground offensive.
RELATED ARTICLE: Joint C41 staff and operations course.
What is in your future? Are you or will you soon be serving at a corps- or theater-level G2 or G6 staff in support of a joint task force (JTF) or working with a JTF Joint Communications Control Center (JCCC) executing requirements associated with an information management plan? Or are you looking at an assignment to one of the theater signal commands or to an Army Service Component Command headquarters as an active duty or Reserve Component officer or senior noncommissioned officer (NCO) and finding yourself involved in exercise planning conferences or exercises such as Lucky Sentinel, Ulchi Focus Lens, Combined Endeavor, or Grecian Firebolt?
If you are concerned with your present situation, consider the following: Maybe you are now an action officer or senior NCO supporting signals intelligence, space operations, or theater missile defense command and control initiatives. Or are you a U.S. Army Training and Doctrine Command (TRADOC) Systems Manager (TSM) or Project Manager action officer or Department of Defense (DOD) civilian who deals with a myriad of interoperability issues or key performance parameters in the command, control, communications computers, intelligence, surveillance, and reconnaissance (C4ISR) transformation arena?
If any of these situations describe you, then the Joint Command, Control, Communications, Computers, and Intelligence (C4I) Staff and Operations Course (JC4ISOC) stands ready to support your joint C41 educational needs. Sponsored by the Joint Staff/J6, the JC4ISOC is four weeks long and taught seven times during the fiscal year (FY). First established in January 1978 by the Deputy Secretary of Defense as a joint C3 systems course at the Armed Forces Staff College, it is now one of the resident courses under the Joint Command, Control, and Information Warfare School (JCIWS), Joint Forces Staff College (JFSC), in Norfolk, Virginia. The mission of JCIWS is to educate and train company- and intermediate-level military staff officers, senior NCOs, and DOD civilian equivalents in the concepts, applications, and procedures associated with C4I and information operations (IO) in a joint and multinational environment.
To support the warfighter’s needs in a network-centric, capabilities-based force, the JC4ISOC curriculum takes a generalist approach. The program meets the school’s objectives and supports the college’s mission by emphasizing a broad understanding of the joint C4I environment, C2 process, and operation, planning, and management of current joint C4I systems. It provides quality C4I instruction for the joint community on topics such as Joint Vision 2020, joint interoperability, battlespace systems, the Global Information Grid, information assurance, and JTF C4I planning.
Reviews from former students indicate the course’s value to their past, current, and upcoming assignments. An Air Force colonel said, “I would have been 300 percent more effective in the job [I had] if I had attended that JC4ISOC course…..the information was that beneficial, especially that part about the C41 contacts and points of contact!”
Annually, JC4ISOC issues messages to major Army commands (MACOMs) and joint agencies of all services announcing FY course dates and prerequisites. We disseminate a separate message a few weeks before the start of each class. The course accommodates a maximum of 25 students. The next four iterations of the course will be:
 Class 03-7 4-29 August 2003
 Class 04-1 27 October-21 November 2003
 Class 04-2 12 January-6 February 2004
 Class 04-3 1-26 March 2004
Students must possess a Top Secret clearance with sensitive compartmented information (SCI) access and be cleared for SCI indoctrination before arrival. Students’ commands must fund their travel, per diem, and billeting, which includes a five-day field trip to the Washington, D.C., area for “up close and personal” experiences with joint agencies and organizations. Administrative information is available through the “Welcome Aboard” and “General Information” sections of the JFSC web site at https://www.jfsc.ndu.edu/ (click on the JCIWS link).
The JC4ISOC quota control point of contact is Lieutenant Commander Katherine Mayer; you may contact her via E-mail at email@example.com or firstname.lastname@example.org and telephonically at (757) 443-6320 or DSN 646-6320; the Army faculty representative is Lieutenant Colonel Reynold Palaganas at email@example.com and by telephone at (757) 443-6328/6331 or DSN 646-6331.
Lieutenant Colonel (Promotable) David C. Grohoski is currently the Deputy Director of Operations for the U.S. Army 1st Information Operations Command (Land). A career Infantry officer, his previous assignments include Battalion Senior Observer/ Controller at the Joint Readiness Training Center (JRTC), Exchange Officer with the British Army, Executive Officer of the 3d Infantry Regiment (The Old Guard), as well as assignments in light infantry and airborne ranger units. He was a Distinguished Military Graduate of Michigan State University with a Bachelor of Arts degree in Social Science and earned a Master of Public Administration degree from the University of Oklahoma.
Major Steven Seybert (U.S. Army, Retired) has provided contract support for more than six years to the U.S. Army 1st IO Command (Land) in planning and conducting IO. He has served with IO field support teams on deployments to various levels of military command from joint task force to division. He performed as an IO targeting officer while deployed to Operation JOINT GUARDIAN and as an IO planner during Operation IRAQI FREEDOM. He is a graduate of the U.S. Military Academy at West Point, New York, and the Command and General Staff Course. Readers may contact him via E-mail at firstname.lastname@example.org.
Major Marc Romanych (U.S. Army, Retired) is a former Air Defense Artillery Officer. He works as a contractor with the U.S. Army 1st IO Command (Land). Since 1998, he has deployed with IO field support teams to Bosnia-Herzegovina, Kosovo, and numerous joint and Army warfighter exercises. Mr. Romanych teaches two courses on IO for American Military University. He holds degrees in Chemistry, Geology, History, and International Relations. Readers may contact him via E-mail at email@example.com.
COPYRIGHT 2003 U.S. Army Intelligence Center and School
COPYRIGHT 2004 Gale Group