Measuring a Government's Capacity to Fight Terrorism
By: Paul Shemella
Along the path to any strategic end, leaders are obliged to ask whether that path is leading them to the outcome they seek. The answer to this question may not always be clear. Measuring the effectiveness of any strategy is difficult. As reflected in the growing body of literature on measurement, the challenge is not measuring effort but results.1 Measuring effectiveness, especially in fighting terrorism, can only be based on how we evaluate the results of our efforts, particularly at the political level. The selection, evaluation, and integration of ‘measures of effectiveness,' or MOE, has become a vital component of strategy development and execution.
But what if we wanted to measure a government's overall capacity to fight terrorism? We would have to look one level higher than strategy; we would have to examine a government's structures and processes for developing strategy in the first place. Without capacity, good strategy cannot be formulated or executed; without good strategy, capacity is largely wasted. And what if a government has succeeded in preventing terrorist attacks altogether? Could its leaders claim they have done everything right, or have they just been lucky? There must be a way to measure readiness for a terrorist event that has not yet happened, especially when the worst-case scenario is so severe. Governments must find a methodology that relies on judgment, not merely on numbers. Such a methodology would give leaders confidence that their policies are minimizing the probability their citizens will be attacked - whether or not terrorist attacks have actually taken place.
Capacity to minimize the odds of terrorist attacks taking place at all cannot be separated from capacity to minimize the damage from terrorist attacks that have already occurred. In other words, governments need a system to evaluate their performance in a variety of functions related to terrorism. This article proposes evaluating a government's capacity to execute a spectrum of four basic functions: Strategy, Institutional Preparation, Intelligence, and Emergency Management. The premise here is that if a government can do all these things well, its citizens can feel relatively safe – and its leaders can claim they have done the best they can against a very difficult threat. But how do they turn their best judgment into numbers that can be analyzed?
Even when the result of an intangible like prevention cannot be measured directly, the processes and systems that lead to the desired outcome can be.2 The place to begin is by identifying ‘desired outcomes,' perhaps stated as "what are the significant achievements we would like to be able to report to our citizens?" We can then list a set of preconditions that must be in place in order to achieve the desired outcomes. Take the example of a hypothetical country with the usual set of root causes for terrorism. Ameliorating root causes will certainly reduce the likelihood of terrorist attacks. One desired outcome might be stated as follows:
"We have eliminated the root causes of terrorism in the country."
This statement cannot be substantiated with raw numbers, but there is a process for converting the government's broad approaches into numbers that can be evaluated. This is indirect measurement, or what could be called ‘institutionalizing measures of effort.' A vital interim step in such a process is the identification of ‘preconditions' that we think will enable us to reach the desired outcome. In this case, the list of preconditions could include the following hopeful statements:
- "There is a process for inter-ethnic dialogue throughout the country."
- "We have strengthened the justice system."
- "We have eliminated corruption from law enforcement institutions."
- "We guarantee universal access to basic health care."
- "We have ensured that every citizen can receive a basic level of education."
- "There is a system for reviewing complaints against the government."
- "There is a lively, open, and responsible press establishment."
- "The armed forces provide support to civilian law enforcement authorities but do not themselves enforce the law."
For the purposes of this example, we can assume that these statements have been formulated over time by a panel of experts in our hypothetical government. The preconditions for getting to our desired outcome can now be evaluated on a scale of 0-5 by another (independent) panel of experts.3 The experts will assign a "0" to statements for which nothing has been done, a "5" to statements where everything has been done, or interim values to reflect partial accomplishment. At the end of this process, the numbers can be aggregated to determine the ratio of the actual score to the maximum possible ("best") score. For eight preconditions, the maximum score is 40. If the actual total comes to 20, it can be said that the government has made progress but probably not enough. This ratio is not very useful by itself, but when compared to other desired outcomes, it may show where the government needs to invest resources, as well as where its previous investments have paid off.
What is more, if the experts score each statement individually rather than by consensus, it will be possible to find an aggregate score for each individual precondition. This can give a more detailed picture of where the most improvement is needed, and how it can be achieved. For instance, weighing the judgment of ten experts means that 50 is the best possible score for each precondition. "We have strengthened the justice system" might get an aggregate score of 20, while "We have eliminated corruption from law enforcement institutions" might score 43. It may then be argued that some methods for bringing police corruption under control might be adapted to strengthening the justice system. In a logical and methodical way, we can transform judgment into numbers, and then compare those numbers to one another for a more thorough analysis.
Reaching into the Toolkit
Every government has an array of tools it can use to develop and execute strategies for all aspects of fighting terrorism. Democracies have a wider variety of tools than authoritarian forms of government, one reason they tend to be more successful in the long run.4 The list of tools, or ‘instruments of national power' as they are sometimes called, might include the following:
| || |
| || |
| || |
| || |
| || |
Governments express these instruments of power through institutions, and it is institutional capacity on which our focus will fall. Individuals cannot produce policies, strategies, and operational plans on their own. They must act together in various teams, each with a clearly defined role, promulgated by political authorities. The fulfillment of these roles should drive funding levels, equipment procurement, and most of all personnel requirements. Smart and motivated individuals are assigned to institutions, improve them as much as they can while there, and then move on to other institutions or the private sector - but it is institutions that produce national security over time.
Institutional roles, however, must be complementary; no one institution can, or should, do everything. Sound national security decision-making relies on the specialization and diversity of views that a balanced set of institutions provides. If one institution goes beyond its mandated role, seeking roles and resources that should go to other institutions, that balance is altered in surprisingly complex ways. Greedy institutions, like greedy individuals, are bad for collective effort. Ministers, directors, and secretaries everywhere must remember that the only institution that really matters is the government that each of them serves.
Institutions act as crucibles for the development of capability and capacity. Without strong and clean institutions, no strategy can be executed and no success against terrorism ever achieved. But capability is not the same as capacity. Capability can be demonstrated once or twice (especially to superiors) but will not by itself produce measurable results. Capacity, however, requires enough resources to execute essential capabilities day after day, year after year. 8 When we talk about the role of institutions in fighting terrorism, we are really talking about institutional capacity. How can we measure how well a government does that?
The indirect measurement technique already described can be used to measure a government's institutional capacity to fight terrorism. This methodology is essentially a framework for self-assessment. The framework assesses capacity in four functional areas: Strategy, Institutional Preparation, Intelligence, and Emergency Management.9 In order to explain the methodology, let us take a sample from the "strategy" functional area:
Strategy. We can list six "desired outcomes" for the strategy function.
- Appropriate government institutions have clear roles in combating terrorism.
- There is a process for coordinating strategy development among government institutions.
- There is a process for developing an accurate and comprehensive strategic analysis.
- There is a legal framework for developing responses to terrorism.
- There is a method for measuring the effectiveness of strategies to combat terrorism.
- There is a political-level strategy for combating terrorism.10
For each desired outcome above, we can list a set of preconditions that will lead a government to that outcome. To use just one example, the preconditions for desired outcome #1 should read something like this:
- The Ministry of Interior, Ministry of Defense, Ministry of Foreign Affairs, and intelligence agencies have written guidance on their responsibilities for combating terrorism.
- Other ministries have written guidance consistent with the government's legal definition of terrorism.
- Institutional roles are not in conflict with each other.
- There are no gaps between institutions in terms of responsibility for combating terrorism.
Applying the Likert Scale, we can assign a number to each precondition, based on the judgment of five experts, selected for the strategy assessment. Those experts can be officials from the government being assessed, terrorism specialists from the private sector, or they can even be from another government with more experience in fighting terrorism. A hypothetical set of values assigned by them might look something like this:
- 2 out of 5
- 4 out of 5
- 2 out of 5
- 2 out of 5.
The total number 10 (out of a possible 20) does not tell us much by itself. The aggregation of totals for each of the six desired outcomes (and their associated preconditions) would give us a total for the strategy function (as well as a comprehensive comparison within the field of preconditions).11 The final number for "strategy" can then be compared to total values from the other three functional areas to indicate relative institutional capacity – and where the most resources should be invested in the future. All Desired Outcomes for the Strategy functional area are listed, with suggested preconditions, can be found in Figure 1 at http://www.ccmr.org/public/library_file_proxy.cfm/lid/5701
Institutional Preparation. The second functional area to be examined is "institutional preparation." The desired outcomes and preconditions for this category are broken into four pieces: operational-level strategies, operational plans, manpower development programs, and infrastructure support. The desired outcomes can be stated as follows.
- Each government institution with a CT role has a current and coordinated operational-level strategy that incorporates measures of effectiveness.
- Each CT institution has a current and coordinated set of operational-level plans.
- Each CT institution has a comprehensive manpower development program.
- Each CT institution has adequate infrastructure to support its expected missions, or it has identified shortfalls.
What do we mean by a "CT Institution?" Any government organization with specific responsibility for preventing or dealing with terrorism should be included under this rubric. That means almost all institutions in nearly all governments. Fighting terrorism can only succeed when a spectrum of institutions, each with clear roles, work together in a systematic way (see the "strategy" module above). Desired outcomes and preconditions for the "institutional preparation" module can be found in Figure 2 at http://www.ccmr.org/public/library_file_proxy.cfm/lid/5701
Intelligence. The third functional area is "intelligence." There is nothing more important in the crafting of strategy, or in the execution of operations, than having timely and accurate intelligence. The process for ensuring that intelligence is accurate requires extensive coordination among government institutions, and perhaps multiple intelligence agencies. If undertaken seriously, this process can serve as an example of how the rest of the government reaches workable interagency decisions. Desired outcomes and preconditions for the "intelligence" module can be found in Figure 3 at http://www.ccmr.org/public/library_file_proxy.cfm/lid/5701
Emergency Management. The capacity for a government to recover from the effects of a terrorist attack can act as a deterrent to further attacks. A government cannot be evaluated as completely prepared for the threat of terrorism unless it has a deep capacity for emergency management (sometimes called ‘consequence management'). Fortunately for under-resourced governments, the same institutions and processes used for responding to natural disasters can be drawn upon in the aftermath of a terrorist attack. All-hazards approaches to risk assessment can lead directly to responsible dual usage. Desired outcomes and preconditions for the "Emergency Management" module can be found in Figure 4 at http://www.ccmr.org/public/library_file_proxy.cfm/lid/5701
Totals for desired outcomes and preconditions in all four functional areas can be aggregated as shown in Figure 5 (that can be found at http://www.ccmr.org/public/library_file_proxy.cfm/lid/5701) and analyzed in comparison to one another. Numbers, drawn from judgment, can tell us quite a lot about where a government has been doing well and where it has not.
Back to Root Causes
The long-term effort to eliminate the root causes of terrorism has been listed as a precondition for achieving a comprehensive political-level strategy.12 It may well be that a government wishes to separate this function from the four we have listed, building a fifth functional area on which to apply the indirect measurement technique (indeed, this article has identified preconditions that could be used as a starting point). For countries with a surfeit of root causes, this might make perfect sense, yielding greater resolution of the problem. Certainly, the government of New Zealand would approach this challenge differently than the governments of India or Israel. Root causes operate across the full spectrum of a society. Identifying and mitigating them will reduce the probability that extremist elements in that society will gain the support they need for a sustained terrorism campaign. The long- term effort to eliminate the root causes of terrorism is most often a strategy for better governance. That is the principal goal of any government and the fervent wish of all citizens.13
All Governance Is Local
What has been introduced here is a method, not a recipe. It could be called ‘Wikipedia' for self-assessment. Every government needs a way to measure its capacity to fight terrorism successfully; and each one could modify the framework according to what makes sense within its own local context. Fighting terrorism is all about context. Certain concepts and principles are universal, but their application can be quite different from society to society. The framework described here should fall within the universal sphere, but individual governments must modify the framework to suit their own historical, economic, cultural, and political conditions. Properly used, it allows government officials to identify their own ‘capacity gaps' and develop a plan to fill them.
Capacity gaps are the raw material for further analysis and concrete actions. But how are they made to benefit the policeman on the street, the medic in the ambulance, or the solider in the field? Government officials must take the capacity gaps they find at the national level and send scarce resources where they are needed most - to those in the field who actually fight terrorism and its effects on society. They must also require leaders all the way down to develop and coordinate operational plans, as well as the supporting tasks needed to confront terrorists where they work. A government that does this can be said to be governing well. In the final analysis, a path toward good governance is the key to fighting terrorism successfully. Measuring capacity is the essential first step on that path.
Paul Shemella is a retired Navy Captain who served as a SEAL officer until 1997. He joined The Center for Civil-Military Relations (CCMR) in 1998. Through a network of theoreticians and practitioners, he manages 'Combating Terrorism' education programs for international officials in all regions.
1 David Kilcullen has done the best work on measuring effectiveness for irregular conflict. See his unpublished essay ‘Measuring Progress in Afghanistan,' December 2009, found on the web. See also Paul Shemella, et al, Fighting Back: What Governments Can Do About Terrorism (Palo Alto: Stanford University Press, 2011), 246-261.
2 Glenn Woodbury, "Measuring Prevention," Homeland Security Affairs 1, no. 1 (Summer 2005).
4 But democracies are more vulnerable to political violence in the short run, prompting government officials to balance liberty with security.
5 The proficiency to manage the consequences of terrorist attacks that cannot be prevented is now just as much an instrument of national power as any of the traditional "DIMEFIL" tools. Known resilience of critical infrastructure, for instance, can deter terrorist attack.
6 Citizens, acting together, can be the most potent tool a society has to address the problem of terrorism.
7 In the fight against terrorism, governments that learn how to harness moral power stand well ahead of those that do not.
8 Competence is another term often equated with capability. Normally obtained through training, competence describes the ability to produce a measureable result. It is useful to think of building capacity as an orchestrated sequence of creating competence, capability, and capacity.
9 The framework for assessing CT capacity was developed by a team from ‘The Center for Civil-Military Relations' (CCMR) consisting of Paul Shemella, Lawrence E. Cline, Edward E. Hoffer, James Petroni, and Matthew King. CCMR is an arm of the Naval Postgraduate School in Monterey, California.
10 Operational-level strategies, which would include the security services, are listed as a desired outcome in the ‘Institutional Preparation' module at Figure 2.
11 The Strategy functional area in this methodology refers to the political level. Without a political-level strategy, individual institutions will not be able to develop operational-level strategies that can be executed in a coordinated manner.
12 There are really three basic strategies for any government to employ against terrorism. The first of these should be targeted at root causes; the others focus on offensive and defense measures.
13 Terrorism is a transnational event that can threaten even the best-governed societies, but good governance – and the trust between the government and its citizens thus created - is the best way to minimize that threat.