Consulting

There is a post at Politico about the adventures of McKinsey & Company, a giant consulting firm, in the world of intelligence:

America’s vast spying apparatus was built around a Cold War world of dead drops and double agents. Today, that world has fractured and migrated online, with hackers and rogue terrorist cells, leaving intelligence operatives scrambling to keep up.

So intelligence agencies did what countless other government offices have done: They brought in a consultant. For the past four years, the powerhouse firm McKinsey and Co., has helped restructure the country’s spying bureaucracy, aiming to improve response time and smooth communication.

Instead, according to nearly a dozen current and former officials who either witnessed the restructuring firsthand or are familiar with the project, the multimillion dollar overhaul has left many within the country’s intelligence agencies demoralized and less effective.

These insiders said the efforts have hindered decision-making at key agencies — including the CIA, National Security Agency and the Office of the Director of National Intelligence.

They said McKinsey helped complicate a well-established linear chain of command, slowing down projects and turnaround time, and applied cookie-cutter solutions to agencies with unique cultures. In the process, numerous employees have become dismayed, saying the efforts have at best been a waste of money and, at worst, made their jobs more difficult. It’s unclear how much McKinsey was paid in that stretch, but according to news reports and people familiar with the effort, the total exceeded $10 million.

Consulting to U.S.-government agencies on a grand scale grew out of the perceived successes in World War II of civilian analysts who were embedded in military organizations. To the extent that the civilian analysts were actually helpful*, it was because they focused on specific operations, such as methods of searching for enemy submarines. In such cases, the government client can benefit from an outside look at the effectiveness of the operations, the identification of failure points, and suggestions for changes in weapons and tactics that are informed by first-hand observation of military operations.

Beyond that, however, outsiders are of little help, and may be a hindrance, as in the case cited above. Outsiders can’t really grasp the dynamics and unwritten rules of organizational cultures that embed decades of learning and adaptation.

The consulting game is now (and has been for decades) an invasive species. It is a perverse outgrowth of operations research as it was developed in World War II. Too much of a “good thing” is a bad thing — as I saw for myself many years ago.
__________
* The success of the U.S. Navy’s antisubmarine warfare (ASW) operations had been for decades ascribed to the pioneering civilian organization known as the Antisubmarine Warfare Operations Research Group (ASWORG). However, with the publication of The Ultra Secret in 1974 (and subsequent revelations), it became known that code-breaking may have contributed greatly to the success of various operations against enemy forces, including ASW.

Analysis for Government Decision-Making: Demi-Science, Hemi-Demi-Science, and Sophistry

Taking a “hard science” like classical mechanics as an epitome of science and, say, mechanical engineering, as a rigorous application of it, one travels a goodly conceptual distance before arriving at operations research (OR). Philip M. Morse and George E. Kimball, pioneers of OR in World War II, put it this way:

[S]uccessful application of operations research usually results in improvements by factors of 3 or 10 or more…. In our first study of any operation we are looking for these large factors of possible improvement…. They can be discovered if the [variables] are given only one significant figure,…any greater accuracy simply adds unessential detail.

One might term this type of thinking “hemibel thinking.” A bel is defined as a unit in a logarithmic scale corresponding to a factor of 10. Consequently a hemibel corresponds to a factor of the square root of 10, or approximately 3. (Philip M. Morse and George E. Kimball, Methods of Operations Research, originally published as Operations Evaluation Group Report 54, 1946, p. 38)

This is science-speak for the following proposition: Where there is much variability in the particular circumstances of combat, there is much uncertainty about the contributions of various factors (human, mechanical, and meteorological) the the outcome of combat. It is therefore difficult to assign precise numerical values to the various factors.

OR, even in wartime, is therefore, and at best, a demi-science. From there, we descend to cost-effectiveness analysis and its constituent branches: techniques for designing and estimating the costs of systems that do not yet exist and the effectiveness of such systems in combat. These methods, taken separately and together, are (to coin a term) hemi-demi-scientific — a fact that the application of “rigorous” mathematical and statistical techniques cannot alter.

There is no need to elaborate on the wild inaccuracy of estimates about the costs and physical performance of government-owned and operated systems, whether they are intended for military or civilian use. The gross errors of estimation have been amply documented in the public press for decades.

What is less well known is the difficulty of predicting the performance of systems — especially combat systems — years before they are commanded, operated, and maintained by human beings, under conditions that are likely to be far different than those envisioned when the systems were first proposed. A paper that I wrote thirty years ago gives my view of the great uncertainty that surrounds estimates of the effectiveness of systems that have yet to be developed, or built, or used in combat:

Aside from a natural urge for certainty, faith in quantitative models of warfare springs from the experience of World War II, when they seemed to lead to more effective tactics and equipment. But the foundation of this success was not the quantitative methods themselves. Rather, it was the fact that the methods were applied in wartime. Morse and Kimball put it well:

Operations research done separately from an administrator in charge of operations becomes an empty exercise. To be valuable it must be toughened by the repeated impact of hard operational facts and pressing day-by-day demands, and its scale of values must be repeatedly tested in the acid of use. Otherwise it may be philosophy, but it is hardly science. [Methods of Operations Research, p. 10]

Contrast this attitude with the attempts of analysts … to evaluate weapons, forces, and strategies with abstract models of combat. However elegant and internally consistent the models, they have remained as untested and untestable as the postulates of theology.

There is, of course, no valid test to apply to a warfare model. In peacetime, there is no enemy; in wartime, the enemy’s actions cannot be controlled. Morse and Kimball, accordingly, urge “hemibel thinking”:

Having obtained the constants of the operations under study… we compare the value of the constants obtained in actual operations with the optimum theoretical value, if this can be computed. If the actual value is within a hemibel (…a factor of 3) of the theoretical value, then it is extremely unlikely that any improvement in the details of the operation will result in significant improvement. [When] there is a wide gap between the actual and theoretical results … a hint as to the possible means of improvement can usually be obtained by a crude sorting of the operational data to see whether changes in personnel, equipment, or tactics produce a significant change in the constants. [Ibid., p. 38]

….

Much as we would like to fold the many different parameters of a weapon, a force, or a strategy into a single number, we can not. An analyst’s notion of which variables matter and how they interact is no substitute for data. Such data as exist, of course, represent observations of discrete events — usually peacetime events. It remains for the analyst to calibrate the observations, but without a benchmark to go by. Calibration by past battles is a method of reconstruction –of cutting one of several coats to fit a single form — but not a method of validation. Lacking pertinent data, an analyst is likely to resort to models of great complexity. Thus, if useful estimates of detection probabilities are unavailable, the detection process is modeled; if estimates of the outcomes of dogfights are unavailable, aerial combat is reduced to minutiae. Spurious accuracy replaces obvious inaccuracy; untestable hypotheses and unchecked calibrations multiply apace. Yet the analyst claims relative if not absolute accuracy, certifying that he has identified, measured, and properly linked, a priori, the parameters that differentiate weapons, forces, and strategies….

Should we really attach little significance to differences of less than a hemibel? Consider a five-parameter model, involving the conditional probabilities of detecting, shooting at, hitting, and killing an opponent — and surviving, in the first place, to do any of these things. Such a model might easily yield a cumulative error of a hemibel, given a twenty-five percent error in each parameter. My intuition is that one would be lucky if relative errors in the probabilities assigned to alternative weapons and forces were as low as twenty-five percent.

The further that one travels from an empirical question, such as the likely effectiveness of an extant weapon system under specific, quantifiable conditions, the more likely one is to encounter the kind of sophistry known as policy analysis. It is in this kind of analysis that one  — more often than not — encounters in the context of broad policy issues (e.g., government policy toward health care, energy, or defense spending). Such analysis is constructed so that it favors the prejudices of the analyst or his client, or support the client’s political case for a certain policy.

Policy analysis often seems credible, especially on first hearing or reading it. But, on inspection, it is usually found to have at least two of these characteristics:

  • It stipulates or quickly arrives at a preferred policy, then marshals facts, calculations, and opinions that are selected because they support the preferred policy.
  • If it offers and assesses alternative policies, they are not placed on an equal footing with the preferred policy. They are, for example, assessed against criteria that favor the preferred policy, while other criteria (which might be important ones) are ignored or given short shrift.
  • It is wrapped in breathless prose, dripping with words and phrases like “aggressive action,”grave consequences,” and “sense of urgency.”

No discipline or quantitative method is rigorous enough to redeem policy analysis, but two disciplines are especially suited to it: political “science” and macroeconomics. Both are couched in the language of real science, but both lend themselves perfectly to the old adage: garbage in, garbage out.

Do I mean to suggest that broad policy issues should not be addressed as analytically as possible? Not at all. What I mean to suggest is that because such issues cannot be illuminated with scientific rigor, they are especially fertile ground for sophists with preconceived positions.

In that respect, the model of cost-effectiveness analysis, with all of its limitations, is to be emulated. Put simply, it is to state a clear objective in a way that does not drive the answer; reveal the assumptions underlying the analysis; state the relevant variables (factors influencing the attainment of the objective); disclose fully the data, the sources of data, and analytic methods; and explore openly and candidly the effects of variations in key assumptions and critical variables.

The National Psyche and Foreign Wars

I belong to a Google Group whose active members are retired scientists, engineers, mathematicians, and economists — some in their upper 80s — who worked on defense issues from the 1940s to the 2000s. The issues ranged in scope from devising improved tactics for naval, air, and ground operations to assessing the costs and effectiveness of proposed new weapon systems.

Most members of the group were government employees and/or employees of government contractors. Their attraction to government service — and its steady and rather handsome paychecks — derives, in good part, from their belief in the power of government to “solve problems,” and in the need for government to do just that. It is only natural, then, that many members of the group hold an unrealistically exalted view of the power of quantitative methods to “solve problems,” while holding naive views about the machinations of government, human nature, and history. (The pioneers of military operations research in the United States, by contrast, were realistic about the relative impotence of quantitative analysis of complex, dynamic processes.)

Here, for example, is a slightly edited exchange I had with an older member of the group:

Older member (OM):

Does anyone know whether the people of the U.S. were as little involved in the Indian Wars and the opening of the west (some would say stealing) as we seem to be involved with the wars in the Middle East and South Asia.

The greatest asset of our military is its “can do” attitude. The greatest weakness of our military is its “can do” attitude.

Me (Thomas):

I’m not sure what it means for a people to be “involved” in a war. If by “involved” you mean the popularity or unpopularity of the various wars, I have no relevant facts to offer.

But the transient popularity or unpopularity of a war (or any governmental action) shouldn’t matter. If public policy responded to the whims of the “man in the street,” we would be in deep trouble. That’s why there are prescribed processes for making governmental policy. Following the processes doesn’t ensure wise policies, but it beats the alternative of capricious governance.

Our present wars were duly authorized by Congress, and are funded by appropriations made by Congress. Given that the members of Congress are elected representatives of the people, then the people are as involved as they can be under any sensible system of government.

As for the military’s “can do” attitude, decisions about going to war — and staying at war — are the province of civilian authority. When given a war to fight, the only sensible way for the military to approach it is with a “can do” attitude. Does the military’s “can do” attitude color the advice it gives when civilian authority is considering whether to go to war, how to prosecute a war, and whether to persevere in a war? Or are military leaders duly cautious in the advice they give civilian authority, knowing the consequences for their troops and the nation if a war goes badly? I haven’t been close enough to the “inside” — nor have I read deeply enough into military history — to essay answers to those questions.

OM:

I wanted to go a little beyond what might be called the legalities and into the national psyche. The decision to go to war is an awesome political and moral  decision. It has often been said that “old men send young men to war”. In our modern adventures only a fraction of the Country has other than a remote financial involvement in our wars. A small fraction of our Legislative Branch have direct Military Service experience (the smallest in history). An even smaller number has sons or daughters in the Armed Services. We are much moe detached than when the Signers of the Declaration of Independence pledged their honor, their fortune, and their good right hand. (Quote not quite accurate).

During the Vietnam War the Country lowered its support as the costs and casualties rose. Now we do not have the draft though even so military leaders warn the political entities that we must not lose the confidence of the people even as we seem to drift away from the “Powell” Doctrine. We certainly see the heavy imprint of the Military-Industrial Complex against which President Eisenhower warned. (That speech is still on Wikipedia).

During Vietnam we had the bugbear of the “Domino Theory”. There are some who argue along those lines now regarding threats to Israel and other major American Interests. Can a small special interests group lead American policy?

I was wondering what other precedents in American History might apply. Most of the 19th Century was dominated by America’s Manifest Destiny (and losses were modes). Then came the War to end all wars. Then the Era of Good Feeling punctuated by the Washington Naval Conference, the Great Depression, and then the rise of Nazi Germany and its Axis with Italy and Japan. Have we found a bizarre combination of of Depression and Manifest Destiny with a liberal dose of hubris as we dismantle our Navy having already essentially worn out much of our Armour and still at the mercy of land mines and IED (a form of landmine).

One parallel that seems to track from the 19th Century is the corruption of the Suttlers [sic] that has transformed nicely into the Military Industrial Complex.

Thomas:

I have great difficulty with the concept of “national psyche,” and thus with generalizations about what “we” (as a nation) have done and should do. I cannot describe my own psyche, let alone the psyches of millions of other Americans, dead and alive, who differ from me (often greatly) in nature and nurture.

In any event, you come close to answering your own questions in your third paragraph, where you ask “Can a small special interests group lead American policy?” My answer is a resounding “yes.” Two, relatively small, interlocking groups of strong-willed individuals were responsible for the Revolution and the Constitution, and those groups were bound by two special interests (at least): independence from Britain (not a universally popular idea at the time) and freedom from Britain’s interference in the colonies’ commerce. (The second interest is a “bad thing” only if one view commercial interests as a “bad thing.” Unlike the historians of the Beard school, I do not.)

Various and shifting coalitions of special-interest groups have determined the foreign and domestic policies of the United States government from its beginning, and always will do so. There is no escape from such an arrangement, given our system of government — the “legalities” to which you refer. Those “legalities” — and the absence of a national psyche which somehow translates the consolidated wisdom of “the nation” into governmental policy — make it inevitable that governmental policy will be the product of various and shifting coalitions of special-interest groups. You (I mean the generic “you” and not you, [OM]) may like the resulting policies in some cases (e.g., if you are a fan of British-style health care you will consider Obamacare a great leap forward) and dislike them in other cases (e.g., if you are an opponent of foreign wars except those that in retrospect seem worthwhile, you will generally oppose foreign wars).

The “dismantling” of the Navy to which you refer is the specific policy of a specific administration (or administrations). It was not the policy of the Reagan administration, nor was it a policy of the Kennedy administration. And, I hope, it will not be the policy of the next administration. In any case, governmental policy toward the Navy is part of a larger set of policies, the combination of which is dictated by the complex interplay of various special interests and the particular psyches of elected and appointed officials. In the present case, the “dismantling” of the Navy arises from a particular view of how to defend Americans and their property and, not coincidentally, also makes certain kinds of domestic government programs more affordable. It should go without saying that the particular view of how to defend Americans (diplomacy, good will, lower defense budgets) finds opposition in millions of Americans’ psyches, as does the present administration’s commitment to various domestic programs. Liberal hawks — to the extent that they still exist — must be having a hard time digesting the present administration’s combination of domestic and foreign policies, just as conservative hawks — whose are legion — had a hard time digesting the previous administration’s combination of domestic and foreign policies.

As for the military-industrial complex, there is a coalition of interests that can be described broadly by that term, though it is a coalition fraught with internal conflicts and rivalries. If that coalition deserves blame for any excesses in defense spending and misadventures in foreign fields, it also deserves a large share of the credit for the outcomes of World War II and the Cold War.

Tip O’Neill said that all politics is local. I say that all political developments reflect the clash, compromise, and collaboration of special interests — and thus cannot be ascribed to a national psyche.