In my days as a defense analyst I often encountered military officers who were skeptical about the ability of civilian analysts to draw valid conclusions from mathematical models about the merits of systems and tactics. I took me several years to understand and agree with their position. My growing doubts about the power of quantitative analysis of military matters culminated in a paper where I wrote that

combat is not a mathematical process…. One may describe the outcome of combat mathematically, but it is difficult, even after the fact, to determine the variables that made a difference in the outcome.

Much as we would like to fold the many different parameters of a weapon, a force, or a strategy into a single number, we can not. An analyst’s notion of which variables matter and how they interact is no substitute for data. Such data as exist, of course, represent observations of discrete events — usually peacetime events. It remains for the analyst to calibrate the observations, but without a benchmark to go by. Calibration by past battles is a method of reconstruction — of cutting one of several coats to fit a single form — but not a method of validation.

Lacking pertinent data, an analyst is likely to resort to models of great complexity. Thus, if useful estimates of detection probabilities are unavailable, the detection process is modeled; if estimates of the outcomes of dogfights are unavailable, aerial combat is reduced to minutiae. Spurious accuracy replaces obvious inaccuracy; untestable hypotheses and unchecked calibrations multiply apace. Yet the analyst claims relative if not absolute accuracy, certifying that he has identified, measured, and properly linked, a priori, the parameters that differentiate weapons, forces, and strategies.

In the end, “reasonableness” is the only defense of warfare models of any stripe.

It is ironic that analysts must fall back upon the appeal to intuition that has been denied to military men — whose intuition at least flows from a life-or-death incentive to make good guesses when choosing weapons, forces, or strategies.

My colleagues were not amused, to say the least.

I was reminded of all this by a recent exchange with a high-school classmate who had enlisted my help in tracking down a woman who, according to a genealogy website, is her first cousin, twice removed. The success of the venture is as yet uncertain. But if it does succeed it will be because of the classmate’s intimate knowledge of her family, not my command of research tools. As I said to my classmate,

You know a lot more about your family than I know about mine. I have all of the names and dates in my genealogy data bank, but I really don’t know much about their lives. After I moved to Virginia … I was out of the loop on family gossip, and my parents didn’t relate it to me. For example, when I visited my parents … for their 50th anniversary I happened to see a newspaper clipping about the death of my father’s sister a year earlier. It was news to me. And I didn’t learn of the death of my mother’s youngest brother (leaving her as the last of 10 children) until my sister happened to mention it to me a few years after he had died. And she didn’t know that I didn’t know.

All of which means that there’s a lot more to life than bare facts — dates of birth, death, etc. That’s why military people (with good reason) don’t trust analysts who draw conclusions about military weapons and tactics based on mathematical models. Those analysts don’t have a “feel” for how weapons and tactics actually work in the heat of battle, which is what matters.

Climate modelers are even more in the dark than military analysts because, unlike military officers with relevant experience, there’s no “climate officer” who can set climate modelers straight — or (more wisely) ignore them.

(See also “Modeling Is Not Science“, “The McNamara Legacy: A Personal Perspective“, “Analysis for Government Decision-Making: Hemi-Science, Hemi-Demi-Science, and Sophistry“, “Analytical and Scientific Arrogance“, “Why I Don’t Believe in ‘Climate Change’“, and “Predicting ‘Global’ Temperatures — An Analogy with Baseball“.)