I have written a lot about modeling and science. (See the long list of posts at “Modeling, Science, and ‘Reason’“.) I have said, more than once, that modeling isn’t science. What I should have said — though it was always implied — is that a model isn’t scientific if it is merely synthetic.
What do I mean by that? Here is an example by way of contrast. The famous equation E = mc2 is an synthetic model in that it is derived Einstein’s special theory of relativity (and other physical equations). But it is also an empirical model in that the relationship between mass (m) and energy (E) can also be confirmed by observation (given suitable instruments).
On the other hand, a complex model of the U.S. economy, a model of Earth’s “average” temperature (called misleadingly a climate model), or a model of combat (to give a few examples) is only synthetic.
Why do I say that a complex model (of the kind mentioned above) is only synthetic? Such a model consists of a large number of modules, each of which is mathematical formulation of some aspect of the larger phenomenon being modeled. Here’s a simple example: An encounter between a submarine and a surface ship, where the outcome is expressed as the probability that the submarine will sink the surface ship. The outcome could be expressed in this way:
S = D x F x H x K x C, where S = probability that submarine sinks surface ship, which is the product of:
D = probability that submarine detects surface ship within torpedo range
F = probability that, given detection, submarine is able to “fix” the target and fire a torpedo (or salvo of them)
H = probability that, given the firing of a torpedo (or salvo), the surface ship is hit
K = probability that, given a hit (or hits), the surface ship is sunk
C = probability that the submarine survives efforts to find and nullify it before it can detect a surface ship
This is a simple model by comparison with a model of the U.S. economy, a global climate model, or a model of a battle involving large numbers of various kinds of weapons. In fact, it is a simplistic model of combat. Each of the modules could be decomposed into many sub-modules; for example, the module for D could consist of sub-modules for sonar accuracy, sonar operator acuity, acoustic conditions in the area of operation, countermeasures deployed by the target, etc.. In any event, the module for D will consist of a mathematical relationship, based perhaps on some statistics collected from tests or exercises (i.e., not actual combat). The mathematical relationship will encompass many assumptions (mainly implicit ones) about sonar accuracy, sonar operator acuity, etc. The same goes for the other modules — C, in particular, which encompasses all of the effects of D, F, H, and K — at a minimum.
In sum, the number of unknowns completely swamps the number of knowns. There is nothing close to certainty about the model — or any model of its kind. (In the case of the model of S, for example, relatively small errors — say, 25 percent from the actual value of each variable — can yield an estimate of S that is three times greater than or one-third as much as the actual value of S.) The mathematical operations involved do nothing to resolve the uncertainty, they merely multiply it. But the mathematical operations nevertheless convey the appearance of certainty because they yield numbers. The numbers merely represent a lot of guesses, but they seem authoritative because numbers mesmerize most people — even scientists who should be always be skeptical of them.
Despite all of that, analysts have for many decades been producing — and decision-makers have been consuming — the results of such models as the basis for choosing defense systems. Models of similar complexity have been and are being used in making decisions about a broad range of policies affecting the economy, health care, transportation, education, the environment, the climate (i.e., “global warming”), and on into the night.
The unfounded confidence that modelers have in their models, because the models produce numbers, captivates most decision-makers, who simply want answers. And so, modelers will go to ridiculous extremes. One not untypical example that I recall from my days as an in-house critic of analysts’ work is the model that purported to compare competing weapons (on of which was still in development) based on their relative contribution to the outcome of a hypothetical battle. The specific measure was the movement of the forward edge of the battle area (FEBA) to within a yard.
Global climate models are like that warfare model: Their creators pretend that they can estimate the change in the average temperature of the globe to within less than a tenth of a degree. If you believe that, I have a bridge to sell you.
Related reading: Robert L. Bradley Jr. “Climate Models: Worse than Nothing?“, Watts Up With That?, June 23, 2021 (Yes. See below.)
Related pages:
Politics and Prosperity 