Psychopathologies of Automata II: Autonomous Weapons and Centaur Systems - Computational Psychiatry: A Systems Biology Approach to the Epigenetics of Mental Disorders 1st ed.

Computational Psychiatry: A Systems Biology Approach to the Epigenetics of Mental Disorders 1st ed.

9. Psychopathologies of Automata II: Autonomous Weapons and Centaur Systems

Rodrick Wallace¹

(1)

New York State Psychiatric Institute, New York, NY, USA

Summary

Powerful asymptotic limit theorems of control and information theories illuminate the dynamics of autonomous weapons and man/machine “centaur” or “cockpit” systems under increasing fog-of-war burdens. Adapting tools from previous chapters, a relatively simple analysis shows there will not be graceful degradation in targeting precision, but sudden, highly punctuated, collapse to a pathological state in which “all possible targets are enemies.” A central focus is the complex structure of the fog-of-war ecosystem itself, resulting in statistical tools not unlike regression models that should be useful in data analysis.

9.1 Introduction

The catastrophic drone wars in the Middle East and Africa (Columbia 2012; Stanford/NYU 2012; Wallace 2016a) will haunt the USA well into the next century, just as the miscalculations that created and followed World War I—including the European colonial “country building” producing Iraq and Syria—haunt us today. The USA and others are poised to move beyond remotely piloted drone systems to autonomous weapons and/or centaur warfighting—enhanced man/machine “cockpits” (Scharre 2016). It is asserted that centaur systems “keeping the man in the loop” will not only outperform automatons, but will also constrain, somewhat, the horrors of war.

Contrary perspectives abound. As Archbishop Silvano Tomasi (2014) puts it,

..[T]he development of complex autonomous weapon systems which remove the human actor from lethal decision-making is short-sighted and may irreversibly alter the nature of warfare in a less humane direction, leading to consequences we cannot possibly foresee, but that will in any case increase the dehumanization of warfare.

As Scharre describes, however, the First Gulf War Patriot missile fratricides (Hawley 2006; Wallace 2016a) raise significant questions regarding the operational reliability of such systems under fog-of-war constraints. The Patriot can be seen as an early example of forthcoming centaur man/machine composites.

Trsek (2014) examines the 1988 US AEGIS system downing of a civilian airliner from a similar perspective, concluding that

[Command responsibility] is already several steps removed from the operator in practice—it is naive to believe that we are relying on biological sensing to fulfill [rules-of-engagement] criteria, where the majority of information is electronically derived.

To address some of these matters, we expand the approach of Wallace (2016a), who examined canonical failure modes of real-time control systems using insights from cognitive theory. That work viewed such failures from the general perspective of the Data Rate Theorem that links control and information theories. Here, using the approach of previous chapters, we explore in considerably finer detail the structure and dynamics of the fog-of-war constraints that can collapse such systems into a ground state pathology in which “all possible targets are enemies.” Such collapse moves beyond Scharre’s “operational risk” into violations of the Laws of Land Warfare that require distinction between combatants and non-combatants.

We review and extend the formal linkage between control and information theories, leading to deeper understanding of fog-of-war constraints.

9.2 The Data Rate Theorem

Unlike an aircraft that can remain in stable flight as long as the center of pressure is sufficiently behind the center of gravity, high-order cognitive systems like human sports and combat teams, man–machine “cockpits,” self-driving vehicles, autonomous weapons systems, and modern fighter aircraft—built to be maneuverable rather than stable—operate in real time on rapidly shifting topological “highways” of complex multimodal demand. Facing these turbulent topologies, the cognitive system must receive a constant flow of sufficiently detailed information describing them. More prosaically, driving on a twisting, pot-holed road after dark and at high speed requires very good headlights.

It will become clear that high-order cognition is inherently unstable in the sense of the Data Rate Theorem (Nair et al. 2007), viewing incoming information about the rapidly shifting topology of demand as the control signal.

The Data Rate Theorem (DRT) states that there is a minimum rate at which control information must be provided for an inherently unstable system to remain stable. The most direct approach is a linear expansion near a nonequilibrium steady state. An n-dimensional vector of system parameters at time t, x_t, determines the state at time t + 1 according to the model of Fig. 9.1, so that

$\displaystyle{ x_{t+1} = \mathbf{A}x_{t} + \mathbf{B}u_{t} + W_{t} }$

(9.1)

Fig. 9.1

A linear expansion near a nonequilibrium steady state of an inherently unstable control system, for which x_t₊₁ = A x_t+B u_t+ W _t. A, B are square matrices, x_tthe vector of system parameters at time t, u_tthe vector of control signals at time t, and W_ta white noise vector. The Data Rate Theorem states that the minimum rate at which control information must be provided for system stability is $\mathcal{H}>\log [\vert \det [\mathbf{A}^{m}\vert ]$ , where A ^mis the subcomponent of A having eigenvalues ≥ 1

A, B are fixed n × n matrices, u _tis the vector of control information, and W_tis an n-dimensional vector of white noise. The Data Rate Theorem (DRT) under such conditions states that the minimum control information rate $\mathcal{H}$ is determined by the relation

$\displaystyle{ \mathcal{H}>\log [\vert \det (\mathbf{A}^{m})\vert ] \equiv a_{ 0} }$

(9.2)

where, for m ≤ n, A^mis the subcomponent of A having eigenvalues ≥ 1. The right-hand side of Eq. (9.2) is interpreted as the rate at which the system generates “topological information.” The Mathematical Appendix uses the Rate Distortion Theorem (RDT) to derive a more general version of this result.

The next step is both deceptively simple and highly significant. Given, for the moment, a scalar parameter ρ as an index of multimodal system demand, here representing the fog-of-war (most simply, for example, the magnitude of the dominant vector in a principal component analysis) we extend Eq. (9.2) as

$\displaystyle{ \mathcal{H}(\rho )> f(\rho )a_{0} }$

(9.3)

where a₀ is constant characteristic of low system demand and f(ρ) is a positive, monotonically increasing function. The Mathematical Appendix uses a Black–Scholes model to approximate the “cost” of $\mathcal{H}$ as a function of the “investment” ρ. The first approximation is, surprisingly (or not), linear, so that $\mathcal{H}\approx \kappa _{1}\rho +\kappa _{2}$ . Taking f(ρ) to similar order, so that

$\displaystyle{ f(\rho ) \approx \kappa _{3}\rho +\kappa _{4} }$

(9.4)

the limit condition for stability becomes

$\displaystyle{ \mathcal{T} (\rho ) \equiv \frac{\kappa _{1}\rho +\kappa _{2}} {\kappa _{3}\rho +\kappa _{4}}> a_{0} }$

(9.5)

For ρ = 0, the stability condition is κ ₂∕κ₄ > a ₀. At large ρ the condition becomes κ₁∕κ ₃ > a₀. If κ ₂∕κ₄ ≫ κ ₁∕κ₃, the stability condition may be violated at high demand densities, and instability becomes manifest. See Fig. 9.2.

Fig. 9.2

The horizontal line represents the critical limit a₀. If κ ₂∕κ₄ ≫ κ ₁∕κ₃, at some intermediate value of demand ρ, the temperature analog $\mathcal{T} \equiv (\kappa _{1}\rho +\kappa _{2})/(\kappa _{3}\rho +\kappa _{4})$ falls below that limit, the system becomes “supercooled,” and sudden, punctuated, “phase transition” failure becomes increasingly probable

9.3 The Fog-of-War Ecosystem

The next stage of the argument is not simple. For a control system embedded in a complex and dynamic demand stream—here, the fog-of-war ecology—there will be several (many) ρ-values that are not independent but interact with each other. There is not, then, a simple scalar index of demand, but rather an n × n matrix $\hat{\rho }$ with elements ρ_i_, j, i, j = 1… n. In general, as opposed to correlation matrices, ρ_i_, j≠ ρ_j_, i, since influences need not be symmetric.

Can there still be a single scalar “ρ” under such circumstances so that the conditions of Fig. 9.2 apply? An n × n matrix $\hat{\rho }$ has n invariants r_i, i = 1… n that remain fixed when “principal component analysis” transformations are applied to data. The invariants are found using the famous characteristic equation

$\displaystyle{ \mathcal{P}(\lambda ) =\det (\hat{\rho }-\lambda I_{n}) =\lambda ^{n} + r_{ 1}\lambda ^{n-1} + \cdots + r_{ n-1}\lambda + r_{n} }$

(9.6)

and the r_ican be used to construct an invariant scalar measure.

Here, det is the determinant, λ is a parameter that is an element of a ring, and I_nthe n × n identity matrix. The invariants are the coefficients of λ in the polynomial $\mathcal{P}(\lambda )$ , normalized so that the coefficient of λⁿis 1.

Since n × n matrices themselves form a ring, one has the classic relation

$\displaystyle{ \mathcal{P}(\hat{\rho }) = 0 \times I_{n} }$

(9.7)

so that a matrix satisfies its own characteristic equation.

Again, an n × n matrix will have n such invariants from which a scalar index Γ = g(r ₁, …, r_n) can be constructed to replace ρ in Eq. (9.5). Typically, the first invariant will be the trace and the last ± the determinant of $\hat{\rho }$ .

The reduction of the complicated interaction matrix $\hat{\rho }$ to the scalar Γ is likely to be an ambiguous matter. We are making an approximation that must be fitted to each fog-of-war ecosystem: there will be no one-size-fits-all simplification methodology, although there may be equivalence classes of different systems that can be mapped onto a particular method.

This method can be seen as a variant of the Rate Distortion Manifold approach of Glazebrook and Wallace (2009) or the Generalized Retina used by Wallace and Wallace (2013, Sect. 10.1). Both find necessary conditions that high dimensional data flows can be projected down onto lower dimensional, shifting, tunable “tangent spaces” with minimal loss of essential information. Here, we ask that a complicated matrix interaction be projected onto a scalar function, a significant constraint that will limit the accuracy of the technique. Permitting a higher dimensional tangent space would improve possible fits to data, but would require significantly more formal overhead, and remains to be done.

Essentially we are defining $\mathcal{T} (\varGamma )$ as a synergistic “temperature” index characterizing fog-of-war conditions. Further analysis might uncover other “thermodynamic” quantities characterizing the combat ecosystem. For example, a more complete description of complex multimodal demand and its impacts on high-order, real-time cognitive systems—autonomous weapons or man/machine cockpit/centaurs—might involve concepts of “friction” in addition to fog-of-war “temperature.” That is, we assume here that a weapon system can act promptly and without loss of effectiveness. Quite often, however, combat operations are confronted both by increasing actuation delays and progressive loss of efficiency under stress. The incorporation of delay in the kind of stochastic differential equation models we use here, however, introduces formidable mathematical complications, as would inclusion of stochastic models of damage accumulation.

The simplest approach would be to combine the fog-of-war “temperature” index with delay and attrition measures into a single “wind-chill” factor.

9.4 The Dynamics of Control Failure

What are the dynamics of $\mathcal{T}$ under stochastic circumstances? That is, how is a control signal u_tin Fig. 9.1 expressed in the system response x_t₊₁? We deterministically retranslate an observed sequence of system outputs Xⁱ = x ₁ ⁱ, x₂ ⁱ, … into a sequence of possible control signals $\hat{U}^{i} =\hat{ u}_{0}^{i},\hat{u}_{1}^{i},\ldots$ and compare that sequence with the original control sequence Uⁱ = u ₀ ⁱ, u₁ ⁱ, …, with the difference between them having a particular value under some chosen distortion measure and hence having an average distortion

$\displaystyle{ <d>=\sum _{i}p(U^{i})d(U^{i},\hat{U}^{i}) }$

(9.8)

where p(Uⁱ) is the probability of the sequence Uⁱand $d(U^{i},\hat{U}^{i})$ is the distortion between Uⁱand the sequence of control signals that has been deterministically reconstructed from the system output.

It is then possible to apply a classic Rate Distortion argument (Cover and Thomas 2006). According to the Rate Distortion Theorem, there exists a Rate Distortion Function, R(D), that determines the minimum channel capacity necessary to keep the average distortion below some fixed limit D (Cover and Thomas 2006). Based on Feynman’s (2000) interpretation of information as a form of free energy, it is possible to construct a Boltzmann-like pseudoprobability density in the “temperature” $\mathcal{T}$ as

$\displaystyle{ dP(R,\mathcal{T} ) = \frac{\exp [-R/\mathcal{T} ]dR} {\int _{0}^{\infty }\exp [-R/\mathcal{T} ]dR} }$

(9.9)

since higher $\mathcal{T}$ must necessarily be associated with greater channel capacity.

The integral in the denominator can be interpreted as a statistical mechanical partition function, and it is possible to define a “free energy” Morse Function F (Pettini 2007) as

$\displaystyle{ \exp [-F/\mathcal{T} ] =\int _{ 0}^{\infty }\exp [-R/\mathcal{T} ]dR = \mathcal{T} }$

(9.10)

so that $F(\mathcal{T} ) = -\mathcal{T}\log [\mathcal{T} ]$ .

Then an entropy-analog can also be defined as the Legendre transform of F

$\displaystyle{ \mathcal{S}\equiv F(\mathcal{T} ) -\mathcal{T} dF/d\mathcal{T} = \mathcal{T} }$

(9.11)

The Onsager approximation to nonequilibrium thermodynamics (de Groot and Mazur 1984) can now be applied, using the gradient of $\mathcal{S}$ in $\mathcal{T}$ , so that the dynamics of $\mathcal{T}$ are represented by a stochastic differential equation

$\displaystyle\begin{array}{rcl} d\mathcal{T}_{t}& =& (\mu d\mathcal{S}/d\mathcal{T} )dt +\beta \mathcal{T}_{t}dW_{t} \\ & =& \mu dt +\beta \mathcal{T}_{t}dW_{t} {}\end{array}$

(9.12)

where μ is a diffusion coefficient and β is the magnitude of the impinging white noise dW_t.

As above, in the context of Jensen's inequality for a concave function, applying the Ito chain rule to $\log (\mathcal{T} )$ in Eq. (9.12) gives the familiar result for the nonequilibrium steady state expectation of $\mathcal{T}$ as

$\displaystyle{ E(\mathcal{T}_{t}) \geq \frac{\mu}{\beta^{2}/2} }$

(9.13)

μ is interpreted as the attempt by the control apparatus—autonomous, centaur, cockpit—to maintain stability. Thus rising system noise can significantly increase the probability that $\mathcal{T}$ falls below the critical limit, triggering a punctuated failure of control.

It is important to recognize, however, that, since $E(\mathcal{T} )$ is an expectation, in this model there will always be some probability that $\mathcal{T} (\varGamma )$ will fall below the critical value a ₀.

Raising μ and limiting β decreases that probability, but cannot eliminate it: sudden onset of instability is always possible, triggering a “false target attack” in the sense of Kish et al. (2009):

Unfortunately…adjusting the sensor threshold to increase the number of target attacks also increases the number of false target attacks. Thus the operator’s objectives are competing, and a trade-off situation arises.

It is not difficult to construct numerical simulations of these results, for example, using the ItoProcess construct available in later versions of the computer algebra program Mathematica.

9.5 The Dynamics of High-Level Cognitive Dysfunction

The DRT argument above implies a raised probability of a transition between stable and unstable behavior if the temperature analog $\mathcal{T} (\varGamma )$ from Eq. (9.5) falls below a critical value, as in Fig. 9.2. We can extend the perspective to more complicated patterns of phase transition via the “cognitive paradigm” of Atlan and Cohen (1998), who view a system as cognitive if it compares incoming signals with a learned or inherited picture of the world, then actively chooses a response from a larger set of those possible to it. Choice implies the existence of an information source, since it reduces uncertainty in a formal way (Wallace 2012, 2015, 2016b).

Given such a “dual” information source associated with the inherently unstable cognitive system of interest, an equivalence class algebra can be constructed by choosing different system origin states and defining the equivalence of subsequent states at a later time by the existence of a high probability path connecting them to the same origin state. Disjoint partition by equivalence class, analogous to orbit equivalence classes in dynamical systems, defines a symmetry groupoid associated with the cognitive process (Wallace 2012). Groupoids are generalizations of group symmetries in which there is not necessarily a product defined for each possible element pair (Weinstein 1996). An example would be the disjoint union of different groups.

The equivalence classes across possible origin states define a set of information sources dual to different cognitive states available to the inherently unstable cognitive system. These create a large groupoid, with each orbit corresponding to a transitive groupoid whose disjoint union is the full groupoid. Each subgroupoid is associated with its own dual information source, and larger groupoids must have richer dual information sources than smaller.

Let $X_{G_{i}}$ be the system’s dual information source associated with groupoid element G_i. Given Eqs. (9.5)–(9.7), we construct a Morse Function (Pettini 2007) in a standard manner, using Γ = g(r₁, …, r _n) in Eq. (9.5) in place of ρ.

Let $H(X_{G_{i}}) \equiv H_{G_{i}}$ be the Shannon uncertainty of the information source associated with the groupoid element G_i. Define another Boltzmann-like pseudoprobability as

$\displaystyle{ P[H_{G_{i}}] \equiv \frac{\exp [-H_{G_{i}}/\mathcal{T} ]} {\sum _{j}\exp [-H_{G_{j}}/\mathcal{T} ]} }$

(9.14)

where the sum is over the different possible cognitive modes of the full system.

Another “free energy” Morse Function $\mathcal{F}$ can then be defined as

$\displaystyle\begin{array}{rcl} \exp [-\mathcal{F}/\mathcal{T} ]& \equiv & \sum _{j}\exp [-H_{G_{j}}/\mathcal{T} ] \\ \mathcal{F}& =& -\mathcal{T}\log \left [\sum _{j}\exp [-H_{G_{j}}/\mathcal{T} ]\right ]{}\end{array}$

(9.15)

As a consequence of the underlying groupoid generalized symmetries associated with high-order cognition, as opposed to simple control theory, it is possible to apply an extension of Landau’s version of phase transition (Pettini 2007). Landau argued that spontaneous symmetry breaking of a group structure represents phase change in physical systems, with the higher energies available at higher temperatures being more symmetric. The shift between symmetries is highly punctuated in the temperature index, here the “temperature” analog of Eq. (9.5), in terms of the scalar construct Γ = g(r₁, …, r _n), but in the context of groupoid rather than group symmetries.

Based on the analogy with physical systems, there should be only a few possible phases, with highly punctuated transitions between them as the fog-of-war “temperature” $\mathcal{T}$ decreases, ultimately freezing the operational phenotype into the usual always-on mode: “Kill everything, and let God sort them out.” Sufficient conditions for the stability of this pathological state are discussed in Sects. 4.3 and 5.9 above.

9.6 Discussion and Conclusions

According to the Data Rate Theorem, if the rate at which control information can be provided to an unstable system is below the critical limit defined by the rate at which the system generates “topological information,” there is no coding strategy, no timing strategy, no control scheme of any form, that can ensure stability. Generalization to the rate of incoming information from the rapidly changing multimodal “roadway” environments in which a real-time cognitive system must operate suggests that there will be sharp onset of serious dysfunction under the burden of rising demand. Here we have analyzed that multimodal demand in terms of the crosstalk-like fog-of-war matrix ρ_i_, jthat can be characterized by situation-specific statistical models leading to the scalar temperature analog $\mathcal{T}$ . More complicated “tangent space” reductions are possible, at the expense of greater mathematical overhead (e.g., Glazebrook and Wallace 2009).

There will not be graceful degradation under falling fog-of-war “temperature” or “wind-chill” factors, but rather punctuated functional decline that, for autonomous, centaur, or man–machine cockpit weapon systems, deteriorates into a frozen state in which “all possible targets are enemies,” as in the case of the Patriot missile fratricides (Hawley 2006; Wallace 2016a). Other cognitive systems display analogous patterns of punctuated collapse into simplistic dysfunctional phenotypes or behaviors (Wallace 2015, 2016b): the underlying dynamic is ubiquitous and, apparently, inescapable. In sum, there is no free lunch for cognitive weapon systems, with or without hands-on human control. All such systems are inherently susceptible to serious operational instabilities under complex fog-of-war environments. Policy based on the business dreams of military contractors and their academic or think-tank clients—promises of precision targeting—will be confronted by nightmare realities of martyred civilian populations, recurring generations of new “terrorists,” and the persistent stench of war crime .

References

Atlan, H., and I. Cohen. 1998. Immune information, self-organization, and meaning. International Immunology 10: 711–717.CrossRefPubMed

Columbia University Law School Human Rights Clinic. 2012. Counting Drone Strike Deaths. http://web.law.columbia.edu/human-rights-institute.

Cover, T., and J. Thomas. 2006. Elements of Information Theory, 2nd ed. New York: Wiley.

de Groot, S., and P. Mazur. 1984. Nonequilibrium Thermodynamics. New York: Dover.

Feynman, R. 2000. Lectures on Computation. New York: Westview Press.

Glazebrook, J.F., and R. Wallace. 2009. Rate distortion manifolds as model spaces for cognitive information. Informatica 33: 309–345.

Hawley, J. 2006. Patriot Fratricides: The human dimension lessons of Operation Iraqi Freedom. Field Artillery, January–February.

Kish, B., M. Pachter, and D. Jacques. 2009. Effectiveness measures for operations in uncertain environments. In UAV Cooperative Decision and Control: Challenges and Practical Applications, ed. T. Shima and S. Rasmussen, Chap. 7 Philadelphia: SIAM Publications.

Nair, G., F. Fagnani, S. Zampieri, and R. Evans. 2007. Feedback control under data rate constraints: An overview. Proceedings of the IEEE 95: 108–137.CrossRef

Pettini, M. 2007. Geometry and Topology in Hamiltonian Dynamics. New York: Springer.CrossRef

Scharre, P. 2016. Autonomous weapons and operational risk. Center for New American Security, Washington, DC. http://www.cnas.org/autonomous-weapons-and-operational-risk.vtISHGO-RiY.

Stanford/NYU. 2012. Living under drones: Death, injury and trauma to civilians from US drone practices in Pakistan. http://livingunderdrones.org/.

Tomasi, S. 2014. Catholic Herald. http://www.catholicherald.co.uk/news/2014/05/15/vatican-official-voices-opposition-to-automated-weapons-systems/.

Trsek, R. (Lt. Col. USAF). 2014. Cutting the cord: Discrimination and command responsibility in autonomous lethal weapons. Available online from the Air War College of Air University, USA.

Wallace, R. 2012. Consciousness, crosstalk and the mereological fallacy: An evolutionary perspective. Physics of Life Reviews 9: 426–453.CrossRefPubMed

Wallace, R. 2015. An Information Approach to Mitochondrial Dysfunction: Extending Swerdlow’s Hypothesis. Singapore: World Scientific.CrossRef

Wallace, R. 2016a. Canonical failure modes of real-time control systems: Insights from cognitive theory. International Journal of Systems Science 47: 1280–1295.

Wallace, R. 2016b. Canonical instabilities of autonomous vehicle systems. https://peerj.com/preprints/1714/.

Wallace, R., and D. Wallace. 2013. A Mathematical Approach to Multilevel, Multiscale Health Interventions: Pharmaceutical Industry Decline and Policy Response. London: Imperial College Press.

Weinstein, A. 1996. Groupoids: Unifying internal and external symmetry. Notices of the American Mathematical Society 43: 744–752.

Page

Contents

If you find an error or have any questions, please email us at admin@doctorlib.org. Thank you!