No comments

Notable reviews of arguments for AGI ruin

  By ,

Here’s a collection of reviews of the arguments that artificial general intelligence represents an existential risk to humanity. They vary greatly in length and style. I may update this from time to time.

  • Here, from my perspective, are some different true things that could be said, to contradict various false things that various different people seem to believe, about why AGI would be survivable on anything remotely remotely resembling the current pathway, or any other pathway we can easily jump to.
  • This report examines what I see as the core argument for concern about existential risk from misaligned artificial intelligence. I proceed in two stages. First, I lay out a backdrop picture that informs such concern. On this picture, intelligent agency is an extremely powerful force, and creating agents much more intelligent than us is playing with fire -- especially given that if their objectives are problematic, such agents would plausibly have instrumental incentives to seek power over humans. Second, I formulate and evaluate a more specific six-premise argument that creating agents of this kind will lead to existential catastrophe by 2070. On this argument, by 2070: (1) it will become possible and financially feasible to build relevantly powerful and agentic AI systems; (2) there will be strong incentives to do so; (3) it will be much harder to build aligned (and relevantly powerful/agentic) AI systems than to build misaligned (and relevantly powerful/agentic) AI systems that are still superficially attractive to deploy; (4) some such misaligned systems will seek power over humans in high-impact ways; (5) this problem will scale to the full disempowerment of humanity; and (6) such disempowerment will constitute an existential catastrophe.
… [continue reading]
No comments

Table of proposed macroscopic superpositions

  By

Here is a table of proposals for creating enormous superpositions of matter. Importantly, all of them describe superpositions whose spatial extent is comparable to or larger than the size of the object itself. Many are quite speculative. I’d like to keep this table updated, so send me references if you think they should be included.

experimentref.object
composition		object
radius (nm)		nucleon
count		superposition
size (nm)		lifetime
(ms)		repetition
rate (Hz)
KDTL[1-3]OligoporphyrinTo achieve their highest masses, the KDTL interferometer has superposed molecules of functionalized oligoporphyrin, a family of organic molecules composed of C, H, F, N, S, and Zn with molecular weights ranging from ~19,000 Da to ~29,000 Da. (The units here are Daltons, also known as atomic mass units (amu), i.e., the number of protons and neutrons.) The distribution is peaked around 27,000 Da.a  ,00∼1.02.7 × 104100,266100,001.2410,000.00
OTIMA[4-6]Gold (Au),0005.06.0 × 106100,079100,094.0010,600.00
Bateman et al.[7]Silicon (Si),0005.51.1 × 106100,150100,140.0010,000.50
Geraci et al.[8]Silica (SiO2),0006.51.6 × 106100,250100,250.0010,000.50
Wan et al.[9]Diamond (C),0095.07.5 × 109100,100100,000.0510,001.00
MAQRO[10-13]Silica (SiO2),0120.01.0 × 101000,100100,000.0010,000.01
Pino et al.[14]Niobium (Nb)1,000.02.2 × 101300,290100,450.0010,000.10
Stickler et al.
… [continue reading]
No comments

GPT-3, PaLM, and look-up tables

  By ,
[This topic is way outside my expertise. Just thinking out loud.]

Here is Google’s new language model PaLM having a think:

Alex Tabarrok writes

It seems obvious that the computer is reasoning. It certainly isn’t simply remembering. It is reasoning and at a pretty high level! To say that the computer doesn’t “understand” seems little better than a statement of religious faith or speciesism…

It’s true that AI is just a set of electronic neurons none of which “understand” but my neurons don’t understand anything either. It’s the system that understands. The Chinese room understands in any objective evaluation and the fact that it fails on some subjective impression of what it is or isn’t like to be an AI or a person is a failure of imagination not an argument…

These arguments aren’t new but Searle’s thought experiment was first posed at a time when the output from AI looked stilted, limited, mechanical. It was easy to imagine that there was a difference in kind. Now the output from AI looks fluid, general, human. It’s harder to imagine there is a difference in kind.

Tabarrok uses an illustration of Searle’s Chinese room featuring a giant look-up table:

But as Scott Aaronson has emphasized [PDF], a machine that simply maps inputs to outputs by consulting a giant look-up table should not be considered “thinking” (although it could be considered to “know”). First, such a look-up table would be beyond astronomically large for any interesting AI task and hence physically infeasible to implement in the real universe. But more importantly, the fact that something is being looked up rather than computed undermines the idea that the system understands or is reasoning.… [continue reading]

2 Comments

Lindblad operator trace is 1st-order contribution to Hamiltonian part of reduced dynamics

  By

In many derivations of the Lindblad equation, the authors say something like “There is a gauge freedomA gauge freedom of the Lindblad equation means a transformation we can to both the Lindblad operators and (possibly) the system’s self-Hamiltonian, without changing the reduced dynamics.a   in our choice of Lindblad (“jump”) operators that we can use to make those operators traceless for convenience”. However, the nature of this freedom and convenience is often obscure to non-experts.

While reading Hayden & Sorce’s nice recent paper [arXiv:2108.08316] motivating the choice of traceless Lindblad operators, I noticed for the first time that the trace-ful parts of Lindblad operators are just the contributions to Hamiltonian part of the reduced dynamics that arise at first order in the system-environment interaction. In contrast, the so-called “Lamb shift” Hamiltonian is second order.

Consider a system-environment decomposition \mathcal{S}\otimes \mathcal{E} of Hilbert space with a global Hamiltonian H = H_S + H_{I} + H_E, where H_S = H_S \otimes I_\mathcal{E}, H_E = I_\mathcal{S}\otimes H_E, and H_I = \epsilon \sum_\alpha A_\alpha \otimes B_\alpha are the system’s self Hamiltonian, the environment’s self-Hamiltonian, and the interaction, respectively. Here, we have (without loss of generality) decomposed the interaction Hamiltonian into a tensor product of Hilbert-Schmidt-orthogonal sets of operators \{A_\alpha\} and \{B_\alpha\}, with \epsilon a real parameter that control the strength of the interaction.

This Hamiltonian decomposition is not unique in the sense that we can alwaysThere is also a similar freedom with the environment in the sense that we can send H_E \to H_E + \Delta H_E and \epsilon H_I \to \epsilon H_I - \Delta H_E.b   send H_S \to H_S + \Delta H_S and H_I \to H_I - \Delta H_S, where \Delta H_S = \Delta H_S \otimes I_\mathcal{E} is any Hermitian operator acting only on the system. When reading popular derivations of the Lindblad equation

(1)   \begin{align*} \partial_t \rho_{\mathcal{S}} = -i[\tilde{H}_{\mathcal{S}}, \rho_{\mathcal{S}}] + \sum_i\left[L_i \rho_{\mathcal{S}} L_i^\dagger - (L_i^\dagger L_i \rho_{\mathcal{S}} + \rho_{\mathcal{S}} L_i^\dagger L_i)/2\right] \end{align*}

like in the textbook by Breuer & Petruccione, one could be forgivenSpecifically, I have forgiven myself for doing this…c   for thinking that this freedom is eliminated by the necessity of satisfying the assumption that \mathrm{Tr}_\mathcal{E}[H_I(t),\rho(0)]=0, which is crucially deployed in the “microscopic” derivation of the Lindblad equation operators \tilde{H}_{\mathcal{S}} and \{L_i\} from the global dynamics generated by H.… [continue reading]

One Comment

Weingarten’s branches from quantum complexity

  By

Don Weingarten’s newI previously blogged about earlier work by Weingarten on a related topic. This new paper directly addresses my previous concerns.a   attack [2105.04545] on the problem of defining wavefunction branches is the most important paper on this topic in several years — and hence, by my strange twisted lights, one of the most important recent papers in physics. Ultimately I think there are significant barriers to the success of this approach, but these may be surmountable. Regardless, the paper makes tons of progress in understanding the advantages and drawbacks of a definition of branches based on quantum complexity.

Here’s the abstract:

Beginning with the Everett-DeWitt many-worlds interpretation of quantum mechanics, there have been a series of proposals for how the state vector of a quantum system might split at any instant into orthogonal branches, each of which exhibits approximately classical behavior. Here we propose a decomposition of a state vector into branches by finding the minimum of a measure of the mean squared quantum complexity of the branches in the branch decomposition. In a non-relativistic formulation of this proposal, branching occurs repeatedly over time, with each branch splitting successively into further sub-branches among which the branch followed by the real world is chosen randomly according to the Born rule. In a Lorentz covariant version, the real world is a single random draw from the set of branches at asymptotically late time, restored to finite time by sequentially retracing the set of branching events implied by the late time choice. The complexity measure depends on a parameter b with units of volume which sets the boundary between quantum and classical behavior.
… [continue reading]
No comments

How long-range coherence is encoded in the Weyl quasicharacteristic function

  By

In this post, I derive an identity showing the sense in which information about coherence over long distances in phase space for a quantum state \rho is encoded in its quasicharacteristic function \mathcal{F}_{\rho}, the (symplectic) Fourier transform of its Wigner function. In particular I show

(1)   \begin{align*} \int \mathrm{d}^2 \gamma |\langle \alpha|\rho|\beta\rangle|^2 = (|\mathcal{F}_\rho|^2 \ast |G|^2)(\beta-\alpha) \end{align*}

where |\alpha\rangle and |\beta\rangle are coherent states, \gamma:=(\alpha+\beta)/2 is the mean phase space position of the two states, “\ast” denotes the convolution, and G(\alpha) = e^{-|\alpha|^2} is the (Gaussian) quasicharacteristic function of the ground state of the Harmonic oscillator.

Definitions

The quasicharacteristic function for a quantum state \rho of a single degree of freedom is defined as

    \[\mathcal{F}_\rho(\xi) := \mathrm{Tr}[\rho D_\xi] = \langle\rho,D_\xi\rangle_{\mathrm{HS}},\]

where D_\xi = e^{i \xi \wedge R} is the Weyl phase-space displacement operator, \xi = (\xi_{\mathrm{x}},\xi_{\mathrm{p}}) \in \mathbf{R}^2 are coordinates on “reciprocal” (i.e., Fourier transformed) phase space, R:=(X,P) is the phase-space location operator, X and P are the position and momentum operators, “\langle \cdot,\cdot\rangle_{\mathrm{HS}}” denotes the Hilbert-Schmidt inner product on operators, \langle A,B\rangle_{\mathrm{HS}}:=\mathrm{Tr}[A^\dagger B], and “\wedge” denotes the symplectic form, \alpha\wedge\beta := \alpha_{\mathrm{x}}\beta_{\mathrm{p}} - \alpha_{\mathrm{p}}\beta_{\mathrm{x}}. (Throughout this post I use the notation established in Sec. 2 of my recent paper with Felipe Hernández.) It has variously been called the quantum characteristic function, the chord function, the Wigner characteristic function, the Weyl function, and the moment-generating function. It is the quantum analog of the classical characteristic function.

Importantly, the quasicharacteristic function obeys |\mathcal{F}_{\rho}(\xi)|\le 1 and \mathcal{F}_{\rho}(0)=1, just like the classical characteristic function, and provides a definition of the Wigner function where the linear symplectic symmetry of phase space is manifest:

(2)   \begin{align*} \mathcal{W}_{\rho}(\alpha) &:= (2\pi)^{-1}\int\! \mathrm{d}^2\xi \, e^{-i\alpha\wedge\xi}\mathcal{F}_{\rho}(\xi)\\ &= (2\pi)^{-1}\int\! \mathrm{d}\Delta x \, e^{ip\Delta x} \rho(x-\Delta x/2,x+\Delta x/2) \end{align*}

where \alpha = (\alpha_{\mathrm{x}},\alpha_{\mathrm{p}}) = (x,p) \in \mathbf{R}^2 is the phase-space coordinate and \rho(x,x')=\langle x | \rho|x'\rangle is the position-space representation of the quantum state. This first line says that \mathcal{W}_{\rho} and \mathcal{F}_{\rho} are related by the symplectic Fourier transform. (This just means the inner product “\cdot” in the regular Fourier transform is replaced with the symplectic form, and has the simple effect of exchanging the reciprocal variables, (\xi_{\mathrm{x}},\xi_{\mathrm{p}})\to (-\xi_{\mathrm{p}},\xi_{\mathrm{x}}), simplifying many expressions.)… [continue reading]

No comments

Gravitational transmission of quantum information by Carney et al.

  By

Carney, Müller, and Taylor have a tantalizing paper on how the quantum nature of gravity might be confirmed even though we are quite far from being able to directly create and measure superpositions of gravitationally appreciable amounts of matter (hereafter: “massive superpositions”), and of course very far from being able to probe the Planck scale where quantum gravity effects dominate. More precisely, the idea is to demonstrate (assuming assumptions) that the gravitational field can be used to transmit quantum information from one system to another in the sense that the effective quantum channel is not entanglement breaking.

We suggest a test of a central prediction of perturbatively quantized general relativity: the coherent communication of quantum information between massive objects through gravity. To do this, we introduce the concept of interactive quantum information sensing, a protocol tailored to the verification of dynamical entanglement generation between a pair of systems. Concretely, we propose to monitor the periodic wavefunction collapse and revival in an atomic interferometer which is gravitationally coupled to a mechanical oscillator. We prove a theorem which shows that, under the assumption of time-translation invariance, this collapse and revival is possible if and only if the gravitational interaction forms an entangling channel. Remarkably, as this approach improves at moderate temperatures and relies primarily upon atomic coherence, our numerical estimates indicate feasibility with current devices.
[Edit: See also the November 2021 errata.]

Although I’m not sure they would phrase it this way, the key idea for me was that merely protecting massive superpositions from decoherence is actually not that hard; sufficient isolation can be achieved in lots of systems.… [continue reading]

No comments

Comments on Baldijao et al.’s GPT-generalized quantum Darwinism

  By

This nice recent paper considers the “general probabilistic theory” operational framework, of which classical and quantum theories are special cases, and asks what sorts of theories admit quantum Darwinism-like dynamics. It is closely related to my interest in finding a satisfying theory of classical measurement.

Quantum Darwinism and the spreading of classical information in non-classical theories
Roberto D. Baldijão, Marius Krumm, Andrew J. P. Garner, and Markus P. Müller
Quantum Darwinism posits that the emergence of a classical reality relies on the spreading of classical information from a quantum system to many parts of its environment. But what are the essential physical principles of quantum theory that make this mechanism possible? We address this question by formulating the simplest instance of Darwinism – CNOT-like fan-out interactions – in a class of probabilistic theories that contain classical and quantum theory as special cases. We determine necessary and sufficient conditions for any theory to admit such interactions. We find that every non-classical theory that admits this spreading of classical information must have both entangled states and entangled measurements. Furthermore, we show that Spekkens’ toy theory admits this form of Darwinism, and so do all probabilistic theories that satisfy principles like strong symmetry, or contain a certain type of decoherence processes. Our result suggests the counterintuitive general principle that in the presence of local non-classicality, a classical world can only emerge if this non-classicality can be “amplified” to a form of entanglement.

After the intro, the authors give self-contained background information on the two key prerequisites: quantum Darwinism and generalized probabilistic theories (GPTs). The former is an admirable brief summary of what are, to me, the core and extremely simple features of quantum Darwinism.… [continue reading]