The interpretation of free energy as bit-erasure capacity

Our paper discussed in the previous blog post might prompt this question: Is there still a way to use Landauer’s principle to convert the free energy of a system to its bit erasure capacity? The answer is “yes”, which we can demonstrate with a simple argument.


Summary: The correct measure of bit-erasure capacity N for an isolated system is the negentropy, the difference between the system’s current entropy and the entropy it would have if allowed to thermalize with its current internal energy. The correct measure of erasure capacity for a constant-volume system with free access to a bath at constant temperature T is the Helmholtz free energy A (divided by kT, per Landauer’s principle), provided that the additive constant of the free energy is set such that the free energy vanishes when the system thermalizes to temperature T. That is,

    \[N = \frac{A}{kT} = \frac{U-U_0}{kT} - (S - S_0),\]

where U_0 and S_0 are the internal energy and entropy of the system if it were at temperature T. The system’s negentropy lower bounds this capacity, and this bound is saturated when U = U_0.


Traditionally, the Helmholtz free energy of a system is defined as \tilde{A} = U - kTS, where U and S are the internal energy and entropy of the system and T is the constant temperature of an external infinite bath with which the system can exchange energy.Here, there is a factor of Boltzmann’s constant k in front of TS because I am measuring the (absolute) entropy S in dimensionless bits rather than in units of energy per temperature. That way we can write things like N = S_0 - S.a   (I will suppress the “Helmholtz” modifier henceforth; when the system’s pressure rather than volume is constant, my conclusion below holds for the Gibbs free energy if the obvious modifications are made.)

However, even in the case of fixed bath temperature, we cannot naively use Landauer’s principle to divide the free energy by kT to get the erasure capacity.… [continue reading]

On computational aestivation

People often say to me “Jess, all this work you do on the foundations of quantum mechanics is fine as far as it goes, but it’s so conventional and safe. When are you finally going to do something unusual and take some career risks?” I’m now pleased to say I have a topic to bring up in such situations: the thermodynamic incentives of powerful civilizations in the far future who seek to perform massive computations. Anders Sandberg, Stuart Armstrong, and Milan M. Ćirković previously argued for a surprising connection between Landauer’s principle and the Fermi paradox, which Charles Bennett, Robin Hanson, and I have now critiqued. Our comment appeared today in the new issue of Foundations of Physics:

Comment on 'The aestivation hypothesis for resolving Fermi's paradox'
Charles H. Bennett, Robin Hanson, C. Jess Riedel
In their article [arXiv:1705.03394], 'That is not dead which can eternal lie: the aestivation hypothesis for resolving Fermi's paradox', Sandberg et al. try to explain the Fermi paradox (we see no aliens) by claiming that Landauer's principle implies that a civilization can in principle perform far more (~1030 times more) irreversible logical operations (e.g., error-correcting bit erasures) if it conserves its resources until the distant future when the cosmic background temperature is very low. So perhaps aliens are out there, but quietly waiting. Sandberg et al. implicitly assume, however, that computer-generated entropy can only be disposed of by transferring it to the cosmological background. In fact, while this assumption may apply in the distant future, our universe today contains vast reservoirs and other physical systems in non-maximal entropy states, and computer-generated entropy can be transferred to them at the adiabatic conversion rate of one bit of negentropy to erase one bit of error.
[continue reading]

FAQ about experimental quantum Darwinism

I am briefly stirring from my blog-hibernationThis blog will resume at full force sometime in the future, but not just yet.a   to present a collection of frequently asked questions about experiments seeking to investigate quantum Darwinism (QD). Most of the questions were asked by (or evolved from questions asked by) Phillip Ball while we corresponded regarding his recent article “Quantum Darwinism, an Idea to Explain Objective Reality, Passes First Tests” for Quanta magazine, which I recommend you check out.


Who is trying see quantum Darwinism in experiments?

I am aware of two papers out of a group from Arizona State in 2010 (here and here) and three papers from separate groups last year (arXiv: 1803.01913, 1808.07388, 1809.10456). I haven’t looked at them all carefully so I can’t vouch for them, but I think the more recent papers would be the closest thing to a “test” of QD.

What are the experiments doing to put QD the test?

These teams construct a kind of “synthetic environment” from just a few qubits, and then interrogate them to discover the information that they contain about the quantum system to which they are coupled.

What do you think of experimental tests of QD in general?

Considered as a strictly mathematical phenomenon, QD is the dynamical creation of certain kinds of correlations between certain systems and their environments under certain conditions. These experiments directly confirm that, if such conditions are created, the expected correlations are obtained.

The experiments are, unfortunately, not likely to offer many insight or opportunities for surprise; the result can be predicted with very high confidence long in advance.… [continue reading]

Links for July 2018

  • The hyoid bone is unique in the human skeleton for being free-floating; it does not share a joint with any other bones, and is only distantly connected to the skull through the Stylohyoid ligament. It is mostly held in place by muscle and cartilage, and helps control the tongue and larynx. Unlike a human’s clavicle, a cat’s clavicle is similarly free-floating, allowing a cat’s shoulders to squeeze through openings as narrow as their skull.
  • Mars Pathfinder

    …was the first of a series of missions to Mars that included rovers, and was the first successful lander since the two Vikings landed on the red planet in 1976…In addition to scientific objectives, the Mars Pathfinder mission was also a “proof-of-concept” for various technologies, such as airbag-mediated touchdown and automated obstacle avoidance, both later exploited by the Mars Exploration Rover mission. The Mars Pathfinder was also remarkable for its extremely low cost relative to other robotic space missions to Mars.

    Here’s Cindy Healy talking about UNIX administration for Pathfinder.

    (H/t Dan Fincke.)

  • Good write-up about the boys rescued from the cave in Thailand.
  • 18-year-old Ewin Tang has proved that the Kerenidis and Prakash recommendation algorithm does not provide an example of an exponential speed up in quantum machine learning. Here’s his advisor Scott Aaronson on the implications:

    Prior to Ewin’s result, the KP algorithm was arguably the strongest candidate there was for an exponential quantum speedup for a real-world machine learning problem. The new result thus, I think, significantly changes the landscape for quantum machine learning; note that whether KP gives a real exponential speedup was one of the main open problems mentioned in John Preskill’s survey on the applications of near-term quantum computers

    More Fuel For The QML Skeptic Game.

[continue reading]

Links for June 2018

[continue reading]

Hennessey on Career Regret

I’ve been mulling for a long time whether to stay in physics, and a colleague pointed me toward this Master’s thesis on career regret by Hennessey.

The study examines the experiences of individuals who, if given their time back, would have chosen a different career path. Despite the fact that career has been consistently documented as a major life regret for many it is rarely mentioned, or only referred to tangentially, in career development literature. Five individual interviews, four female, one male, with people retired or transitioning to retirement are presented to explore the experience of regret as it persists throughout the adult lives of participants. Although the narratives shared by participants are unique and deeply personal, common themes emerged through qualitative analysis. Four themes relate to perceptions of the past: Early Influences, Why I Regret My Choice, The Passage of Time, and Balancing Work and Family. One theme relates to the present: If I Could Do It Over Again, and one to the future: What the Future will Be. Findings from the current study add to the limited research on the topic of career regret and implications for theory and practice are examined.

In this blog post I’ll mostly just pull out notable excerpts. I encourage you to read the thesis if this catches your interest. (See also Hanson on deathbed regrets.)

From the introduction:

If you work full time for thirty years the number of hours spent on the job would be approximately 60,000…

What if you never figured out what you want to do with your life? What if you spent your whole life searching and never found the work you wanted?

[continue reading]

Links for April-May 2018

Public service announcement: Feedback from my readers is eagerly sought. Let me know in the comments or by email what you do and don’t find interesting, and maybe a bit of background about yourself. (EDIT: 0.3% response rate? Get it together!)

Now back to your regularly scheduled programming…

  • Complete lifecycle of HIV in 3D”. This really drives home how insane the world is going to be once intelligent agents are accurately designing machines on the molecular scale.
  • Chris Shroeder on China’s Belt & Road Initiative:

    It’s the largest global engagement strategy since the Marshall Plan — only…like 40 X as large in real dollars.

    Here’s a slightly hokey 6-minute introduction from Vox (“7 out of the 10 biggest construction firms in the world are now Chinese”):

    (H/t Malcom Ocean.)

    Relatedly, here’s diplomat Kishore Mahbubani on the potential for conflicts between the US and China (45 minute of lecture and 45 minutes of questions):Interestingly, I’ve found when increasing video playback speed that YouTube on Chrome has fewer skips and clips that impede intelligibility than VLC does playing back the file (at the same speed). Does anyone know why? Or can anyone recommend an alternative to VLC (or a new VLC plugin)?a  

    (H/t Julia Peng.) Some of the important/interesting claims: (1) The Chinese people are largely accepting of authoritarianism and generally believe that their long history makes democracy less suitable there. (2) The Chinese economic rise has been meteoric, demonstrating that economic liberalism can be pretty cleanly separated from political liberalism. (3) The US ought to submit to more multi-lateralism and international rule-of-law now in order to establish norms that will constrain China later.

[continue reading]

Tishby on physics and deep learning

Having heard Geoffrey Hinton’s somewhat dismissive account of the contribution by physicists to machine learning in his online MOOC, it was interesting to listen to one of those physicists, Naftali Tishby, here at PI:


The Information Theory of Deep Neural Networks: The statistical physics aspects
Naftali Tishby
Abstract:

The surprising success of learning with deep neural networks poses two fundamental challenges: understanding why these networks work so well and what this success tells us about the nature of intelligence and our biological brain. Our recent Information Theory of Deep Learning shows that large deep networks achieve the optimal tradeoff between training size and accuracy, and that this optimality is achieved through the noise in the learning process.

In this talk, I will focus on the statistical physics aspects of our theory and the interaction between the stochastic dynamics of the training algorithm (Stochastic Gradient Descent) and the phase structure of the Information Bottleneck problem. Specifically, I will describe the connections between the phase transition and the final location and representation of the hidden layers, and the role of these phase transitions in determining the weights of the network.

Based partly on joint works with Ravid Shwartz-Ziv, Noga Zaslavsky, and Shlomi Agmon.


(See also Steve Hsu’s discussion of a similar talk Tishby gave in Berlin, plus other notes on history.)

I was familiar with the general concept of over-fitting, but I hadn’t realized you could talk about it quantitatively by looking at the mutual information between the output of a network and all the information in the training data that isn’t the target label.… [continue reading]