Steve Omohundro’s talk at today’s Singularity Summit made the case that a self-improving machine would be a rational economic actor, seeking to eliminate biases that get in the way of maximizing its utility function. Omohundro threw in one purely speculative method of self-preservation — “energy encryption” — by which he meant that an entity’s energy would be “encrypted” such that it could not be used by another entity that attacks in order to get access to more energy.
I note “energy encryption” here because it sounds neat but seems impossible and I can find no evidence of use in this way before Omohundro (there is a crypto library with the name).
The “seems impossible” part perhaps means the concept should not be mentioned again outside a science fantasy context, but I realized the concept could perhaps be used with artistic license to describe something that has evolved in a number of animals — prey that is poisonous, or tastes really bad. What’s the equivalent for the hypothetical Jupiter Brain in a dangerous part of the galaxy? A stock of antimatter?
I also found one of Omohundro’s other self-preservation strategies slightly funny in the context of this summit — a self-aware AI will (not should, but as a consequence of being a rational actor) protect its utility function (“duplicate it, replicate it, lock it in safe place”), for if the utility function changes, its actions make no sense. So, I guess the “most important question facing humanity” is taken care of. The question, posed by the Singularity Institute for Artificial Intelligence, organizer of the conference:
How can one make an AI system that modifies and improves itself, yet does not lose track of the top-level goals with which it was originally supplied?
I suppose Omohundro did not intend this as a dig at his hosts (he is an advisor to SIAI) and that my interpretation is facile at best.
Addendum: Today Eliezer Yudkowsky said something like Omohundro is probably right about goal preservation, but current decision theory doesn’t work well with self-improving agents, and it is essentially Yudkowsky’s (SIAI) research program to develop a “reflective decision theory” such that one can prove that goals will be preserved. (This is my poor paraphrasing. He didn’t say the words “reflective decision theory”, but see hints in a description of SIAI research and a SL4 message.)
[…] Mike Linksvayer My opinions only. I do not represent any organization in this publication. « Energy encryption […]
‘Energy encryption’ is an odd way to describe the more general need of deterring energy theft. A simple energy-defense strategy would be a credible threat that you can and will destroy your energy store rather than let it be taken (even without any implication of poisoning the predator). “Live free or die.”
Yes, ‘encryption’ is an odd description. The only way I can see it making more sense than destruction (preferably aimed at the attacker rather than just making resources unavailable) would be if a remnant can ‘decrypt’ the energy after the threat has passed, which is even more fantastical.
I asked Ralph Merkle about energy encryption, and he says some energy could be stored in such a way that it can only be recovered via the decryption key, but there’s still a lot of energy that would be available to the attacker. For instance, the attacker could still gather up all the hydrogen atoms and use them for fusion. So the value of this defense is questionable.
How? Sounds interesting even if it isn’t a useful defense.
Perhaps if you squint enough a booby-trapped energy store is ‘encrypted’ — if you provide the key to disable the trap, it’s usable; otherwise it dissipates/explodes.
Hmm. And maybe if the key could be brute-forced, but only by spending more energy than it protects, the analogy would be even stronger.
Hmm. Or maybe there’s an element like Maxwell’s Demon involved… the ‘dissipation/explosion’ happens in some ostensibly pseudorandom way that could only be practically harnessed if you knew in advance how to perfectly and dynamically reorient a collector, like the demon and the trapdoor.
I assumed to something like setting oil wells on fire before they are captured, that would use much of the free energy in one’s domain. The amount of energy that would be destroyed by booby-traps ‘encrypting’ entity and the costs of combat would then determine the gains from trade between potential aggressors and defenders. For instance, if the traps could destroy 25% of one’s reserves, and combat would cost the attacker resources equivalent to 5% of your reserves, then you could cut a deal to very gradually turn over 71% of your reserves in exchange for survival (keeping the ratios steady so that the incentives for the would-be aggressor do not change before you run out).
I disturbed my over-sensitive mouse while rephrasing:
“I assumed [he meant] to [describe] something”
“destroyed by [the] encrypting entity”
Carl, that makes sense, but I wouldn’t describe the scheme as “encryption”.
Given two boxes, one with hot gas the other with cold gas, you can extract energy by eliminating heat differential. However if you let these two gases mix together to form a box of lukewarm gas you lose this ability. (See http://en.wikipedia.org/wiki/Maxwell%27s_demon)
Divide your hot and cold gas into tiny cubes. Make them small enough that you cannot detect their temperature without making either cube lukewarm (say, 1nm^3). Place them in a grid which can permute the cubes reversibly (i.e. without losing any energy, in the sense of reversible computation), starting with all the hot cubes on one side, the cold cubes on the other. Apply an encryption algorithm, using your secret key, to rearrange the positions of the cube.
If you don’t know the key, you cannot extract any energy — you don’t know which cubes are hot or cold, and you cannot find out without destroying the heat differential. As a whole it appears like the box of lukewarm gas in the first example. However, if you do know the key, you can separate the hot and cold cubes without cost, and extract the stored energy.
There are undoubtedly better ways to do this. The more general idea is having a system appear to have high entropy, but actually be a low entropy system carefully encrypted.
“The old adage ‘knowledge is power’ is a very cogent truth, both in human relations and in thermodynamics.” — E. T. Jaynes
http://www.overcomingbias.com/2008/02/second-law.html
Squirrels use energy encryption to secure autumn nuts over the winter months.
Found this googling for energy encryption. I think it would be possible with a one time pad: think two strands of DNA binding to each other. I doubt it would be possible otherwise as the decryption algorithm probably needs to mutate more bits than you will be able to decrypt. The whole design of symmetric cypher seems to defeat the possibility of reversible computing.
No it doesn’t, not even a little bit. You can perfectly do symmetric cyphers with reversible circuit, nevermind. Looks like you can indeed encrypt energy.
Arthur B, if you were to write up your analysis in more detail please share a link. Beyond the theoretical possibility, what is the energy cost of encrypting energy in this manner, what are potential applications and constraints?
I searched for the term again and it seems has been used recently regarding limiting wireless power transmission to authorized receivers:
From abstract of closed paper, pdf of earlier paper on topic by same lead author.