Text of prepared comments

by Ralph C. Merkle
Principal Fellow, Zyvex

at the April 1, 2000 Stanford Symposium

Will spiritual robots replace humanity by 2100?

organized by Douglas Hofstadter
(see http://www.stanford.edu/dept/symbol/Hofstadter-event.html)

(A video of the talk is available on the web. Note that the actual talk departed from the prepared comments at various points)

Bill Joy says we should "relinquish" nanotechnology [see "Why the future doesn't need us" at http://www.wired.com/wired/archive/8.04/joy.html]. While it's not entirely clear what this means, it seems to embrace one of the worst strategies we could adopt. To block or slow research, to block development, or hinder a careful and systematic analysis of the weapons potential of this new technology is a very bad idea for several reasons:

• First, we can not have an informed public discussion without an active research program. The discussion of the feasibility of molecular machines (a feasibility which has only recently been accepted by the scientific and technical community) was sidetracked for decades by confusions about basic physics issues such as thermal noise or quantum uncertainty. You can imagine how well the discussion of more complex issues will faire without an active research program and, just as important, an active community of researchers to provide a check on obviously spurious or incorrect arguments.

• Second, we can only stop research if we know about it and if those funding the research pay attention to us. In large part, this means public research in democracies. We can not stop secret research by undemocratic regimes. You can pick your favorite bad guy, but I doubt very much if Saddam Hussein would pay the slightest attention to any US-lead effort to "relinquish" nanotechnology. The only practical result would be to give less scrupulous nations the economic and military advantages of developing this technology before we did. Abandoning the economic advantages would cost us money, permitting irresponsible regimes to first develop the military capabilities could cost us much, much more.

• Third, we would be turning our backs on technological advances that could save lives and improve the often substandard living conditions of all humanity. While those of us here today enjoy a standard of living which gives us the luxury of turning away from economic gains, there are many others in the world who are less fortunate. For them, economic gain does not mean a third computer or a faster modem, but adequate food and basic medical care. For us to turn our backs on technological advance is to condemn the greater part of the world's population to a life of continued poverty.

Today, there is (finally) general acceptance that nanotechnology is feasible, but there is still considerable confusion about what form it will take and what it will be capable of. One major example is self replication.

To mention "artificial self replication" is to conjure up images of Mickey Mouse from The Sorcerer's Apprentice frantically trying to stop ever more and ever smaller brooms from fetching ever greater floods of water. What are the potential dangers posed by artificial self replicating manufacturing systems?

The only self replicating systems we are familiar with are living systems, and we unconsciously assume that artificial self replicating systems will be similar. But the machines people make bear little resemblance to living systems. While birds showed us that heavier than air flight was possible, airplanes are very different from birds. The image of a 747 going feral, swooping out of the sky to clutch an unsuspecting horse in its landing gear, seems incongruous. Machines don't behave in that way. They lack the wonderful adaptability of living systems. A 747 requires. It can convert this artificially refined fuel into energy using engines that can run on little else. Cut off from refined fuel, airstrips, maintenance crews, spare parts, navigational systems and all the other paraphernalia that keeps it flying and a 747 is just a large piece of scrap metal.

A bird, an adaptable living system which can also fly, can live on berries, seeds, worms, insects, small rodents, fish and bits of bread tossed to it by amused tourists. Its remarkably adaptable digestive system can convert all these and more into energy and essential raw materials for power and self repair. It thrives in the complex and ever changing natural world.

How many people have heard of the broadcast architecture? [only a few hands go up] In biological self replication, each cell has a complete set of blueprints in its DNA. When a cell divides, it makes a copy of these blueprints so that each new cell can have its own complete copy. In the broadcast architecture, the self replicating device doesn't have a copy of its own blueprints. Instead, the blueprints are broadcast to it from an outside source. Each replicating device simply follows the instructions that are broadcast to it. This means the replicating device is simpler: it doesn't need to remember and decode its own instructions. It also means it can be more rapidly redirected to manufacture useful products -- you just broadcast new instructions to all the cooperating devices and tell them to manufacture what you need at the moment. And finally, it is inherently safe in the sense that a single replicating device is unable to replicate by itself. The replicating device can't do anything unless it's constantly given new instructions.

It will be challenge enough to design artificial self replicating manufacturing system able to function in a controlled artificial environment using a single specific source of highly refined energy, let alone in the wild disarray of the natural world where it would have to adapt to whatever came its way. The fear that we will accidentally destroy the planet seems remote. Merely economic incentives lead us towards designs that are simple, inflexible, and highly efficient in a very specialized environment.

I do not expect that everyone will immediately agree with this assessment. The point is not to resolve this issue today, but that we, as a society, need a mechanism for resolving this and other complex issues. We can hardly decide on a wise course of action when there are still fundamental disagreements about the basic nature of the technology whose development we are trying to guide. We need a research community actively investigating the design of artificial self replicating manufacturing systems, such as assemblers, whose members can provide us with multiple perspectives and a wide range of views: views developed over years and decades of careful investigation of such systems.

Accidents are only one cause for concern. A greater cause for concern is deliberate abuse. New technologies are used to make new weapons. A fundamental question raised by new technologies is: does it favor defense or offense? Castles favored defense: they made it easier to block attack. Nuclear weapons favored offense: they made it easier to attack and harder to prevent attack.

Will nanotechnology favor offense or defense? This is a complex question which I and others have considered for decades. To the best of my knowledge, there is as yet no clear answer. From a practical point of view, the answer to this question matters a great deal. If nanotechnology favors defense, then we should learn and understand how to make such defensive systems as rapidly as possible. If nanotechnology favors offense, then we need to know this unwelcome news as soon as possible so that we can start seriously investigating the issues surrounding arms control: proliferation, verification, and the like.

There is an important area of research which needs to be pursued, but which is not being pursued today. That is the theoretical and computational investigation of artificial self replicating manufacturing systems in general and assemblers in particular. If we are to gain some collective insight into where technology might reasonably go over the course of the next 20 years, we must use theoretical and computational tools. Experimental research tells us what we can do, and what we might be able to do in the next few years. But it cannot address the longer term issues.

What, then, should we do? Pursue an active research program, coupled with informed public discussions to guide public policy towards the safest courses of action.