Stanford CIS

Robots, Ethics & War

By Patrick Lin on

Related to my work here in robot ethics, the following is an advance look at my paper forthcoming in Journal of Military Ethics:

Military 2.0: Ethical Blowback from Emerging Technologies

ABSTRACT: The military is a major driver of technological, world-changing innovations which, like the Internet, often have unpredictable dual uses and widespread civilian impact (‘blowback’). Ethical and policy concerns arising from such technologies, therefore, are not limited to military affairs, but can have great implications for society at large as well. This paper will focus on two technology areas making headlines at present: human enhancement technologies and robotics, representing both biological and technological upgrades to the military. The concerns we will raise are about (1) the unintended effects that these technologies could have on society and (2) the possibly self-defeating nature of these military programs in making war itself more, rather than less likely to occur. Given that society also can influence military programs (e.g., through civil protests) it is important for the military to attend to such concerns.

The first ethics concern with any new military weapon or system is usually whether its development or use would violate existing international law. However, emerging technologies can also raise much larger worries, given their revolutionary nature. For instance, using robots for dangerous jobs promises to help reduce the number of casualties on the deploying side. But might this effect prove counterproductive to the larger goal of peace, to the extent that these machines also lower the political costs associated with declaring war, and so make engaging in war as a preferred or convenient method of conflict resolution easier rather than more difficult? Toward the same goal of soldier survivability and reducing abuses and accidents, at some point we may be able to eliminate human emotions such as fear or anger or hatred—but must these enhancements (and others, such as super-strength) be temporary or reversible, considering that warfighters usually return to civilian life?

These and other questions not only determine how such technologies might be used by the military, but how they might also impact society at large, should we end up, for instance, entering into ever more armed conflicts because they are ‘risk-free’ or, in quite another sense, if armies of biologically-enhanced and advantaged veterans suddenly flooded back into the civilian workforce. ‘Blowback’ is the term frequently invoked to draw attention to these unintended and largely unanticipated effects that radical or innovative new technologies originally designed for military purposes might come to exert on civilian life. Given that the feedback loop also routes back in the other direction—from social criticism to military changes—the broader impacts of war technologies should be a concern for the military. To that end, this paper will examine the intertwined relationship between the military and society, particularly how the use of human enhancement (also called human performance optimization) technologies and advanced robotics in war may cause a blowback of significant and unintended effects that humanity will need to confront.

Enhancement technologies and robotics are particularly significant, because they represent opposite directions to the same goal of creating a super-warfighter: one is an engineering approach that works with mechanized systems, and the other is a biomedical approach that works with organic systems. Our discussion also applies broadly to other emerging military technologies, such as to both non-lethal and cyberweapons. However, we will focus on robotics and enhancement technologies here, since they more directly impact the human dimension—by either replacing human combatants from the theater of war or upgrading them.

1. The role of the military in society

To appreciate the ethical threads that run through both society and the military, let us first briefly acknowledge the relationship between the two. Since the beginning of human history, we have been engaged in violent conflicts of varying scale, from individual fights to tribal battles to multinational wars. We would like to think that rational individuals as well as societies strongly prefer to live in peace rather than in war, which has terrible costs. Even a purely self-interested person, who may naturally want to gain power and property, can understand that the benefits of peace outweigh the disvalue or ‘incommodities’ of war (Hobbes 1651). But we also recognize that sometimes it may be necessary to go to war, for instance, in self-defense, and that not everyone is rational, at least to our standards. In that regard, a standing military is useful to defend ourselves, even if its historic use includes ignoble purposes such as conquest and empire expansion.

Around 350 BC, Aristotle pronounced, ‘[We] make war that we may live in peace’ (Aristotle, X.7), variations of which are attributed also to great military leaders from Julius Caesar to Carl von Clausewitz. If that is true, then the fates of the military and society are inextricably interwoven. Of course, this relationship also has been disputed: Nobel laureate Albert Einstein insisted, ‘You cannot simultaneously prevent and prepare for war’ (Nathan & Norden 1960). We need not resolve the debate here, since the important point for this discussion is merely that the military plays an influential role in society as a matter of fact. Further, neither Einstein nor others seem to dispute the basic point that the purpose of war is—or ought to be—to achieve peace, as implied by Aristotle, Clausewitz, and others. As Hobbes recognized, society cannot endure in a constant state of war.

In the United States, for example, the relationship between the military and society is substantial. The military and defense-related programs in the US—which ranks first in national military spending, with China a distant second place with about one-seventh of the US spending level—command nearly a third of the entire national budget (Office of Management and Budget 2010; GlobalSecurity.org 2010). The 2010 fiscal year budget for those activities is approximately $1 trillion and accounts for about 40% of US taxpayer dollars (Internal Revenue Service 2010).[1] A significant portion of this funding is directed at research and development (R&D). The Fiscal Year 2010 US budget for R&D is approximately $143 billion, of which about 60% goes toward national defense, not even including related areas such as space technologies, basic science, and energy. In comparison, only 20% of the total R&D budget is earmarked for health-related projects (Office of Management and Budget 2010).

These staggering sums are perhaps understandable: Insomuch as life is a continuous struggle, one of our most basic and greatest needs is security—from the cold and rain, from hunger, and from enemies, both real and perceived. At least for the moment, we have solved many of these insecurities, such as with housing, heating, and food science, and we are making steady progress in medicine. But the intentions and attitudes of our enemies continue to vex us; the consequences of losing a conflict are usually dramatic; and the temptations of power are most difficult to resist. For many, these and other influences make heavy investments in military R&D seem not only sensible but also urgent and necessary.

Ethical and policy issues arising from the use of military force, or from the development of new military technologies, can also spill over to the civilian realm. This is the part of the feedback loop between military and society on which we will primarily focus. A second part of the feedback loop, however, is that society can also effect changes in the military: Public opinion is a significant force and can manifest itself in several forms, whether these be anti-war protests (e.g., protesting the U.S.-led war in Iraq, or the detention and interrogation of suspected terrorists at Guantanamo Bay Naval Base), or significant public policy movements (such as the grassroots movement to effect an international ban against the production and use of landmines by the United Nations in 1997), or, at present, to regulate the use of unmanned aerial vehicles (UAVs) or military robotic aircraft (Bowcott 2010). Though important, we will not be concerned with the dynamics of this second part of the feedback loop here.

2. Military technologies

It would be difficult to imagine our modern life apart from the numerous inventions originally inspired, for better or worse, primarily for military purposes. For instance, it is well known that the Internet has deep roots in the military’s ARPANET, a packet-switching network developed in the 1960s by the US to allow communications in the event that a nuclear attack disabled the telephone and telegraph circuit-switching networks that were then the principal means of communication. The global positioning system (GPS) was also created by the US Department of Defense in the 1970s as a necessary improvement on navigation systems, especially for ballistic missiles, and is now integrated into automobiles, mobile phones, and other computing devices worldwide. In fact, the ubiquitous computer itself has origins in military necessity: Germany’s Z3 (1941), Britain’s Colossus (1943), the Harvard Mark 1 (1944), ENIAC (1946), and others performed number-crunching operations that humans could not easily or ever do, such as aerodynamics analysis, code breaking, thermonuclear calculations for the design of the hydrogen bomb, and more.

The list of other technologies invented or greatly improved upon by the military is long, but to mention a few more here: communications technologies, including walkie-talkies and radio telephony (to contact ships and airplanes from land), as some of the first attempts at mobile or wireless communication; nuclear power, as introduced by nuclear weapons, both fission and fusion; microwave power, which was accidentally discovered by a Raytheon engineer working on an active radar set; radar, which comes from the US Navy’s acronym RADAR, or radio detection and ranging, but developed in the 1930s by several nations independently and in great secrecy; medicine, including the practice of triage, surgical advances, and prosthetics; and gunpowder, invented in ancient China. And, of course, the military extensively uses and enhances vehicles for land, air, sea, and space, including rockets and missiles. For instance, the first automobile is believed to have been a French military tractor created in 1769, and several auto companies today can trace their lineage to military operations, such as BMW, Saab, and Jeep.

Whether or not these technologies would have been invented or advanced anyway, without military research, is of course a matter of considerable historical debate. Notwithstanding, for the purposes of this paper, it is enough that we acknowledge that society today is, in fact, very much shaped by technologies from the proving grounds of the military. So to that extent, we can expect new and emerging military technologies to make their way into society. And where technology gives rise to challenges that society must confront—from mutually assured destruction (MAD) with nuclear weapons to intellectual property, privacy, ‘cyberbullying’, and many other issues with the Internet—by looking at military research today, we can begin to identify and work through the ethical and policy challenges they will bring tomorrow.

Today, we find a broad range of inventive and strategic projects funded by the US military, such as: cyborg insects; robots that ‘eat’ (that is, derive energy from ingesting local waste products, or even human corpses constituting battlefield casualties); energy weapons; telepathic communication; quantum computing; submersible aircraft; exoskeletons; enhanced warfighters; dynamic armor; invisible shields and cloaks; accelerated therapeutics; real-time language translations; and programmable matter (DARPA 2010a, 2010b). For the sake of brevity, as mentioned earlier, we will focus only on human enhancement and robotics technologies in the analysis that follows.

2.1 Human enhancement

Broadly understood, human enhancement or optimization is about endowing an individual with an ability that goes beyond the typical level or statistically normal range of functioning for Homo sapiens (Daniels 2000). For instance, anabolic steroids may give an individual greater strength and speed than most others (at least of the same size, build, etc.), thereby enabling the competitor to break athletic records. We may also understand human enhancement to be distinct from therapy, which is about treatments aimed at pathologies that compromise health or reduce one’s level of functioning below this species-typical or statistically normal level (Juengst 1997). Thus, anabolic steroids given to a muscular dystrophy patient would be a form of therapy and not human enhancement, since the goal is to help return the patient to a level of strength that he otherwise would have had without the affliction.

And while many instances of human enhancement and therapy may be biomedical in nature, e.g., Ritalin or modafinil which is used by patients with attention-deficit hyperactivity disorder (ADHD) as well as by otherwise-healthy students who want to focus more intently on their studies, some may be technological, e.g., the same carbon-fiber legs given to South African runner Oscar Pistorius could also be surgically attached to a healthy runner looking for a competitive edge. This suggests another distinction. Human enhancement technologies are not just mere external tools, such as a desktop computer or binoculars. Instead, they are either integrated into the body or so closely worn or connected that they confer a similar 24/7, ‘always-on’ advantage as an internal or organic enhancement that transforms the person.

As critics have noted, however, these and other conceptions of human enhancement may have definitional exceptions. For instance, would vaccinations count as (preventive?) therapy or an enhancement to the immune system? There are good reasons to support either answer, suggesting that the line between enhancement and therapy is blurry (Bostrom & Roache 2009). We will not address this continuing controversy here, but let us proceed on the assumption that ‘human enhancement’ and ‘human optimization’ are intelligible concepts and delineate at least some technological or biomedical applications from their therapeutic counterparts, including the ones that the military categorizes as such.

With that understanding, the US military supports a host of research projects designed to optimize or enhance the warfighter, including (DARPA 2010b, unless otherwise noted):

While these and other projects may currently be in a research phase or involve devices too large to integrate with the human body, technologies such as nanoelectronics are shrinking the size of computers, making it foreseeable that these devices eventually can be embedded or wearable, thus falling in the domain of human enhancement, as opposed to mere tools. Innovations in the civilian sector may have dual-use potential, serving also military purposes, such as an electronics-packed contact lens that can acts as a digital display for augmented reality (Parviz 2009), in addition to advanced prosthetics and pharmaceuticals already mentioned, such as carbon-fiber legs and Ritalin.

2.2 Robotics [2]

Robotics too poses a definitional challenge, as obvious as a definition might seem to the layperson. There is still a lack of consensus among roboticists on how they define the object of their craft. For instance, an intuitive definition could be that a robot is merely a computer with sensors and actuators that allow it to interact with the external world; however, any computer that is connected to a printer or can eject a CD might qualify as a robot under that definition, yet few roboticists would defend that implication. A plausible definition, therefore, needs to be more precise and distinguish robots from mere computers and other devices.

As with human enhancement, we do not presume we can resolve this great debate here, but it is important that we offer a working definition. In its most basic sense, we understand ‘robot’ to be an engineered machine that senses, thinks, and acts: ‘Thus a robot must have sensors, processing ability that emulates some aspects of cognition, and actuators. Sensors are needed to obtain information from the environment. Reactive behaviors (like the stretch reflex in humans) do not require any deep cognitive ability, but on-board intelligence is necessary if the robot is to perform significant tasks autonomously, and actuation is needed to enable the robot to exert forces upon the environment. Generally, these forces will result in motion of the entire robot or one of its elements, such as an arm, a leg, or a wheel’ (Bekey 2005).

This definition does not imply that a robot must be electromechanical; it leaves open the possibility of biological robots, as well as virtual or software ones. But it does rule out as robots any fully remote-controlled machines, since those devices do not ‘think’, e.g., many animatronics and children’s toys. That is, most of these toys do not make decisions for themselves; they depend on human input or an outside actor. Rather, the generally accepted idea of a robot depends critically on the notion that it exhibits some degree of autonomy or can ‘think’ for itself, making its own decisions to act upon the environment.

Thus, the US Air Force’s Predator unmanned aerial vehicle (UAV), though mostly teleoperated by humans, makes some navigational decisions on its own and therefore would count as a robot. By the same definition, the following things are not robots: conventional landmines, toasters, adding machines, coffee makers, and other ordinary devices. Weaponized machines such as TALON SWORDS and MAARS also do not count as robots, properly understood, since they do not seem to have any degree of autonomy, as their manufacturer admits (Qinetiq 2010).

War robots with fierce names, e.g., Harpy and Global Hawk, currently perform a range of duties in the military theater, such as spying or surveillance (air, land, underwater, space), defusing bombs, assisting the wounded, inspecting hideouts, and attacking targets—in other words, they perform the ‘three Ds’: jobs that are dull, dirty, or dangerous. Those used or under development by the US military include the following (US Dept. of Defense 2009, unless otherwise noted):

Other nations are already using sentry robots to guard borders, for instance, in Israel and South Korea. These robots will challenge trespassers that they encounter and, if proper identification is not provided, can shoot them (Telegraph editors 2010). We can also expect space robots that serve military purposes in the future, if they are not already now (Singer 2009). And, intersecting with human enhancement, research with robot-human integrations or cyborgs can be seen today with advanced prosthetics, such as a bionic knee (Zamiska 2004), so we can expect that to increase in the future as well.

Again, human enhancement and robotics can be considered as two different approaches to the same goal of creating a better warfighter, one from a biological starting point and the other from a mechanized one. And this makes their ethical issues of special interest, to the extent that they are about an important human dimension of war, as discussed next.

3. Some issues

Military technology ethics is not a new area of inquiry. For instance, in 1139, Pope Innocent II banned the use of crossbows, calling them ‘hated by God’, because they were so devastating, even by an untrained fighter, against the powerful, noble, and revered knight in plate armor (Headrick 2009). The ban, as we know, was ignored. In 1215, article 51 in the Magna Carta singled out crossbows as threats to society: ‘As soon as peace is restored, we will banish from the kingdom all foreign-born knights, crossbowmen, sergeants, and mercenary soldiers who have come with horses and arms to the kingdom’s hurt’ (Constitution Society 2010). The crossbow, apparently, was a moral abomination that made warfare into a depersonalized and dishonorable ‘point-and-click’ affair, disrupting the existing code of ethics and balance of power.

With respect to human enhancement and robotics, some questions already have been raised in the areas of risk, policy, ethics, and philosophy. What, for example, are the health risks of military human enhancements such as steroids, Ritalin, and anthrax vaccines (Allhoff et al. 2009)? What is the safety risk of autonomous and armed robots, given reports of accidental deaths (Shachtman 2007) and wayward UAVs veering toward the US capital (Bumiller 2010)? How do military robots force us to reconsider policy, e.g., the requirement to have ‘eyes on target’—need they be human eyes (Lin et al. 2008)? Should consent be needed for military human enhancements, such as anthrax vaccines (Cummings 2002)? Do we become less than human with some forms of human enhancement, e.g., drugs that can remove emotions such as fear (President’s Council on Bioethics 2003)?

While there is much literature on the ethics of human enhancement (e.g., Savulescu & Bostrom 2009; Sandel 2007; Garreau 2005; Mehlman 2009), little discussion exists within a military context (e.g., Moreno 2006; Wolfendale 2008). The literature on robot ethics is growing, a good deal of which is about military applications (e.g., Sharkey 2007; Asaro 2008; Sparrow 2007; Borenstein 2008). As I mentioned above, the two ‘big picture’ issues that possibly arise from military human enhancement and robotics, which are also relevant to other technologies, are: (1) an unintended spillover of effects from military to society, given the close link between the two, and (2) the possibility of exacerbating a culture of war, which might prove contrary to peace as the goal of war in the first place.

3.1 Civilian blowback

Society can be affected by military technologies in a number of ways. As it evolved from the military’s original ARPANET, the Internet was adopted and built upon by civilian users, ushering in ubiquitous functions from email to marketing and shopping as we know them today. GPS did not need to evolve, but was simply incorporated into computing devices for civilian purposes, such as traffic navigation and location-based services. Military Jeeps and Humvees became trendy for the everyday driver. Nuclear weapons have fortunately not yet been adopted in any meaningful sense by non-military users, but they continue to cast a shadow of the threat of horrific harm across society, and this anxiety and fear have economic, political, and social costs.

Human enhancement technologies, however, may infiltrate society in an altogether different way. Depending on how they are implemented, enhancements that change the human warfighter’s body or mind—whether some drug or, in the future, a neural implant or microcomputer embedded in one’s head—ought to be a concern, because every warfighter is a potential veteran, returning to society from their military service. Approximately 23 million veterans today live in the US, accounting for about one in 10 adults, plus three million active and reserve military personnel (US Dept. of Veteran Affairs 2010). Thus, the civilian blowback from military human enhancements may be significant and direct, given the size of the veteran population alone, even beyond the blowback that might be anticipated from adoption or development of dual-use capacities of such technologies by other civilians for non-military purposes.

For instance, suppose that some warfighters have been enhanced with prosthetics or gene therapy or pharmaceuticals that gave them super-strength. Assuming that many of these individuals will eventually return to the civilian workforce, we would suddenly be faced with a new class of individuals in society with that unique ability (again depending on specific details of the technology’s implementation), representing a competitive advantage that may be disruptive to society (e.g., by displacing other civilian workers or skewing athletic contests). Other enhancements may raise similar issues: A job candidate with a neural implant that enables better data retention, faster information processing, or instant access to Google may consistently beat out unenhanced candidates; a person with super-human hearing or sight could circumvent existing privacy protections and expectations by easily and undetectably eavesdropping or spying on others; more students (and professors) using Ritalin may grab admission or tenure at all the best universities, reducing those opportunities for others; life-extension technologies may bankrupt current pension plans and increase pressure on energy, food, and jobs; and so on (Allhoff et al. 2009).

Thus, the precise manner in which military human enhancements are rolled out or implemented is important to consider, since some plans may impact society more than others. Some of the questions that need to be investigated include [3]:

  1. As a firewall for society, should some enhancements be restricted to only special or elite teams, and which types of enhancements and why? Could and should these elite teams be prohibited from returning to society? Does being enhanced require a change either way in the length of a warfighter’s service commitment? Is the ability to remain enhanced after leaving the service a legitimate recruiting incentive?
  2. Must the enhancements be reversible? If so, should they be reversed routinely upon discharge? Should the warfighter be able to refuse to have them reversed upon discharge? If the enhancements are not reversed, what effects will they have on the ability of the enhanced warfighter to make a successful return to civilian life, and how can adverse effects be minimized?
  3. What effect will this have on the US Department of Veterans Affairs (VA), insofar as enhancements are service-connected? Legally and ethically, how safe should these technologies be before they are deployed? What are the additional costs expected for VA hospitals to deal with adverse health effects from enhancements?
  4. What kinds of social disruptions can be expected from enhanced warfighters? How do we evaluate issues of fairness, equity, and access (as alluded to in the above scenarios)? Would there be a divide between the enhanced and unenhanced, similar to that between the rich and the poor, and to what effects?

Military robots also pose spillover effects—working their way back home from deployment abroad onto a nation’s own borders and eventually within the nation itself (Calo 2010a). As with Jeeps and Humvees, there seems to be a natural migration path of military robots to civil robots, especially those that would perform the same functions of security and surveillance. Police and security robots today are guarding borders and buildings, assisting in hostage situations, scanning for pedophiles and criminals, dispensing helpful information, reciting warnings, and more (Sharkey 2008). There is also a growing market for home-security robots, which can shoot pepper spray or paintball pellets and transmit pictures of suspicious activities to their owners’ mobile phones (e.g., Wang 2009).

In the future, we can expect both military and civilian robots—again, driven in large part by military research—to be more autonomous, networked, and sensitive (Sharkey 2008). For instance, biometric tools can enable robots to identify individuals by recognizing faces, retina scans, or fingerprints. Molecular sensors can enable robots to sniff out drugs as well as suspects. Surveillance technologies can enable robots to detect weapons at a distance, see through walls, and eavesdrop on conversations. Integration with databases, e.g., financial, medical, driving, and shopping, can enable robots to immediately pull up personal information on individuals they encounter. These scenarios, however, raise challenges to civil liberties and civil law, such as the following:

  1. Would greater powers of surveillance by robots count as searches? Currently in the US, a search usually requires a judicial warrant and therefore has some oversight, and mere surveillance is not considered to be a search (e.g., Katz v. US 1967; US v. Knotts 1983). But a robot that can detect weapons or drugs hidden on a person’s body—which cannot be detected by usual methods, i.e., by the naked human eye—may force us to reconsider the distinction between surveillance and search.
  2. How do such civil robots as well as home robots impact current notions of privacy, if they can be outfitted with surveillance and recording capabilities? Networked robots may allow the government to access homes and other spaces that have been historically protected, and humans may be more likely to disclose information and grant access to an anthropomorphic robot than to other technologies (Calo 2010b).
  3. On the battlefield, militaries are immunized from certain unintended killings of noncombatants, or collateral damage, and weapons manufacturers also are released from liability to a great extent; but in civil society, those immunities derived from international law, e.g., Geneva Conventions, would no longer apply. So who would be liable for malfunctions, errors, and unpredictability with civil security robots, from accidentally running over children to firing upon misidentified individuals?
  4. While replacing human soldiers with robotic ones seems to be desirable given potential (friendly) lives saved, replacing human workers in society with robotic ones may be more problematic, if those humans cannot find other work. So what is the economic impact of advanced robotics in society, whether in security roles or performing other labor? What human jobs might become obsolete?
  5. What is the environmental impact of a growing robotics industry? In the computer industry, ‘e-waste’ is a growing and urgent problem (e.g., O’Donoghue 2010), given the disposal of heavy metals and toxic materials in the devices at the end of their product lifecycle. Robots as embodied computers will likely exacerbate the problem, as well as increase pressure on rare-earth elements needed today to build computing devices and energy resources needed to power them.

These represent only some of the civilian-blowback issues posed by the emerging technologies. By themselves, they are not arguments that we should not develop robotics and human enhancement technologies. For instance, the Internet created a host of issues related to intellectual property, privacy, information security, brick-and-mortar retailers, and so on, yet these are not persuasive reasons to think that we should never have developed the Internet. Rather, they are simply challenges that society needs to confront, and anticipatory thinking on these issues may help to mitigate negative effects.

3.2 Culture of war

A key issue that does suggest we should not pursue such technologies, however, has already been levied against military robotics. Some critics have argued that because the primary point of military robotics is to remove risk to ourselves in war, this lowers the barriers to starting or engaging in wars in the first place, including social, economic, and political costs from incurring casualties (e.g., Sharkey 2007; Asaro 2008; Sparrow 2007; Borenstein 2008). (A similar charge could be made against human enhancement technologies, though as far as we can tell, this has not yet occurred, perhaps because the advantage gained is incremental and not as dramatic as replacing human warfighters entirely with machines.)

Just-war theory, and jus ad bellum specifically, is premised on the fact that war is a moral disaster and should be engaged in only as a last resort (Walzer 1977). Civil War general Robert E. Lee affirmed this sentiment: ‘It is well that war is so terrible, or we would grow too fond of it’ (Alexander 1907). If war were not so terrible—say, with robots and enhanced warfighters that can more quickly achieve victory with fewer friendly casualties—then we lose an important disincentive to engage in war, so the argument goes. This seems to be supported by human psychology, that we respond to positive and negative incentives, as at least a partial explanation of our motivations and behavior (Skinner 1953). Indeed, classical economics is built on such an assumption, e.g., that demand for a product (or, in this case, for war) will decrease as its price goes up (Marshall 1890).

A problem with this argument, though, is that it may imply that war ought to be as terrible as possible, to more certainly ensure that we engage in it as a last resort (Lin et al. 2008). That is, we should not do anything that makes armed conflict more palatable, which includes reducing friendly casualties, improving battlefield medicine, and conducting any more robotics and human-enhancement research that would make victory more likely and quicker. Taken to the extreme, the argument seems to imply that we should raise barriers to war, to make fighting as brutal as possible, such as using primitive weapons without armor or even acid attacks, so that we would never engage in it unless it were truly the last resort. However, such a position appears counterintuitive at best and dangerously foolish at worst, particularly if we expect that other nations would not readily adopt a policy of relinquishment, which would put our side at a competitive disadvantage. Thus, it is unclear how literally or to what degree we should take the argument—are there good reasons not to make the cost of war as high as possible?

This problem aside, we can agree that weapons proliferation is real: As any given side develops some war technology, others will soon develop the same capability—negating whatever competitive advantage was initially created. For instance, while much of the international media focuses on the US military robots in the Middle East, more than 40 other nations have them as well, including China, Russia, Pakistan, and Iran (Singer 2009; Harris 2009). This proliferation too contributes to a culture of war, continuing a cycle of armed conflicts that can be won with the (temporarily) new technologies and keeping international tensions high.

This is a paradox. No one, we presume, really wants war; rather, we prefer to live in peace. Yet it nevertheless seems sensible in the interim for every nation to develop or acquire offensive weapons, such as tanks, and defensive tools like armor, with the aim of strengthening and protecting itself in an often-hostile world. However, the relentless race for new and better weapons, by making war itself easier to resort to, could hold a lasting, true peace out of reach. This schizophrenic culture of war then raises a number of questions, including the following:

  1. Ought we to develop military robotics and human enhancement technologies at all? Clearly, the technologies offer tremendous benefits on the battlefield, but that is beside the larger point here, which is about rational consistency in the enterprise of warfare. Are there moral limits to such research that we should observe? Even if we agree that, morally speaking, we should not pursue such military research, is it practical to stop—is it too late, having already opened Pandora’s box?
  2. How would the laws of war and rules of engagement need to evolve or be clarified to account for new technologies? For example, do current international laws pertaining to the treatment of prisoners of war also apply to enhanced warfighters who can tolerate greater physical or mental abuse? The Predator UAV pilots in Las Vegas, half a world away from the robots they operate, would seem to be classified as combatants and therefore lawful targets of attack even when they are at home with family, after their working shifts have ended and they have left the military base—does this align with our intuitions, i.e., does the ‘part-time’ character of this job challenge existing rules on who may be targeted and when? This impacts society, in that international law may allow for enemies to attack the US in certain locations such as Las Vegas—bringing the ‘War on Terrorism’ back to American soil.
  3. The use of military robots by the US is seen as cowardly and dishonorable by enemies in the Middle East, and this is expected to fuel resentment, anger, and hatred—and therefore recruitment efforts—by those enemies (Singer 2009; Shachtman 2009). This too appears to be counterproductive to the larger goal of lasting peace. Would enhanced warfighters be viewed the same way, and what effect would this have on mission success? Are there reasons to prohibit militaries from giving warfighters unnatural and inhuman enhancements, such as horns or an otherwise frightening appearance, to instill fear in the enemy, cf. Viking and Japanese samurai helmets?
  4. If people respond to incentives, then it would seem that technologies that give one side a sizable competitive advantage would act as a negative incentive or deterrent for the other side against starting, provoking, or otherwise engaging in armed conflicts. For instance, nuclear weapons seemed to have prevented, at least, total wars if not some conflicts altogether. Other technologies, such as machine guns or Tomahawk cruise missiles, cannot be obviously credited with deterrence, since history shows a continuation of conflicts even with those weapons in play. So under what conditions are technologies effective deterrents—do robotics and human enhancement technologies satisfy those conditions? If they are not good deterrents, then they may further enable the cycle of war.
  5. Relatedly, to the extent that new technologies are not effective deterrents, how might the enemy attempt to negate the effects of enhancements and robotics? We cannot reasonably expect determined enemies to simply throw their hands up and surrender in the face of superior technology or even a superior military, and they are responding in new ways, such as attacking human UAV controllers or pilots, since it is so difficult to attack the UAVs in flight and potentially devastating to allow those UAVs even to leave the base (MSNBC editors 2010). Some enemies may resort to bringing the fight onto US soil, where the Predators are not, and this will have a chilling effect on society (Lin et al. 2008), e.g., if terrorists work harder to obtain and detonate a ‘dirty bomb’ that could contaminate large cities, causing residents to flee.
  6. What are the implications for space? We are on the verge of developing a commercial space industry, the first steps to extending society—giving us the means to leave the planet and perhaps re-colonize elsewhere. This is a long-term goal, and during which time we can expect military activity to increase in space, despite international restraint today. The existence of wars in space, including those in which robots are deployed, would be incredibly destructive to the progression of orderly space settlements and society. These problems may range from creating areas of conflict to more floating debris in space to land-grabs on other planets and moons, notwithstanding existing treaties.

4. Conclusion

The robotics industry is rapidly advancing and already giving rise to a great many additional questions in society of a non-military nature: e.g., whether we are morally permitted to reassign our duty to care for elderly and children to robotic caretakers. The human enhancement industry, likewise, is quickly moving forward and creating new questions for society to confront: e.g., whether cognition-boosting drugs such as modafinil should be allowed in schools or the workplace, echoing the steroids debate in sports. But as discussed above, many of the advances in these fields will come from the defense sector, as the military is often on the frontlines of technology innovation. How these technologies are actually developed and deployed by the military will, in turn, have ethical and policy implications for society, given a tight relationship between the military and society.

We need to attend to these issues now, rather than wait for the technologies to be developed first, as ethics is usually slow to catch up with science and technology, which can lead to a ‘policy vacuum’ (Moor 1985). Recall that the Human Genome Project was started in 1990, but it required the ensuing 18 years for the US Congress to finally pass a bill to protect citizens from discrimination based on their genetic information. Right now, society is still fumbling through privacy, copyright, and other intellectual property issues in the Digital Age, nearly 10 years since Napster and other peer-to-peer file-sharing sites were first shut down.

To be clear, this is not a call for a moratorium or even necessarily limiting the development of the technologies themselves. Military technologies promise great, perhaps irresistible, benefits for not only defense-related activities but also broader society as dual-use technologies. Yet we can already anticipate that they would disrupt current military and social systems. Disruption and change by themselves are not necessarily bad things. But failing to manage change and disruption may lead to negative effects that might have been mitigated with some forethought—including public backlash against military programs as warranted, the second return path of the military-society feedback loop mentioned earlier. Alan Turing, one of the founding fathers of the technological age, foretold of our task: ‘We can only see a short distance ahead, but we can see plenty there that needs to be done’ (1950).

ACKNOWLEDGEMENTS:

This work is supported by grants from The Greenwall Foundation and the US Office of Naval Research. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the funding organizations.

ENDNOTES:

[1] To put this number in perspective, the wealthiest person in the US is currently Bill Gates, co-founder of Microsoft Corporation and the largest philanthropist in world history, whose net worth is estimated at $50 billion (Miller & Greenberg 2009)—which means that, every year, the US spends about 20 times Gates’ entire fortune on defense programs.

[2] Some of the discussion in this section draws from my book in progress, Robot Ethics: The Ethical and Social Implications of Robotics, to be published by MIT Press in 2011 (Lin et al. forthcoming).

[3] Most of these questions arise from recent collaborative efforts and inquiry of the Consortium for Emerging Technologies, Military Operations, and National Security (CETMONS), headquartered at Arizona State University.

REFERENCES:

AeroVironment (2009) DARPA Awards AeroVironment Phase II Contract Extension for Nano Air Vehicle Development Program, accessed September 19, 2010, available at http://www.businesswire.com/portal/site/google/?ndmViewId=news_view&news...

Alexander, E.P. [ca. 1907] (1993) Military Memoirs of a Confederate (New York: De Capo Press).

Allhoff, F., Lin, P., Moor, J. & Weckert, J. (2009) Ethics of Human Enhancement: 25 Questions & Answers, a US National Science Foundation-funded report, accessed September 19, 2010, available at http://www.humanenhance.com/NSF_report.pdf

Aristotle [ca. 350 BC] (1998) The Nicomachean Ethics, Ross, W.D. (trans.), Akrill, J.L. & Urmson, J.O. (eds.), (Oxford: Oxford University Press).

Asaro, P. (2008) How Just Could a Robot War Be?, in: Briggle, A., Waelbers, K. & Brey, P. (eds.) Current Issues in Computing and Philosophy (Amsterdam: IOS Press).

Bekey, G. (2005) Autonomous Robots: From Biological Inspiration to Implementation and Control (Cambridge: MIT Press).

Borenstein, J. (2008) The Ethics of Autonomous Military Robots, Studies in Ethics, Law, and Technology 2(1): article 2, accessed September 19, 2010, available at http://www.bepress.com/selt/vol2/iss1/art2

Boston Dynamics (2010a) LittleDog—The Legged Locomotion Learning Robot, accessed September 19, 2010, available at http://www.bostondynamics.com/robot_littledog.html

Boston Dynamics (2010b) PETMAN—BigDog Gets a Big Brother, accessed September 19, 2010, available at http://www.bostondynamics.com/robot_petman.html

Bostrom, N. & Roache, R. (2008) Ethical Issues in Human Enhancement, in: Ryberg, J., Petersen, T.S., & Wolf, C. (eds.) New Waves in Applied Ethics (New York: Palgrave Macmillan).

Bowcott, O. (2010) Robotic Warfare: Campaigners Call for Tighter Controls of Deadly Drones, The Guardian, accessed September 19, 2010, available at http://www.guardian.co.uk/technology/2010/sep/16/robot-warfare-conferences

Bumiller, E. (2010) Navy Drone Violated Washington Airspace, The New York Times, accessed September 19, 2010, available at http://www.nytimes.com/2010/08/26/us/26drone.html

Calo, R. (2010a) Robots: The Law, podcast interview, accessed September 19, 2010, available at http://www.robotspodcast.com/podcast/2010/09/robots-the-law/

Calo, R. (2010b) Robots and Privacy, in: Lin, P., Bekey, G. & Abney, K. (eds.) Robot Ethics: The Ethical and Social Implications of Robotics (Cambridge: MIT Press, forthcoming), accessed September 19, 2010, available at http://ssrn.com/abstract=1599189

Constitution Society (2010) Magna Carta, accessed September 19, 2010, available at http://www.constitution.org/eng/magnacar.htm

Cummings, M.L. (2002) Informed Consent and Investigational New Drug Abuses in the US Military, Accountability in Research 9:2, pp. 93-103.

Daniels, N. (2000) Normal Functioning and the Treatment-Enhancement Distinction, Cambridge Quarterly of Healthcare Ethics 9, pp. 309-322.

Defense Agency Research Projects Agency (2009) Interview with Jon Mogford, Program Manager, DSO, accessed September 19, 2010, available at http://www.darpa.mil/podcasttranscript_december2009.html

Defense Agency Research Projects Agency (2010a) Ongoing Research Programs, accessed September 19, 2010, available at http://www.darpa.mil/off_programs.html

Defense Agency Research Projects Agency (2010b), Programs (A-Z), accessed September 19, 2010, available at http://www.darpa.mil/dso/programs.htm

Garreau, J. (2005) Radical Evolution: The Promise and Peril of Enhancing Our Minds, Our Bodies—and What It Means to Be Human (New York: Doubleday).

GlobalSecurity.org (2010) China’s Defense Budget, accessed September 19, 2010, available at http://www.globalsecurity.org/military/world/china/budget.htm

Harris, M. (2009) iRobot, Where the Terminator is Coming to Life, The Sunday Times, accessed September 19, 2010, available at http://technology.timesonline.co.uk/tol/news/tech_and_web/article6389337...

Headrick, D. (2009) Technology: A World History (New York: Oxford University Press).

Heary, J. (2010) Top 5 DARPA Projects of 2010, Network World, accessed September 19, 2010, available at http://www.networkworld.com/community/node/57992

Hobbes, T. [ca. 1651] (2009) Leviathan, J. Gaskin (ed.), (New York: Oxford University Press).

Internal Revenue Service (2010) Data Book, 2009 (Washington, DC: US Government Printing Office), accessed September 19, 2010, available at /content/files/pub/irs-soi/09databk.pdf

Juengst, E. (1997) Can Enhancement Be Distinguished from Prevention in Genetic Medicine?, Journal of Medicine and Philosophy 22. pp. 125-142.

Katz v. United States (1967) 389 US 347, accessed September 19, 2010, available at http://laws.findlaw.com/us/389/347.html

Lin, P., Bekey, G. & Abney, K. (2008) Autonomous Military Robotics: Risk, Ethics, and Design, a US Department of Defense/Office of Naval Research-funded report, accessed September 19, 2010, available at /content/files/onr_report.pdf

Lin, P., Bekey, G. & Abney, K. (forthcoming) Robot Ethics: The Ethical and Social Implications of Robotics (Cambridge: MIT Press).

Marshall, A. [ca. 1890] (2006) Principles of Economics (New York: Cosimo).

Mehlman, M. (2009) The Price of Perfection: Individualism and Society in the Era of Biomedical Enhancement (Baltimore: Johns Hopkins University Press).

Miller, M. & Greenberg, D. (eds.) (2009) The Richest People in America in Forbes.com, accessed September 19, 2010, available at http://www.forbes.com/lists/2009/54/rich-list-09_William-Gates-III_BH69....

Moor, J. (1985) What Is Computer Ethics?, Metaphilosophy 16:4, pp. 266-75.

Moreno, J. (2006) Mind Wars (New York: Dana Press).

MSNBC editors (2010) Al-Qaida: CIA Attack Avenges Drone Strikes, MSNBC.com, accessed September 19, 2010, available at http://www.msnbc.msn.com/id/34737634/

Nathan, O. & Norden H. (1960), Einstein on Peace (New York: Avenel Books).

Northrop Grumman (2010) Human-Aided Optical Recognition/Notification of Elusive Threats (HORNET), accessed September 19, 2010, available at http://www.es.northropgrumman.com/solutions/hornet/

O’Donoghue, A.J. (2010) E-Waste is a Growing Issue for States, Deseret News, accessed September 19, 2010, available at http://www.deseretnews.com/article/700059360/E-waste-is-a-growing-issue-...

Office of Management and Budget (2010) Budget of the US Government: Fiscal Year 2010 (Washington, DC: US Government Printing Office), accessed September 19, 2010, available at http://www.gpoaccess.gov/usbudget/fy10/pdf/fy10-newera.pdf

Parviz, B. (2009) Augmented Reality in a Contact Lens, IEEE Spectrum, September 2009, accessed September 19, 2010, available at http://spectrum.ieee.org/biomedical/bionics/augmented-reality-in-a-conta...

Plato [ca. 380 BC] (2004) Republic, Reeve, C.D.C (trans.), (Indianapolis: Hackett Publishing Company).

President’s Council on Bioethics (2003) Beyond Therapy: Biotechnology and the Pursuit of Happiness (Washington, DC: US Government Printing Office), accessed September 19, 2010, available at http://bioethics.georgetown.edu/pcbe/reports/beyondtherapy/beyond_therap...

Qinetiq (2010) TALON Family of Military, Tactical, EOD, MAARS, CBRNE, Hazmat, SWAT and Dragon Runner Robots, accessed September 19, 2010, available at http://foster-miller.qinetiq-na.com/lemming.htm

Sandel, M. (2007) The Case Against Perfection: Ethics in the Age of Genetic Engineering (Cambridge: Belknap Press).

Savulescu, J. & Bostrom, N. (2009) Human Enhancement (New York: Oxford University Press).

Shachtman, N. (2007) Robot Cannon Kills 9, Wounds 14, Wired.com, accessed September 19, 2010, available at http://blog.wired.com/defense/2007/10/robot-cannon-ki.html

Shachtman, N. (2009) Call Off Drone War, Influential US Advisor Says, Wired.com, accessed September 19, 2010, available at http://www.wired.com/dangerroom/2009/02/kilcullen-says/

Sharkey, N. (2007) Automated Killers and the Computing Profession, Computer 40, pp. 122-124.

Sharkey, N. (2008) 2084: Big Robot is Watching You, a commissioned report, accessed September 19, 2010, available at http://staffwww.dcs.shef.ac.uk/people/N.Sharkey/Future%20robot%20policin...

Singer, P. W. (2009) Wired for War: The Robotics Revolution and Conflict in the 21st Century (New York: Penguin Press).

Skinner, B.F. (1953) Science and Human Behavior (New York: The Free Press).

Sparrow, R. (2007) Killer Robots, Journal of Applied Philosophy 24:1, pp. 62-77.

Telegraph editors (2010) South Korea Deploys Robot Capable of Killing Intruders Along Border with North, Telegraph, accessed September 19, 2010, available at http://www.telegraph.co.uk/news/worldnews/asia/southkorea/7887217/South-...

Turing, A. (1950) Computing Machinery and Intelligence, Mind 59:236, pp. 434–460.

UC Irvine (2009) MURI: Synthetic Telepathy, accessed September 19, 2010, available at http://cnslab.ss.uci.edu/muri/research.html

United Nations (1997) Convention on the Prohibition of Use, Stockpiling, Production, and Transfer of Anti-Personnel Mines and on their Destruction, ratified in 1997, accessed September 19, 2010, available at http://www.un.org/Depts/mine/UNDocs/ban_trty.htm

United States v. Knotts (1983) 460 US 276, accessed September 19, 2010, available at http://laws.findlaw.com/us/460/276.html

US Department of Defense (2009) FY2009-2034 Unmanned Systems Integrated Roadmap (Washington, DC: US Government Printing Office), accessed September 19, 2010, available at http://www.acq.osd.mil/psa/docs/UMSIntegratedRoadmap2009.pdf

US Department of Veteran Affairs (2010) VA Stats at a Glance, accessed September 19, 2010, available at http://www1.va.gov/VETDATA/Pocket-Card/4X6_summer10_sharepoint.pdf

Walzer, M. (1977) Just and Unjust Wars: A Moral Argument with Historical Illustrations (New York: Basic Books).

Wang, V. (2009) Home Security Bots, Popular Science, accessed September 19, 2010, available at http://www.popsci.com/gear-amp-gadgets/article/2009-05/home-security-bots

Wolfendale, J. (2008) Performance-Enhancing Technologies and Moral Responsibility in the Military, American Journal of Bioethics 8:2, pp. 28-38.

Zamiska, N. (2004) Bionic Knee ‘Learns’ How to Walk, The Wall Street Journal, accessed September 19, 2010, available at http://online.wsj.com/ad/article/philips/SB108907039283655627.html

Published in: Blog , robots , human enhancement , war , peace , ethics , Drones , Robotics