Throughout the history of war we have witnessed the birth of numerous game changing weapons and technologies. From the sword in the Middle Ages to guns in the European Renaissance. And from the aircrafts and tanks during World War I to nuclear weapons. More recently, remote controlled systems (so called drones) have become part of modern weapon arsenals. We may think we have seen it all, but in the past decade technology, particularly in the field of computing and electronics, has become increasingly advanced. This has resulted in efforts to develop fully autonomous weapons, so called killer robots. Whereas drones still have a human operator, killer robots would search and engage targets without any meaningful human intervention. This means life and death decisions would no longer be made by humans, but, instead, would be made by machines. This is a game-changing technology that would not only change the way how war is fought, but changes for the first time in history the very identity of who fights it.
As with many revolutionary military technologies little is known about the potential risks these robotic weapons pose to humans. Nuclear weapons and drones have already triggered a great deal of discussion, but only after they were developed and used. Although some precursors of killer robots (mostly fixed base autonomous defensive weapons) are already being used, the discussion about the potential risks these weapon systems pose is still in the preliminary stages. In May 2011 IKV Pax Christi published a report on drones that also covered the issue of fully autonomous weapons. 1 In November 2012 Human Rights Watch and Harvard Law School issued the report ‘Losing Humanity’ that fueled the global discussion. 2 Just weeks after ‘Losing Humanity’ came out the U.S. Defence Department published a policy on autonomous weapon systems. 3 Even though the policy stated that autonomous weapons will only be used to deliver non-lethal force, this policy does not completely rule out the development and use of autonomous weapons in the future.
Campaign to Stop Killer Robots
Many NGOs recognized the problem fully autonomous weapons pose and shared their concerns during a meeting in New York in October 2011. Consequently, they decided to work together to start an international campaign to stop the development, production and use of fully autonomous weapons. In April 2013, the international coalition of NGOs officially launched the Campaign to Stop Killer Robots in London. In May 2013 UN Special Rapporteur Christof Heyns presented his report on Lethal Autonomous Robotics (LARs) to the Human Rights Council in Geneva. 4 The Campaign calls on all states to implement the recommendations of this report and to start national and international talks about the issue. As a result, an impressive number of states has already spoken out in the Human Rights Council in Geneva and the UN First Committee in New York, but also in national fora the discussions have started. 5 Generally, all states recognize the pressing concerns and all states seem open to further negotiations.
These weapons raise various ethical, moral and legal questions. Do we want to delegate the power over life and death to machines?
Objections and Concerns
Proponents of killer robots generally emphasize that by deploying fully autonomous weapons we save military lives and we reduce the human workload required to operate systems and thereby reduce costs. Proponents seem to focus mostly on the anticipated military advantage, thereby failing to acknowledge the dangers these weapon systems could pose to civilians. We believe however that these weapons raise various ethical, moral and legal questions. Most importantly, we believe we should ask ourselves an ethical question: do we want to delegate the power over life and death to machines? UN Special Rapporteur Christof Heyns stipulates the urgency of this question in his report on LARs:
“Even if it is assumed that LARs [Lethal Autonomous Robotics] could comply with the requirements of IHL [international humanitarian law], and it can be proven that on average and in the aggregate they will save lives, the question has to be asked whether it is not inherently wrong to let autonomous machines decide who and when to kill. […] If the answer is negative, no other consideration can justify the deployment of LARs, no matter the level of technical competence at which they operate.” 6
According to the Campaign to Stop Killer Robots there should always be a human in the loop that exercises meaningful control over the decision to use force against a human target. Without this, it seems that no one can be held sufficiently accountable for violations of international law caused by the robot. It should thus not come as a surprise that killer robots are generally perceived as inherently abhorrent. Or as Peter Singer, political scientist and one of the world’s leading experts on changes in 21st century warfare, points out: “It [War] is about human suffering, about loss of human lives and consequences for human beings”. 7 Therefore, killing with machines is the ultimate demoralization of war. Even in the hell of war we find humanity, and that must remain so. 8
Besides ethical questions there are also legal questions that need urgent discussion. In order to protect civilians in armed conflicts, killer robots will need to be programmed to act accordingly international humanitarian law (hereinafter: IHL) and international human rights law (hereinafter IHRL). More specifically, fully autonomous weapons must, at least, be capable to discriminate between civilians and combatants and they must be able to weigh the harm done to civilians and civilian objects against the expected military gain. These rules are called the principle of distinction and the principle of proportionality and they must be assessed on a case-by-case basis time and time again. 9
Although some proponents, such as roboticist and Professor at the Georgia Institute of Technology Ronald Arkin, argue that machines may be as good as humans at discrimination within the next 20 years, the more general belief is that it will be highly unlikely that killer robots can be designed in a way that they would be able to comply with IHL and IHRL in the same way as humans can. However, these principles require a case-by-case approach and a thorough and complex analysis of the context, motives and intention of the actors. These principles, in particular the principle of proportionality, are perceived as the most difficult rules of international law to understand and apply. Particularly in contemporary warfare where it has become increasingly difficult to discriminate between civilians and combatants. Hence, it is difficult to imagine how to develop software coding in order to effectively frame the robot’s behavior. According to the International Committee of the Red Cross developing an autonomous weapon system that can implement IHL represents a monumental programming challenge that may well prove impossible. 10 It is therefore that Noel Sharkey, Professor of Artificial Intelligence and Robotics at the University of Sheffield and co-founder of the International Committee for Robot Arms Control (ICRAC), stresses “It is humans, not machines, who devised the laws of war and it is humans, not machines, who will understand them and the rationale for applying them”. 11
Debate on killer robots is necessary so that the international community can develop legislation to prevent these weapons from causing grave humanitarian suffering.
Not only could these weapons cause grave humanitarian suffering when deployed, but also in the preliminary stages of war these weapons will likely have profound effects on decision making. Killer robots will make it easier for leaders to go to war. On the one hand, killer robots might save military lives, but on the other hand, in so doing they will increase the distance between the public and the war because no loved ones will return from the battlefield in a body bag. This may sound cruel but body bags play a vital role in warfare. They are a major inhibitor as they can cost politicians their votes. By deploying killer robots the public will turn into passive observers, thereby giving politicians more space in deciding when and how to go to war. This could lead to an increase in duration and amount of conflicts around the globe.
Conclusion
The advantages of killer robots may seem quite logic and straightforward. Killer robots save military lives, they might be cheaper and they do not kill out of revenge or anger. Nevertheless, the lack of emotions in warfare is extremely dangerous and the low-cost of these robots makes them prone to worldwide proliferation. At some point, these weapons will also fight each other and with the wide variety of unknown algorithms the consequences will be unpredictable and devastating. In the above paragraphs we have given you a brief overview of some of the dangers these weapons may cause. Notably, this list is not exhaustive and only aims to give you an idea of the problem. Nonetheless, the message is clear: killer robots need to be discussed so the international community can develop legislation to prevent these weapons from causing grave humanitarian suffering. Fortunately, only seven months after the launch of the Campaign, the Convention on Conventional Weapons (CCW) adopted a mandate to discuss concerns related to “lethal autonomous weapon systems”. Discussions on these weapon systems, also known as fully autonomous weapons or killer robots will be held in May 2014. The killer robots are on the agenda and they will not go away until there is a ban.
1. IKV Pax Christi, Cor Oudes i Wim Zwijnenburg, Does Unmanned Make Unacceptable?, 2011.
2. Human Rights Watch i Harvard Law School International Human Rights Clinic, Losing Humanity – the Case Against Killer Robots, 2012.
3. “Directive 3000.09”, accessed December 9, 2013, Available here.
4. For more information on the report and Heyns’ recommendations see: “Report of the Special Rapporteur on extrajudicial summary or arbitrary executions, Christof Heyns”, available here. accessed December 9, 2013.
5. Since the topic was first debated at the Human Rights Council on 30 May 2013, a total of 44 countries have spoken publicly on fully autonomous weapons. See the link.
6. “Report of the Special Rapporteur on extrajudicial summary or arbitrary executions, Christof Heyns.”
7. Peter W. Singer, “Interview with Peter W. Singer”, International Review of the Red Cross 94, 886 (2012), p. 476.
8. IKV Pax Christi will publish another report on the eight most pressing concerns caused by the development, production and potential use of killer robots. See: Merel Ekelhof and Miriam Struyk, Outsourcing Morality: 8 objections against killer robots, to be published January 2014.
9. According to article 48, 51 and 52 of the Additional Protocol I to the Geneva Conventions all parties to a conflict shall at all times distinguish between civilians and combatants as well as between civilian objects and military objectives and, accordingly, only direct their operations against military objectives. The principle of proportionality is codified in article 51 (5) (b) of Additional Protocol I and repeated in article 57 and basically means that harm to civilians and civilian objects must not be excessive relative to the expected military gain.
10. “International Humanitarian Law and the challenges of contemporary armed conflicts.” International Committee of the Red Cross , accessed December 9, 2013, p.40. Available here.
11. Noel Sharkey, “The Evitability of Autonomous Robot Warfare”. International Review of the Red Cross 94, 886 (2012), p. 796.
Photo: US Air Force. Modified. Link to license.
© Generalitat de Catalunya