The final meeting of the Group of Governmental Experts (GGE) on Lethal Autonomous Weapons Systems (LAWS) took place in Geneva from 31 August to 4 September under the Convention on Certain Conventional Weapons (CCW).
Some of the leading figures in AI are warning that the technology is moving too fast and far beyond human control. This is seen acutely when it comes to AI in the military. Experts have developed ideas for a LAWS treaty to limit those risks, but a handful of powerful states have stepped in to try to derail those efforts behind closed doors.
After three years of negotiations, governments have agreed a framework that could provide a starting point for future talks on a legally binding instrument on autonomous weapons. But significant progressive elements – put forward during those negotiations – have been stripped out, leaving a document that is far less ambitious than many states and civil society organisations had hoped. It was the last opportunity for governments to shape the GGE’s proposed “set of elements” before the CCW’s Seventh Review Conference in November, when states will decide whether to move towards formal negotiations.
The timing could hardly be more significant. Autonomous weapons technology is developing at extraordinary speed, while the international rules governing its use remain largely unchanged. Systems capable of operating with increasing degrees of autonomy are being developed, tested and, according to growing reports, deployed in armed conflicts. This means that the debate is no longer about a distant technological possibility. It is about establishing rules for technologies that are already changing the way warfare is conducted.
The implications are profound. When machines are given increasing responsibility for identifying and engaging targets, questions arise not only about compliance with international humanitarian law, but also about accountability, ethics and human control. Ultimately, the issue is whether decisions involving life and death can be delegated to machines without undermining fundamental principles of human responsibility.
The urgency of the situation was highlighted shortly before the Geneva meeting when UN Secretary-General Antonio Guterres and Mirjana Spoljaric Egger, President of the International Committee of the Red Cross, made a joint appeal for governments to stop moving incrementally and take decisive steps towards regulation. They called for negotiations on a legally binding instrument containing strong prohibitions and restrictions on autonomous weapons, warning that failing to act quickly would have consequences for human life.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines. Innovation must serve humanity – not endanger it.”
António Guterres, UN Secretary-General
Geneva negotiations
There are a number of states ready to heed the call of the SG. Seventy-six states have now indicated their support for beginning negotiations, including 70 High Contracting Parties to the CCW. That represents a majority of the Convention’s voting members and marks an important increase in political support for an international treaty.
This momentum was visible from the outset of the Geneva meeting. Algeria, speaking on behalf of the African Group, and Brazil, representing a group of 47 states, both expressed support for moving towards negotiations on the basis of the work undertaken by the GGE. Civil society organisations, including Stop Killer Robots Global who made a statement from the floor, noted the significant progress and areas of convergence reflected in the draft report of the group.
The work of the GGE was guided by its Chair, Ambassador Robert in den Bosch of the Netherlands, who worked to preserve the progress made during the negotiations and to carry it forward into the final text. Among the most significant developments was a clearer working understanding of autonomous weapons and the measures that could be required to retain meaningful human control over their use. The draft also recognised the importance of human judgement and the principle that responsibility for the use and consequences of weapons cannot simply be transferred to a machine.
This element of progress came under increasing pressure during the final days Several highly militarised states, including the United States and Russia, challenged provisions that had previously attracted broad support. As negotiations entered their final stages, language that had been debated, amended and refined over a number of years was reopened. The result was a substantial watering down of the document.
The final version removed all references to responsibilities across the production and transfer chain of autonomous weapons. Key requirements relating to the design, development and use of these systems, including the need for explainability, predictability, reliability and traceability were also dropped. Notably, even the design and development of autonomous weapons were removed from the definition of what constitutes such a system. Ethical considerations were also omitted, while the scope of the law that would apply was significantly restricted, shifting from international law as a whole to international humanitarian law alone.
These omissions are significant because they concern precisely the characteristics that determine whether meaningful human control is possible. A weapon whose behaviour cannot be adequately understood, predicted or traced presents very different risks from a system operating within clearly defined human parameters.
Concerns over consensus
There is also a wider concern about the way the negotiations were conducted. Although the GGE’s formal programme lasted five days, governments also held extensive informal discussions during lunch and evening sessions. Civil society, academics and other observers were excluded from these meetings. It is deeply concerning if multilateral negotiations, where important decisions increasingly take place away from public scrutiny and opportunities for meaningful participation by civil society are reduced.
For those working to prevent the development and use of weapons that could make life-and-death decisions without meaningful human control, this is particularly troubling. At the heart of the problem is the CCW’s consensus-based decision-making system.
In theory, consensus is intended to ensure that agreements have broad support. In practice, it can give a small number of states the ability to prevent language supported by a much larger majority from surviving the negotiating process. That is effectively what happened in Geneva.
The outcome illustrates how the interests of a relatively small number of powerful military states can constrain the ambitions of the wider international community. It is particularly important to consider what this means for countries in the Global South, many of which have consistently argued for stronger international rules and for human beings, rather than machines, to remain responsible for decisions involving the use of force. There is a clear lesson from previous humanitarian disarmament agreements. The Convention on Cluster Munitions emerged after efforts within the CCW failed to deliver an adequate response. The Treaty on the Prohibition of Nuclear Weapons followed a similar logic, developing through a process in which nuclear-armed states could not exercise an effective veto over whether negotiations should take place.
These treaties demonstrated that international humanitarian law can advance when the protection of people from unacceptable harm is placed above the preferences of the most powerful military actors. That history should inform the debate over autonomous weapons.
Key takeaways and next steps
The final GGE outcome should therefore be viewed neither as a breakthrough nor as a complete defeat. There are elements worth preserving. The agreement provides a workable characterisation of autonomous weapons, acknowledges the importance of human judgement and control, identifies measures intended to maintain that control, and reinforces the principle that accountability cannot be delegated to a machine. Those achievements provide a platform for further negotiations.
The final document falls significantly short of the breadth of work undertaken during the GGE process and does not provide sufficiently strong prohibitions and restrictions on autonomous weapons. Most importantly, it leaves unanswered the question of which systems should be prohibited altogether because meaningful human control cannot be guaranteed.
The next major test comes in November at the CCW’s Seventh Review Conference. Governments will have to decide whether to establish a mandate for negotiations on a legally binding instrument. The level of support already demonstrated by states suggests that there is a genuine opportunity to move forward.
The states that successfully pushed for weaker language during the GGE will have the same ability to resist progress at the Review Conference. If the consensus system once again prevents governments from agreeing to begin negotiations, states committed to maintaining meaningful human control over the use of force should be prepared to pursue an alternative route. There is no shortage of evidence demonstrating why this matters. The international community has been discussing autonomous weapons for more than a decade. Meanwhile, technological development has continued at pace and the use of increasingly autonomous systems in conflict is becoming a present reality.
The central issue is therefore not whether the technology can be developed. It is whether governments are willing to establish clear limits on how it can be used.
The Geneva meeting has produced a document that is weaker than many had hoped for. But it has also revealed something important: a majority of states are prepared to move towards legally binding negotiations. The November Review Conference offers governments the opportunity to turn years of discussion into a genuine negotiating process. The international community has already spent long enough debating whether autonomous weapons require new rules. The question now is whether governments have the political courage to create them. The future of human control over the use of force should not be decided by the states most invested in removing it.