• IT Manager
• Senior Project Manager - Melbourne
• National Delivery Manager
• Chief Information Officer
• IT MANAGER

The ethics of making robots that can kill

Mike Bowern, Information Age

15/02/2007 12:14:35

In his column in The Australian newspaper in January, ACS President Philip Argy wrote about developed by Samsung in South Korea, to protect that country's northern border from possible attack. I provided some comments on the ethical aspects of the robot story in The Australian, and this article expands on those points.

The background to the story is that Samsung's Techwin division, jointly with a South Korean university, has developed an Intelligent Surveillance and Security Guard Robot to replace human guards, particularly for use on the northern border of South Korea.

This static robot is fitted with a daytime camera (range 4km), and an infra-red device (range 2 km) for night vision; a laser range finder; pattern recognition technology to identify human targets; loud speakers and voice recognition features for recognising passwords; and a number of options for weapons, both lethal and non-lethal. It can operate autonomously, or with human control with the operator using a joystick and touch screen control. This is considered to be the first device that can fire on human targets autonomously.

Several of these Information Age columns have, as a theme, a discussion of the role that the ACS Code of Ethics can play in day-to- day issues which arise in the ICT industry. In this story there are a range of ethical issues related to the manufacture of a robot with the capability to kill autonomously; and the professionalism of the engineers designing the product.

The first ethical issue relates to the robot product and the company that makes and sells it. Should such a robot have ever been made? The publicity about these guard robots produced by Samsung suggests that the robots are superior to human guards because they will not be affected by fatigue or inclement weather. However, a human soldier can take responsibility for his or her actions and can choose not to open fire if there is any doubt about the person being challenged.

By contrast, a robot has no sentient reasoning and can only take the course of action for which it has been programmed. That programmed course of action may require the robot not to shoot to kill, or even to open fire, but that course of action is hardly a deterrent to an invader. Also, the robot can been fitted with several types of offensive weapon, so there is a good chance that they will be used.

There may be a strong case for the Samsung robots to be used as border guards to defend South Korea from invasion from the north, but we cannot be certain that this will be their only use. Samsung is a large company, with the usual desire to grow bigger, make profits, and so on. What will it do if other countries, companies or affluent private citizens want to buy these robots to guard their borders, factory premises or personal estates? I expect Samsung would meet these market demands.

With the proliferation of these robots and their use in a variety of situations, what are the chances of a malfunction not resulting in a truly fail-safe operation? What if a child or other individual is mistakenly taken to be a threat and fired upon? Who would be responsible for the death of an innocent person in such situations?

In 1979 computer ethicist James Moor wrote a paper asking the question: "Are there decisions computers should never make?" Moor argues that the root concern about increasing computerisation should focus on the issue of responsibility. In this case, the Samsung robots would not be legally or morally responsible for their decisions.

So who would be responsible - the human guards working with the robots; the maintenance technicians; the owners of the robots; Samsung Corporation as a whole; or the designers, developers and testers of the robot? Bring on the lawyers!

Another issue is the quality of the final product. With software and hardware defects common in many commercial applications, the potential for an undetected fault or software defect to influence the robot's performance must be considered.

I understand that parts of Samsung have been assessed to CMM level 5 and other areas use recognised quality techniques like Six Sigma, which could indicate that the company has processes in place to design, manufacture and test a reliable product. However I could find no information that these quality practices are used in the Techwin division of Samsung.

Nor could I find information about the experience of the engineers and academics actually involved in designing and building this robot, nor the risk management practices they are following to identify potential problems and include fail-safe functions.

Also, we do not know if the designers and developers subscribe to a code of ethics or code of professional conduct such as the ones that ACS members are required to follow. We do know that South Korea has the Government-operated Korea Information Science Society (KISS), which appears to have a very technical focus with little, if any, attention being given to professional or ethical issues. KISS is not an autonomous professional association like the ACS, and little information is available from the English version of the KISS Web site or that of the Korean Ministry of Information and Communication.

KISS members could gain access to training on ethical issues through the association's affiliations with international professional organisations like the ACM, IEEE and IFIP, but we do not know if the Techwin people have done this, or whether they are members of other, autonomous professional societies.

How could the ACS Standards of Conduct assist an ICT professional in a case like this? Many of these Standards of Conduct relate to the honesty, competence, and professionalism of a person, and provide guidance for everyday ethical matters, regardless of the type of product or system the professional is involved with.

There are some other Standards which could give guidance and support in cases where the professional has a serious ethical dilemma, such as being involved in the development of a robot which had the potential to kill autonomously.

Clause 4.5.6 states: I must advise my clients and employers as soon as possible of any conflicts of interest or conscientious objections which face me in connection with my work.

Other clauses which might support the position of the person conscientiously objecting to building this type of product are Clause 4.6.7: I must go beyond my brief, if necessary, in order to act professionally.

and Clause 4.8.1: I must protect the health and safety of those affected by my work.

Taking an ethical stand against a client or employer can be a daunting task, and so Clause 4.10.7 could provide help, as it states: I must seek advice from the Society when faced with an ethical dilemma I am unable to resolve by myself.

It is the responsibility of all technology professionals to ensure that they are both responsible and ethical in the way they develop and apply technology, and this is especially so when there is a risk of harm to people from that technology. The ACS Standards of Conduct have been written to help professionals meet this responsibility.

Mike Bowern is a member of the ACS Committee on Computer Ethics.