For a number of years, we have grown accustomed to seeing movies that demonstrate the–then future–problems that the application of artificial intelligence (AI) may entail. On many occasions the representation of this future, and now almost present, reality shows a relationship with humanity that is a bit disheartening, one in which the awakening consciousness of machines through individual and collective artificial intelligence challenges us as they wish to become more than mere servants. Who doesn’t remember films as successful as The Terminator, I, Robot, and the legendary and hallowed Blade Runner? All of these films focus on the physical part (robotics) of AI in order to demonstrate its actions and consequences, but it is precisely this aspect that least represents AI, because AI actually lives in the most ethereal part of engineering: the representation of knowledge of our environment and the ability to learn by using models and algorithms that are constantly running on one of the millions of processors surrounding us every day. How can one of these algorithms located in the memory of a processor in a data center in some remote location become so dangerous?
This question has led a large number of scientists and experts in the area to consider the ethical dilemma regarding what artificial intelligence should and should not be able to do with regard to its inclusions within healthcare, transit, education, energy, and in the most compromising scenario, defense. Should AI decide whether a young person takes priority over an older person when receiving a kidney donation? Should a robot be able to identify and attack people without human supervision? Should an AI system be able to declare war?
This leads to an intense debate about how AI should align with human values and also ensure that this alignment is respected and fulfilled by all players involved without impeding progress and advancement to improve social wellbeing. What we’re really seeing is no more than a great opportunity to define and regulate how we would like our relationship with the autonomous smart systems that will be developed in the coming years and decades to be. And for this one must ask: what makes an AI algorithm ethical? There are three characteristics necessary for this.
First, it must be transparent. This means that past behavior can be inspected and audited and that the model on which said behavior is based is described (for the sake of clarity and to use industrial terminology, this would be the system’s patent). Second, said behavior must be predictable, i.e., we must know beforehand how it will react when faced with a certain stimulus. This involves a context of exhaustive ethical control tests that is currently undefined and that almost no AI systems currently consider (the case of self-driving cars may be the only exception). Third, it must be able to withstand potential manipulations. The first two characteristics are worthless if we cannot ensure that they cannot be distorted by attacks to manipulate AI and therefore its ethical criteria.
In one of my recent talks  I asked students in the summer school: What characteristics do you think an AI-based system should have in order to ensure it causes neither physical
nor moral harm to people? And with their student wisdom, they answered: “It needs to be able to be turned off.”