Righteousness and Conscience as a Path to Socially Acceptable Autonomous Behavior

Systems engineering is advancing the autonomous decision-making abilities of systems on many fronts: from self-driving cars that platoon on the highways and collaborate on developing traffic conditions, to weapon systems that work alone and in heterogeneous swarms. What will govern the behavior of these unsupervised systems when life, property, or mission is at stake, and when previously unanticipated situations arise? A nearly infinite variety of environmental situations is possible, and impossible to test in advance. Independent of the environment, systems fail, and complex systems can exhibit complex unanticipated behaviors. Systems are also targeted by intelligent and persistent adversaries intent on system intervention. How can we turn these things loose—with any confidence? Maybe we can take a lesson from the social systems we know. I would argue that society works when an individual conscience governing personal behavior connects to a collective righteousness that governs group behavior. As Allison George (2012) summarizes Jonathan Haidt's new book The Righteous Mind: Why Good People Are Divided by Politics and Religion (2012), in its original meaning, righteous means just, upright, and virtuous. But in a colloquial sense, it often means self-righteous, judgmental, moralistic. The evolutionary story Haid tells in his book is one where moral judgment is the ability to create moral matrices and punish, shame, and ostracize those who do not behave rightly. As we will see, judging the behavior of others is not limited to humans in the animal kingdom. The authors of Wikipedia have defined conscience as " an aptitude, faculty, intuition or judgment of the intellect that distinguishes right from wrong. Moral judgment may derive from values or norms (principles and rules). In psychological terms conscience is often described as leading to feelings of remorse when a human commits actions that go against his/her moral values. " A common metaphor for conscience is the " voice within. " Artificial autonomous systems present a particular challenge for morality. Work led by Ronald Arkin at Georgia Institute of Technology is concerned with the ethical behavior of unmanned autonomous systems (UAS) used in military operations, and recognizes the potential for peer monitoring: " When working in a team of combined human soldiers and autonomous systems, they [unmanned autonomous systems] have the potential capability of independently and objectively monitoring ethical behavior in the battlefield by all parties and reporting infractions that might be observed. This presence alone might possibly lead to a reduction in human …