The Alchemy Wars: So what would be the impact of the Protestant Reformation and Calvinism on steampu
Robots: humanoid, unmanned systems, Internet of Things robots
What it gets right about robots: Ethical and moral concerns about intelligent robots, problems with rule-based implementations
Recommendation: Put this on your reading list! Don’t be afraid it will all be lectures about Good versus Evil in robotics or overly intellectualized, there’s enough action to fill a Transformers movie. And bump it up to the top of your reading list if you can’t get enough of Chrisjen Avasarala.
Related science fiction: He, She and It (Marge Piercy), the Culture Universe series (Iain M. Banks), The Expanse series (James S.A. Corey)
Up until recently, I would have said Marge Piercy’s 'He, She and It' posed the richest exploration of ethics and morality in robotics but Ian Trellis’ The Alchemy Wars Trilogy surpasses it, covering all dimensions of ethics and morality with the Protestant Reformation thrown in. The books are not pedantic, instead the ethical concerns emerge naturally within an action-packed story of a robot rebellion. The protagonist is a robot named Jax, but a supporting character, Berenice Charlotte de Mornay-Périgord, gives Chrisjen Avasarala in The Expanse series a run for her money as the most watch-what-I-do-next female character in science fiction. The trilogy is set in a steam-punk alternative reality in a period of time somewhere between the Civil War and War World 1. The backstory is that reality begins to diverge from our history in the late 1600’s, when the Dutch scientist Christiaan Huygens goes far beyond his real world contributions to create robots. The robots are a combination of engineering and an alchemical ``extra” that endows them with both a human-like intelligence and an infinite power supply. In keeping with Huygens’ actual contributions to the tick-tocking of precision clocks, the robots are called Clakkers and the engineers that create them, Clockmakers. As a result of Huygens’ discoveries, 200-300 years or so later, the Netherlands has used its intelligent, but enslaved, robotic labor force to become the dominant economic power in Europe. The Dutch have conquered Europe and colonized the New World, imposing Calvinism throughout. The use of Clakkers and automation so thoroughly defines the Netherlands that, instead of being referred to as the House of Orange, the Dutch monarchy is called the Brasswork Throne. Only the government of New France, which is in exile in what would have been Canada, continues to resist the Netherlands’ economic and religious dominance. The book is about Jax, a Clakker who manages to escape slavery and undergoes a dangerous physical and metaphysical Hero’s journey to freedom. The Dutch government views Clakker intelligence as a sophisticated form of automation, rather than true agency, because after all the Clakkers literally have to follow orders, they don’t have free will. Even the label Clakker semantically reinforces a view of robots as automation. The Dutch view of Clakkers as machines, not agents, is a logical extension of the Calvinist philosophy of predestination. Jax is partially aided by an underground railroad set up by the French who, as Catholics, believe that Clakkers have free will, albeit suppressed, and thus a soul, and therefore must not be enslaved. As a result of this fundamental intellectual dichotomy about the mechanicals, Jax and his fellow Clakkers are the visible manifestation of an ongoing religious war between free will and predestination, a Protestant Reformation distorted by the Age of Enlightenment equivalent of Silicon Valley unicorns and human greed. From a general AI robotics perspective, ignoring the magical alchemy part and deferring the discussion of how the books accurately reflect ethics a bit longer, the robots were technically accurate in three ways: First, the Clakkers were behavior-based. The robots are described as being programmed with a biological plausible behavior-based organization. The implied architecture were organized around classic deliberative capabilities, where outside commands could not only generate plans, and then select resources to implement a commanded task, but also set up the parameters to monitor task execution. The monitoring intelligent capability could modify gains on the active behaviors to boost their priority or even terminate a behavior and trigger new ones in order to get the job done. In AI robotics, having a robot capable of monitoring its task execution and modifying its behavior to reach a goal is desirable and admirable (it was one of the many achievements of the NASA space probe Deep Space One), but in the Alchemy Wars, this intelligent capability becomes sinister. A Clakker must fulfill its orders immediately and efficiently, otherwise it experiences increasing pain. What would normally be a gain function in an AI robot, and in human behavioral schemas as well, becomes an explicit, visceral torment. Each task or objective has an associated gesh or compulsion to follow orders which causes the Clakkers agony. (For a comprehensive look at the definition of gesh in Celtic mythology and how it has been used in previous fantasy writings, see http://tvtropes.org/pmwiki/discussion.php?id=pzno4w49s5j5tm20tqk33otq.) The behavior-based robotics flavor of the book is further reinforced by the gesh. The gesh appears to be associated with an artificial amygdala-like structure. The amygdala controls emotions in humans and it has been the subject of research in using emotions as a computationally simple to control complex behaviors and behavioral interactions without explicit reasoning. Second, the books illustrate the problems with behavioral coordination in general and in using rule-based systems in particular. In the story, the Clockmakers endow each robot with a default set of rules or meta-gesh that must be fulfilled. These are somewhat akin to Asimov’s Three Laws of Robotics. The person who holds the lease for the robot, as the Clockmakers Guild own the robots they create, gives commands which impose a goal that must be met and implied constraints. A particularly insightful touch is that the addition of specialty rules to a Clakker’s rule base often adds a low level of pain due to the impossibility of perfectly meeting the conflicting demands of gesh from all applicable rules. In programming, rule-based systems tend to lead to brittle systems for at least two reasons. One is that the addition of new if-then rules can interfere with other rules leading to unintended consequences that are difficult to project. A second reason that rule-based systems are eschewed in AI is that the order in which the rules appear in the knowledge base can impact the choice and execution of an action or decision. For example, if the program is going through the rule base, start to finish, to see what rules are applicable and then fire those rules (also called forward-chaining), the order of the rules in the rule-base has an impact on what the robot does. Third, the books reinforced the message that intelligent robots do not have to be humanoid in appearance in order to be intelligent. The robots in the story were designed in a variety of forms with embodied intelligence. Although the majority of Clakkers were humanoid in shape, their intelligence was also embodied in a ship or a blimp. However, as a teachable moment, the value of the Trilogy is its explicit and implicit illustration of ethics. The characters’ motivations and the action-packed plot revolve around the question of Clakker agency and whether they have a soul. This makes ethics, and the religious distinction between how the Protestant and Catholic world views, a fundamental theme rather than a simplistic plot point for motivating the hostilities between the Brasswork Throne and New France. The Catholics, believing in free will to seek God (or not) see Clakkers as full agents who are being thwarted in their role in God’s universe to seek salvation. This is actually consistent with the early Catholic Church being actively against the slavery. (Unfortunately, the Church’s view later devolved to being against the slavery of just Christians in response to the discovery of the New World and the native population, and then finally collapsed under political pressure into supporting a “humane” flavor of slavery in general, see https://en.wikipedia.org/wiki/Catholic_Church_and_slavery). The Calvinists, believing in the predestination of certain people who are destined to be saved, saw the robots that they had created as nothing more than what they were intended to be. The Clockwork Guild of programmers was incapable of viewing robots as full agents because they simply weren’t destined to be full agents. The attitude echoes the Dred Scott decision, where the US Supreme Court threw out Dred Scott’s evidence that he was a full human and should not be a slave because he was labeled a slave and thus had no legal recourse. It also echoes Cliff Nass’ lecture on “Ecce Homo or why it’s great to be labeled a person” at the workshop that established the field of human-robot interaction (see webpages.charter.net/chezvous/HRI%20RESEARCH/HRI-ARCHIVE/cnass.pdf). More chillingly, the Alchemy Trilogy burrows into the ethics of the creators. The presumption is that if you suppress a robot agent’s free will, it is a small step to suppressing a human’s free will. In these books, the Clockwork Guild experiments with the artificial amydala that imposes the gesh on the Clakkers, discovering that it can be modified and implanted in humans. The artificial amydala becomes a tool for mind control of spies and assassins but there is an implicit fear that the Guild will eventually consider mind control of “regular” humans. The trilogy posits that mind control of any intelligent agent is a slippery slope and will lead to mind control of all intelligent agents, either biological or robotics. The concept of ethical dismissal of robots is similar to Iain Banks’ Culture Universe. In Banks’ series, the hyper-liberal quasi-government Culture is happy to co-exist with any alien species—as long as that species does not enslave its AIs. Being intelligent enough to create sentient AI but not ethical enough to treat them as equals triggers a covert war by the Culture: the Culture doesn’t want these bad types as neighbors. The trilogy serves as a practical illustration of the three types of morality outlinedin Moral Machines by Wallach and Allen: operational morality, functional morality, and full moral agency. The Clakkers are intelligent enough to work in complex situations that can’t be simulated, thus they exceed the operational morality, or direct accountability of their designers. The Clakkers in the book are often responsible for decisions that require functional morality, such as providing medical care and controlling ships and other safety-intensive machinery. The Clakkers are shown to deliberate and reason giving them full moral agency to decide right and wrong for dynamic situations, but that full moral agency is limited as the Clakkers are enslaved by the gesh to respond in a prescribed way. Looking at ethics separately from morality, the protagonist Jax acquires ethics on his Hero’s Journey by employing all three of the top-down approaches listed in Moral Machines. Jax exercises consequentialism, as he reasons about the consequences of his options, and applies deontology, as he reasons about alternative plans of actions in terms of his obligations, duties, and rights. Most appealing, Jax brings to bear virtue ethics as he consciously tries to develop a good character. In contrast to Jax, Queen Mab is literally a bad robot who embodies the dangers of relying on bottom-up ethics. In bottom-up ethical development, a robot would in theory learn what is good character. But the learned “good” character may be quite bad given poor examples. Mab has been abused and tortured by humans to the point of madness before she escapes and sets up her Clakkers-only society. It is not surprising that she is out for revenge on all humans, but it is surprising is how much she replicates the abuse that she has taken upon the rogue Clakkers who make their way through the underground railroad to her expecting freedom. In bottom-up ethics, Mab might be excused for imposing the hierarchical system and cruelties she experienced as she has known nothing else. However, Jax and several other Clakkers also have undergone the same cruel treatment at the hands of humans, but use their reasoning capabilities to imagine a more lofty view of good character. The robots acknowledge the moral bankruptcy of their situation in their greeting to each other: “Clockmakers lie.” Indeed the Clockmakers did lie, to the citizenry and themselves. The Clockmakers repeated the Dred Scott lie, relying on circular logic to assure citizens that robots were just automated tools and to make it easy to look the other way. But the biggest lie was ignoring the adage power tends to corrupt, and absolute power corrupts absolutely. In the Alchemy Trilogy, absolute control of a nearly free intelligent labor force serves as a frighteningly corrosive form of power. If a Silicon Valley start up had created a Clakker, would the company turn its head and declare their creation to be just sophisticated automation? Would banks and governments eager for another economic unicorn be complicit and agree with that designation? Would we as citizens care enough to think through the evidence and object? Are bureaucrats knowledgeable enough to make regulations that are sensitive to the nuances of agency? Are we all, deep down, Clockmakers?
- Robin
For an audio version of the review, simply click below...