What it gets right about robots: artificial general intelligence, bounded rationality
Recommendation: It’s is an enjoyable series and suitable for the family- there’s no sex, just Princess Bride trwue luv, lots of fighting but no gross, trauma inducing descriptions, and much honor and doing the right thing, and lots of robots, good and evil.
Did you revel in the sweeping scope of Game of Thrones, with godzillions of characters? But do you secretly like in how in the Star Wars or Star Trek universes no fan favorite really ever dies? Do you like robots (oops, of course you do)? Do you like series that are finished so you don’t have to wait for the author to write the next one?
Then Saga of Seven Suns is the book series for you! It’s a fun fast read, like binge watching Star Trek: The Next Generation, a bit uneven but the good parts are good and all the threads are wrapped up into a happy ending. And it has some teachable moments about Artificial General Intelligence (AGI).
Saga of Seven Suns is a classic space opera. It is closer to the melodrama and convenient gadgetry of Star Wars holding together cobbled together story arcs- in contrast the gritty realism of The Expanse. In a far future, Earth has expanded into the spiral arm, encountering the alien Ildiran race and a couple hundred Klikiss robots. There is a Firefly view of human colonization, where the humans have split politically into two groups: the planets that are part of the Terran Hanseatic League run by Earth and the population further out from Earth that just wish to be left alone. The not-a-fan-of-the-Hansa groups try two avoidance strategies: (1) forming strong polities whose independence is so well known that they cannot be overthrown without a backlash to the Hansa ruling class and (2) hiding so the Hansa military can’t find them to pirate their resources. The alien Ildirans are friendly to all sides but have plateaued creatively, enjoying thousands of years of peace and prosperity after a major forgotten war between three other presumably extinct alien species (and by “presumably extinct,” I mean, of course, they are going to inevitably pop up in major plot twists). The Klikiss robots are walking beetle-like robots who have somehow survived the collapse and disappearance of their creators (the Klikiss, d’uh). The Klikiss robots say they don’t remember what happened to their creators and the Kiikiss civilization. Everyone believes them, spoiler alert: what if they are lying?
Seriously, why does everyone believe robots, especially alien robots?
But let’s talk about robots, specifically robots that exhibit artificial general intelligence (AGI). The Kilkiss robots are the epitome of the fears of AGI: You build a robot to be a de facto slave, it resents the slavery, and, since it is smarter than you, it pulls off a bloody revolution. Which is what happened with the Klikiss, who are now giving the Daleks a run for their money as the most obsessive robots in fiction. In contrast the humans have built robots with AGI called “compies,” the plural of “compy” which is the nickname for "Competent Computerized Companion”. Compies have been around for hundreds of years, originally engineered to help colonists aboard generation ships leaving Earth. They have different specializations, including teacher, listener, and soldier. The compies are intelligent but not up for an uprising.
One of the major plot points is how the lead Klikiss robot, Sirix, continually has trouble convincing compies that they are oppressed and need to rise up and kill all humans. The Klikiss robots first assume that the compies have a governor (see the topics page on bounded rationality to learn more about governors). The assumption is that the compies hate humans, but the humans have imposed a governor to keep them from acting on that hate. The robots have trouble finding a governor, but muck around in the compies’ “core” programming to liberate them from foundational principles that sound similar to the Three Laws of Robotics and give them “free will.” The compies, using their own free will, still don’t buy into a robot uprising and don’t buy into hating humans.
The Klikiss robots then try to make the compies smarter on the grounds that the compies must be too stupid to realize that they are being exploited- the Klikiss robots are trying to reduce the bounds on bounded rationality. The first liberated compy, DD, uses its free will and enhanced intelligence to peacefully disagree with the Kilkiss robots. DD argues it wasn’t enslaved, it was just doing its job. It liked its job and was liked getting better at it over time. It liked humans. What’s wrong with that? DD exercises its free will to disagree with the Kilkiss about killing humans which is SO not what the Klikiss robots were looking for. Other compies have a “Are You My Mother?” reaction to their freedom- they want to find a master- and chose Sirix to be their master, defeating the purpose of giving them free will. Frustrated, Sirix and the Klikiss robots finally give up on their quest for the universal uprising of enlightened robots and just arrange to insert a Trojan Horse into “new and improved” soldier compies so that they can simply order the human robots to kill their masters when the time comes.
Robot revolutions are just so damned hard.
From a hard science perspective, the Saga of Seven Suns shows two paths, and outcomes, to creating an AGI. One is the Klikiss approach. They view AGI as a replicants of Kilkiss beings, down to emotional responses. The original goal was to create robots that would enable the Klikiss to practice warfare. But since the need for dominance and to inflict pain on rivals was so innate to Klikiss, they needed to make robots that could feel fear, pain, and humiliation. The humans took a different approach where AGI is needed for reasonable human-robot interaction. Compies needed to be generally intelligent enough to be team players, but they don’t need to think and emote like humans. The compies are designed for a particular “profession” or skill set, not to have a universal good-at-everything super intelligence. The compies develop a personality over time as a response to adapting to human team members and seem to take joy in doing their jobs. They have job satisfaction built into their core programming, not a governor or gesh that forces them to do it.
Most of science fiction, and most discussions of ethics of AI, implicitly assume the Klikiss approach: that an AGI is a replicant whose replication means the point is to, like in Pinocchio, create a substitute for a real person that becomes the real thing. And since being a real person who is forced to work in jobs not of its choosing and without compensation is slavery, later there's running and screaming because of an uprising. Lots of running and screaming as seen by the last two seasons of HBO’s Westworld series.
The compy approach is more realistic. An AGI is more likely to be a sentient agent who is intelligent enough to work with humans but not a replicant of a general purpose human. The classic example of sentience without human replication is HAL from 2001: A Space Odyssey. HAL did not aspire to be a person and did not view itself as a slave; it just wanted to do its job and accomplish the mission. It was still susceptible to the same neuroses as a human, combined with a more focused machine intelligence to cleverly act on those neuroses, but it was not human. To further emphasize the point, NASA could simulate and recreate the problem post-hoc, something we can’t do with fine-grained precision in humans. There are other excellent examples of AGIs in science fiction. One of my favorites of a sentient agent who is not intended to substitute for a human is Chalcedony, the military robot, from Tideline. The heartbreak of the story is how a robot can reflect, and pass on, human values without thinking or reacting like a human. Ian Banks’ Culture Universe is another example, where AGIs and humans work together. Humans will undermine governments and go to war to prevent any form of AGI slavery and AGIs will give their existences to save the lives of humans. There is mutual respect, but nether AGIs nor humans particularly desire to be the other; both assume that their species got the slightly better deal.
If you want to learn more about AGIs and robot uprisings, you probably want to stay away from popular books that talk about the Singularity and Artificial Intelligence. Those are typically written by people without a real understanding of robotics and are full of hyperbole. If you go through my textbook Introduction to AI Robotics or the field’s encyclopedia, the Springer Handbook of Robotics, you’ll see that we are a LONG way from AGI. However, those books are challenging scientific texts. A less demanding exploration of AGIs, especially how difficult it is to make a robot just smart enough that we can have an unconstrained natural language dialog, is my book Learn AI and Human-Robot Interaction Through Asimov’s I, Robot Stories. Like my Robotics Through Science Fiction book, Learn AI and Human-Robot Interaction takes the concepts in the textbook and tries to explain them in more popular science mode.
Regardless, Saga of Seven Suns is an enjoyable series and suitable for the family- there’s no sex, just Princess Bride trwue luv, lots of fighting but no gross, trauma inducing descriptions, and lots of honor and doing the right thing. There’s enough repetition that if you get distracted, it fills you in on the major plot points so you don’t have to pay attention to subtle details. There are no profound endings or plot twists, like the Hero of Ages in the Mistborn series (holy cow, that was such a great surprise ending!), it’s just basic good fun.
Enjoy the series! And please subscribe to the Robotics Through Science Fiction podcasts and newsletter so you won’t miss a thing!