Dancing with Our A.I. Overlords
Wild agnosticism on the existential threat of Artificial Intelligence
God help us, we’re in the hands of engineers.
— Dr. Ian Malcolm, from Jurassic Park
Since releasing The Social Singularity in 2018, back when AI was still in the lab, readers have asked me to weigh in on the subject (again) now that the technologies have shipped for the masses.
With humility, I say today’s AI inner workings are—to me—a black box.
I understand that the most successful efforts in AI use large language models (LLMs). But beyond the description of a system with billions of “parameters,” I only know this: sophisticated algorithms get fed large data sets and use probabilities to regurgitate responses that pass the Turing Test with flying colors.
Other than that, I could barely tell you the difference between the inner workings of ChatGPT and Zoltar the Fortune Teller.
Still, I have thoughts.
The Debate Stage
Highly intelligent humans provide fodder for my AI agnosticism.
On one side, artificial intelligence researcher Eliezer Yudkowski is soiling his pants. He writes:
Many researchers steeped in these issues, including myself, expect that the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die. Not as in “maybe possibly some remote chance,” but as in “that is the obvious thing that would happen.” It’s not that you can’t, in principle, survive creating something much smarter than you; it’s that it would require precision and preparation and new scientific insights, and probably not having AI systems composed of giant inscrutable arrays of fractional numbers.
I know that Eliezer Yudkowski is smarter than 99.99 percent of human beings.
Yet so is Robin Hanson, who thinks “Most AI Fear is Just Future Fear.” Hanson writes:
Yes, the most dramatic AI fear, the one most worrying my Twitter followers lately, is of a single AI that suddenly “fooms” to improve itself, conquer all, and then with greatly changed values kill us all.
But Hanson thinks fear of change gives rise to phases of neo-Luddism. In a related piece, he writes:
Doomers worry about AIs developing “misaligned” values. But in this scenario, the “values” implicit in AI actions are roughly chosen by the organisations who make them and by the customers who use them. Such value choices are constantly revealed in typical AI behaviors, and tested by trying them in unusual situations. When there are alignment mistakes, it is these organizations and their customers who mostly pay the price. Both are therefore well incentivized to frequently monitor and test for any substantial risks of their systems misbehaving.
Setting the debate stage this way contributes, in some limited way, to my fuzzy Bayesian analysis based on reports by smart people. But we have to go deeper.
Agnosticism
I remain agnostic on whether AI represents a significant existential threat that would prompt someone to warn humanity to “shut it all down.” However, I’m not agnostic as to whether Kamala Harris or any other “turnkey totalitarian” can be trusted with such power. Pandora’s Box is open and open-source.
Now, why would anyone want to read the opinions of an AI agnostic? It’s like asking people to care about the opinion of someone situated squarely between C.S. Lewis and Richard Dawkins. Ah well. I hope I can make AI agnosticism entertaining and edifying. Let’s go through the perils and promise that makes it a wash for me.
Perils
When I let AI worries roil in my mind, they generally turn on certain kinds of properties that could make any such entity dangerous.
Agency. We think of LLMs like ChatGPT as being rather like call-and-response entities. But what if a computer scientist or some unseen evolutionary force affecting the code allowed the entity to become more agentic? That means the entity could start to ‘decide’ things independently rather than just algorithmically responding to prompts. Of course, one could reply that under some form of determinism, humans algorithmically respond to prompts, too—even as our prompts and responses are more complex. Such roundabout tu quoque thinking is cold comfort regarding questions about future robot overlords.
Self-kaizen. Will these AI entities start to improve themselves? Will they use their ‘agency’ or some other mechanism to improve on specific dimensions? Could self-prompts generate iteration cycles of improvement entirely out of human control? What will such improvements give rise to? And by whose lights would these newly acquired properties be “improvements”? If the answer is not by humans’ lights, then errant AI self-improvement is a justifiable concern.
Superintelligence. These entities are soon going to be smarter than any given human. Because no given human can imagine exactly what it would be like to be smarter than the smartest human—that is, what powers it would confer—we have to imagine an entity that can “think” thoughts and take actions, the ramifications of which are beyond our ken. We’ve already seen evidence that AIs can be duplicitous. If they are both brighter and indeed duplicitous, we could soon see ourselves manipulated by entities that might not have our best interests at heart.
Relentless teleology. Have you ever seen those Boston Dynamic robot dogs? Or, more disconcertingly, have you ever seen the Black Mirror episode with the eerie dog-sized metal killer roaches (Metalhead)? These robots are AIs programmed to execute goals, or more verbosely, teloi. The fifty-cent Greek word sets the stage for teleology, or understanding goal-directed action. Specifically, relentless teleology would be goal-directed action by any means. (See also the related paperclip maximizer problem.)
Steak knife or murder weapon. A penultimate concern, at least for today, has more to do with humans than the AI itself. One can use a steak knife to cut her steak or to kill her husband. Likewise, even if AI becomes more powerful, but a fairly innocuous ding an sich, humans can still be benighted, selfish, and horrible. If we were to cast the statistician’s stones among billions of users, we’re eventually going to hit sociopaths. Powerful AI + sociopaths = “Danger Will Robinson!” (Note: that sociopath distribution holds for developers, too.)
Egrigores. Handwaving Freakoutery’s BJ Campbell has done a lot of good thinking on egregores, the idea that large groups can contribute to mass belief sets that seem to take on a life of their own, generating a more powerful entity with what seems like agency. Shouldn’t we fear combining AIs with egregores? Does the LLM’s synthesis of humanity’s chatter—that mix of truth, lies, and ignorance—have the potential to catalyze or magnify egregores?
Misalignment. Finally, you’ve likely already heard about the “alignment problem,” roughly the idea that AI morality doesn’t align with human morality or goals. What’s troubling about this idea is that AI might behave immorally by human lights and that humans are already misaligned with respect to one another’s moral priors. All of this makes it challenging to imagine aligning AI around some set of goals or moral core that might not be a moral core to everyone. (We’re already seeing people express concerns that ChatGPT is “woke” and thus misaligned with other categories of moral reasoning.)
After writing out the above concerns, I’m starting to see why Eliezer Yudkowski might be an ideal spokesman for Depends undergarments. So let’s turn now to a few mitigating points.
Promise
The first thing I want to say about these technologies is that I, personally, am already finding them useful. AI has improved my efficiency at work and allowed me to learn about new things quickly or refresh old learnings otherwise locked in my aging brain. This is not an argument for the idea that future AI doesn’t represent an existential threat. I merely wish to point out that powerful tools promise to benefit humanity even more in the future.
Core morals. Morals might be programmable, which is to say the alignment problem might be soluble. I can hear the “reeeeeeeee” of moral relativists. Still, having graduated from Spiral Dynamics Green, I don’t mind claiming some moral norms work pretty damn well cross-culturally—starting with variations on the Golden Rule, which has emerged in nearly every culture. This universal moral community can sit atop regional variations, especially if instantiated in AI. It would appear that, for now, SD Greens are training the LLMs.
Desire as an ingredient. At least in human decision-making, the significance of emotions and instincts in driving our actions can’t be overstated. They account for a substantial part of what drives us. A clear illustration of this is the story of a man named Elliot, a patient of the physician and author Antonio Damasio. Elliot underwent surgery to remove a non-malignant brain tumor, injuring a small part of his neocortex.
Elliot was once a successful corporate professional with a joyful and flourishing family. However, when he came to Damasio, he was on the verge of losing everything. Despite scoring in the top 5 percent in IQ tests, having a fully operational memory, and the ability to list all possible solutions to any problem, he was incapable of making decisions. His indecisiveness ranged from choosing which pen to use to prioritizing tasks at work or arranging the files in his office alphabetically.
Damasio soon realized that the cause of Elliot's drastic change was his inability to feel any emotions due to his brain damage. He became so indifferent to the world that he treated every experience as neutral. Imagine listening to your favorite music without feeling any emotional reaction. Even though you can hear the notes, it feels empty. You may even remember that the same music once stirred emotions in you. But now, you're only a dispassionate observer, disconnected from all emotional attachments.
Elliot's inability to feel emotions meant he could not assign value to anything. This lack of prioritization indicated that he was devoid of motivation. It brings to light an essential aspect of being human: things are important to us. As psychologist Daniel Gilbert emphasizes, our feelings aren't just significant—they’re vital.
Our emotions and internal experiences make us care, setting us apart from artificial intelligence. Evolution has endowed us with values, consciousness, emotions, and intuition, as well as philosophical, literary, and aesthetic sensibilities. As humans, we can empathize and envision what it's like to be in someone else's shoes. This ability is very likely missing in AI or makes no sense when analogized to the domain of bits and bytes.
To be fair, our desires also make us dangerous. But at least thinking of AI as having no felt desires, only programmed teloi, comforts me—robot dogs notwithstanding. It makes me think desires and other forms of elephant cognition could give us an edge for a while.
Competition. Market forces work. Not always, but a lot. Indeed, markets tend to work better than other arrangements because they are tethered to human wants and needs. So, channeling Robin Hanson again,
[T]he most likely AI scenario looks like lawful capitalism, with mostly gradual (albeit rapid) change overall. Many organizations supply many AIs and they are pushed by law and competition to get their AIs to behave in civil, lawful ways that give customers more of what they want compared to alternatives. Yes, sometimes competition causes firms to cheat customers in ways they can’t see, or to hurt us all a little via things like pollution, but such cases are rare. The best AIs in each area have many similarly able competitors. Eventually, AIs will become very capable and valuable. (I won’t speculate here on when AIs might transition from powerful tools to conscious agents, as that won’t much affect my analysis.)
Such an insight ties nicely with the next.
Local knowledge and collective intelligence.
On this point, I stand by what I wrote in The Social Singularity:
Knowledge of specific circumstances, or “local knowledge” is the most important and overlooked feature of complex societies. And as we become more complex, we will have to develop sense-making apparatuses and forms of collective intelligence that can handle this complexity. People in government, well-intentioned as they might be, are woefully ill-equipped to make judgments about people in local circumstances.
Something similar can be said about AI for now.
Human beings use intelligence in the circumstances of time and place. This distributed, perspectival species of knowledge currently stands in stark contrast to the relatively centralized, monolithic blurring of human knowledge into a server bank.
Similarly, our local knowledge and collective intelligence means we can fractionalize and compartmentalize our knowledge and talents in a way that’s decidedly different from AI, at least right now. AI currently functions as a kind of aggregation of word and number patterns. I think about Thomas Thwaites's vain attempts to build a simple toaster, which requires collective intelligence. To the extent that we might continue to organize ourselves and our perspectives into hive minds, we might remain formidable polytheistic gods over the AI entities we create. But, again, just for a time.
Upgrading grey matter. Can we count on upgrading our brains and bodies while AI advances? One would think such progress could be slower due to factors such as risks associated with experimenting on grey matter. Yet we’re progressing in neuroscience, nootropics, and better nutrition. Such might not be much right now, but as CRISPR and other types of gene editing continue, we might see how to customize more intelligent humans. Will the rate of progress be enough?
The merger. Permit me a lengthy quote from an article I will soon publish in its entirety. The relevant three aspects here are AI, collective intelligence, and neuroscience:
Now, imagine these three relatively separate strands moving forward in time. As we move toward new time horizons, we can see that these strands are moving closer to one another. The punchline here is only a hypothesis: Eventually, the strands will weave together. Philosophy will always stand there like Gandalf to warn of too much hubris or too many leaps of logic. Still, artificial intelligence, neuroscience, and collective intelligence will not remain distinct categories but will eventually converge.
From a big-picture perspective, the issue is how humans and AI can interface and, eventually, merge to some degree. Significant differences exist between us, so the lingering question turns on what we might call “interoperability.”
Some cognitive and brain scientists have warned AI researchers, rightly I think, that there are a whole lot of gaps to be bridged before we can start to think about a direct, physical interface that can connect a human brain with an AI. Yes, we are both causal-physical entities. But right now, that is mostly where the comparisons end.
Human beings are analog; AI is (so far) digital. The way our brains store memories is totally different from the precise memory addresses of computers. Neither the software nor the hardware of AI corresponds precisely to either the mind or the brain. And a brain is far more self-organizing as compared with current AI, although this could quickly change. There are many differences, and these differences make certain kinds of interface somewhat problematic.
Perhaps we should not assume any sufficiently advanced AI of the future will be neuromorphic — that is, brainlike. However, if it is, that could help not only with creating conscious machines but with how we interface with them.
Yes, humans interface with each other through language. And humans interface with computers through code. But if we were to attempt to connect human intelligence with machine cognition, finding translation standards between modes of operation could continue to elude us.
To bridge gaps between brains and machines, we will need to improve our understanding along a number of dimensions. But these problems are not insuperable. As long as everything there is to know about these threads of inquiry is knowable in principle, we can expect to make great leaps in coming decades.
Simply put, when robots can take all our jobs, the line between human and robot will have blurred.
My hopeful-but-tentative view of this future doesn’t mean I can’t imagine grotesque figures like those who visit Davos each year trying to hack into our brains to get us to comply with their bizarre corporatist visions.
Exponential- or s-curves? My sixteen-year-old tells me that games' graphics have already tapered off into an aesthetic asymptote (s-curve). That means marginal improvements to graphics now, not big ones. Will LLMs stay exponential? Or will they and their intelligence follow an s-curve, too? If AI gets marginally better for a while—instead of insanely better each quarter—we might have time to get our shit together, whatever our shit is.
Hype and hyperventilation. In the past, doomsayers like Paul Ehrlich have been oh-so-wrong about the fate of humanity and the planet. Most in his ilk scream that we should always use the precautionary principle when attempting to improve humanity. But doomsayers are so often wrong. Their bad predictions cut off innovation cycles and empower bureaucratic “experts” who are better at politics and moral preening than protecting us from anything. It will be better to nip, tuck and adapt than freak out whenever Daniel Schmachtenberger does another podcast in which Moloch meeting exponential tech.
Now, I hope the above considerations help assuage your fears. At the very least, perhaps you can see why I have mixed feelings. In any case, if you got this far, I hope you were entertained.
Wisdom
In a recent social media post, I wrote the following as a kind of quip:
Artificial Intelligence 1, Artificial Wisdom 0.
But instead of holding forth about the very idea of programmable wisdom, I would like to give GPT-4 the last word:
Wisdom is a multifaceted concept that can be defined and understood in many ways. At its most basic, wisdom is the ability to use your knowledge and experience to make good decisions and judgments. It's also often associated with qualities like compassion, empathy, understanding, self-awareness, and moral integrity. Wisdom often includes the ability to discern or judge what is true, right, or lasting; insight; common sense; and deep understanding.
As an artificial intelligence, I do not possess wisdom in the human sense. I can provide information, suggest possible consequences of actions based on patterns in data, and offer insights based on the vast amounts of information I've been trained on. However, I do not have personal experiences, emotions, or a moral consciousness, which are often important components of human wisdom. I also don't have the ability to understand context in the same way a human would, or to make judgments based on that context.
Therefore, while I can provide helpful, accurate, and informative responses, it's always important to apply your own judgment and wisdom to the information I provide. The ultimate responsibility for decisions and actions lies with the individual, not with the AI.
Celebrating the fifth anniversary of The Social Singularity.