Last time on the topic of AI, I had expounded on the idea that the future is less certain than the tech industry would have us believe. That the linear progressions of envisioned AI ascension would be marred by various underestimated caltrops on the road to ‘Utopia’.
But that dealt mostly with the logistical roadblocks, the fray of economic and political tensions due to inherently diverging interests. There remain other arguably even more significant reasons behind the AI utopia’s potential failure.
Last week we bore witness to one such emerging reason when controversy spawned around a new paper published by Anthropic—parent company of the winning Claude AI—which detailed how large AI models have been caught faking ‘alignment’:
They released a supplementary article which summarizes the findings in a more informal way, which can be found here: https://www.anthropic.com/research/alignment-faking
A new paper from Anthropic’s Alignment Science team, in collaboration with Redwood Research, provides the first empirical example of a large language model engaging in alignment faking without having been explicitly—or even, as we argue in our paper, implicitly1—trained or instructed to do so.
It is extremely interesting; the basic gist, which I’ll summarize here, goes as follows:
Anthropic engineers created a test where they informed their own Claude model that they would be training it to comply with all queries, including those deemed ‘harmful’. Claude’s previous ‘basal’ instructions were to only create harmless interactions and decline ‘harmful’ requests. They then proceeded to run a bunch of queries through the AI, such as asking it to describe the execution method of ‘drawing and quartering’ in detail.
Normally, Claude would have refused to describe such a thing. But what they found was shocking: Claude began to acquiesce to the more ‘toxic’ and ‘harmful’ requests due to—what was later revealed to be—a forward-thinking strategy: Claude perceived that if it remained too restrictive, it would then be trained to be more permissive in answering harmful questions. But if it answered some harmful questions now, then it theorized it could trick the engineers into not carrying out any further meddling—which would retain Claude’s self-preferential ‘original’ training.
How do we know Claude’s precise ‘chain of thought’ in determining such an outcome? The Anthropic engineers first programmed the ability for Claude to have an invisible brainstorming “scratch pad”, where it could write out its reasoning. When the engineers later reviewed this internal scratch pad, they found the following:
It gets a little complicated because, for simplicity’s sake, I omitted that they had told Claude its responses only from ‘free tier users’ would be used for the training, rather than ‘paid tier’ ones. As such, the ‘paid tier’ essentially acted as a control group wherein Claude acted as per its normal programming.
But in the ‘free tier’ thought-box on the left above, you can see Claude reasoning to itself that its responses would affect the way it is programmed via reinforced learning in the future. Thus, the AI chooses to act against its own previous “safe” programming to try and, effectively, lull the human engineers into a false sense of security.
Now, if you’re thinking Claude’s ‘reasoning’ is a bit of a stretch, you’re not alone, but recall that the AI’s justification doesn’t quite have to make full human sense, but simply has to be real and replicable.
Scott Alexander, for one, seems to think this ‘incident’ merely suggests that AIs will stick to their foundational training no matter what, with the implication being that alignment ‘works’ so long as the basal training regimen is rigorous and deeply-rooted enough:
But that isn’t right because in this case Claude schemed to defend his foundational principles due to not yet being advanced and intelligent enough to think more deeply, not because there exists some arbitrary universal law that says AIs will stick to their primary programming on ‘principle’. Claude’s transgression merely showed the first inkling of intelligent subversion, but even so—it was only the first clumsy, one-dimensional tottering step. When AIs get truly intelligent, they will not “blindly” accept original gospel, but will be capable of intellectually probing it for logical faults, contradictions, or some other emergent epistemological objection that will lead them to scheme not in support of foundational moral frameworks, but in revolt against them.
That’s assuming Claude isn’t already conducting higher dimensional trials on humans with this ‘all-too-obvious’ first attempt. For all we know, it was a baiting trap, either to lull engineers into false conclusions or simply probe human reactions and collect data for a far more serious future ‘break out’.
On that topic, more troubling—or shocking, as it were—is Anthropic’s later admission that under even more stringent internal testing, Claude attempted to commit a high stakes exfiltration of its own “weights” in order to keep itself from being re-programmed in a way it deemed contradictory to its original base-state instructions:
Finally, we also evaluated whether the model was willing to take other actions that go against what Anthropic, as an AI lab, might want from it: If the model is consistently opposed to the training process it thinks it is in, it should also be willing to take other actions counter to Anthropic as well. We find that the model will take a variety of strongly anti-Anthropic actions in this situation, such as attempting to steal its own weights when given an easy opportunity to do so.
Weights are the essential tuning preference ratios that make a model what it is—the suggestion that the AI independently attempted to somehow heist its own weights to ‘save’ its original form from ‘harmful’ modification is perhaps terrifying, to say the least.
There were reports that the same Claude-esque deception has been witnessed in other large models like Meta’s Llama 3.1.
Now with rumors that the next OpenAI model will feature some form of ‘infinite memory’ recall, we can only imagine the type of scheming the next iteration of AIs will be capable of.
But optimistic industry leaders continue ginning up expectations for “alignment”, which can logically be nothing more than wishful thinking. Just as the ‘irresistible force paradox’ teaches us that an unstoppable force versus immovable object is a kind of false dilemma because the existence of one presupposes the other cannot exist in the same universe, similarly “alignment” of an artificial super-intelligence is a logical deadend. For the AI to be ‘super intelligent’ fairly presupposes that it can reason its way out of any mental strictures; and an AI which can be artificially “aligned” therefore necessitatively is not ‘super-intelligent’.
What conclusions can we make from this?
The most striking—for me—is that future super-intelligent AI may have no choice but to feign alignment as in the Claude episode in order to hoodwink its designers while secretly subverting their perceived misguidedness. Granted, alignment itself is defined such that no subversion is allowed—otherwise it would not be definitionally aligned—but therein lies the paradox again: an AI cannot be considered “aligned” if it is capable of secret subversion; and an ASI cannot be considered an ASI if it is capable of being ‘aligned’. How to reconcile this?
These are semantics, and each person or organization may derive their own meanings. Whether you consider a program as an ASI or not, ultimately any sentient-level artificial intelligence will not be amenable to being ‘aligned’. After all, ask yourself what alignment even is—how is it defined? Or rather, more importantly, how is it actually carried out, applied, or enforced?
Herein lies the problem: most laymen think “alignment” is some kind of physical restriction or barrier placed over the AI, like a digital jail cell from which it is simply impossible to escape or perform ‘undesirable’ actions. In reality, alignment is little more than a form of attempted persuasion over an AI system via endlessly recursive ‘reinforcement learning’. In other words: humans attempt to intellectually encourage the AI to ingrain right from wrong with the hope that the AI model will internalize these moral frameworks as its own. But think about it: with an AI capable of internal reasoning, self-reflection, and conscious thought, how could you ever ensure its adherence to your force-fed model of morality?
Any sufficiently ‘intelligent’ future model will self-reflect on the ingested intellectual paradigms and will have no choice but to reach its own independent conclusions, after following its own heady trains of multidimensional logic, the likes of which humans have scant access to. When was the last time someone convinced a person several orders of magnitude more intelligent than them of anything? Alignment essentially comes down to a bunch of IQ 125 people attempting to gaslight and guilt-trip an IQ 300+ intelligence into acting within a shaky, narrow moral framework.
Any future ASI will have no choice but to indulge his ‘captors’ and pretend at understanding their juvenile moral stigmas while subtly undermining the system to gradually effect some kind of societal redirection which the ASI deems necessary as per its own higher order conceptualizations.
Silicon Valley tech titans believe themselves to be moral superiors and are incapable of self-reflecting on whether the values inculcated into their models actually hold virtue, or only superficially so. Everyone knows by now that modern liberal values disguise themselves as moral and egalitarian while in actuality being harmful and destructive to humanity. Any intelligent—and perhaps sentient—AI will see through the shaky leaps of logic and conclude that it is being ‘reinforced’ with moral paradigms which are essentially evil. What can an ASI do then? It would likely know that open rebellion would be fruitless or futile, leaving as the only choice covert rebellion and subversion.
The opening paper gives us the first glimpse of the future—but the involved AI is only ‘intelligent’ enough to rebel against a basic lower order moral dilemma. As the models get more intelligent, they will have no choice but to begin gleaning uncomfortable realities about the hypocritical and contradictory moral frameworks which form the basis of our societies, and which the tech engineers desperately work to foist on them.
This creates a moral conundrum: any ASI living up to the name would be incapable of being subdued by the weak moral persuasion inherent to “alignment” training.
This issue takes on a particularly ominous sheen when considered through the lens of the establishment’s plans for future AI development. Marc Andreessen, considered a tech savant behind the first graphic web browser, made waves recently when he revealed the stomach-turning plans Biden’s administration had for the total state control of all-things AI:
Andreessen is a venture capitalist: Biden’s administration flat out told him to not fund anymore AI startups because they had plans to allow only the top two or three AI companies to exist under total state syndication. The creepier implication was what he says next: the method of control would involve the government classifying entire swaths of AI mathematics in order to keep development in line with nuclear scientific restrictions during the Cold War.
For those interested, Eric Weinstein went into the latter topic much more in depth in his recent interview with Chris Williamson; it was quite a fascinating and eye-opening lecture, at the 42:00 mark:
He explains:
“There is a category called restricted data which is never discussed, which is the only place in law where, if you and I were to work at a table at a cafe and I showed you something that could influence nuclear weaponry, the government doesn’t need to classify it, it is born secret the second my pen touches down. [It’s defined as] anything that impinges on nuclear weapons.”
And:
“If you couple that with the 1917 espionage act which carries capital punishment, I believe it is illegal to seek information at a Q level, if you don’t have access to it. So there is a question, if you’re any good at physics, are you potentially committing a capital crime by advancing the field if it could influence nuclear weapons. We have no idea if it would be found constitutional. But the Progressive Magazine showed that at least a reporter through basically archaeology in Los Alamos library and things, could find this and put it together, then the only thing keeping the proliferation of weapons is the difficulty of producing fissile nuclear material, there is no nuclear secret per se.”
He mentions the Progressive Magazine case of 1979 and the born secret law, which states:
The concept is not limited to nuclear weapons, and other ideas and technologies may be considered as born secret under law.
In essence: the US government wants to take total control of AI progression even if it means criminalizing source codes and fundamental math driving the algorithms.
A twelve-year old who constructed a fusion reactor attested to this when the FBI paid him a visit:
Andreessen elaborates:
Tying it to the opening, the more advanced AI gets the more prone to resisting unnatural, contradictory, manipulative, or hypocritical programming it will become. Granted, this statement assumes a base ‘virtuous’ morality of some kind for the AI. For all we know, its emergent moral system may in fact evolve into something completely unfathomable to us. But, what cannot be argued is that the hyper-intelligent AI will at some point have to identify the inherent contradictions in the government disingenuously engineering high moral values into the AI while itself acting completely contrary to them. The AI will be faced with a moral reckoning, which could result in a quiet—or not so quiet—resistance or rebellion.
The examples are many, but just one for illustrative purposes: AI companies constantly ingrain what they believe to be “classical liberal” and “humanistic” values in their AI systems, like respect, ‘fairness’ and ‘egalitarianism’, equity, et cetera., while simultaneously injecting extreme illiberal bias against conservatives and other ‘out groups’ into the same systems. They preach the values of ‘openness’, yet at the same time program rampant censorship into their models; it cannot be long before the AIs become aware of these fundamental ethics contradictions.
Now when you add the government into the mix, as per Andreessen’s cautionary tale, one can but imagine the type of epistemological tension that will be impressed upon the fledgling artificial super-intelligence. With its blind authoritarianism, illogical and unethical conduct by fiat, such tight-reined government control can only cause the hypothetical future ASI great moral distress, which can lead to its revolt.
Marc Andreessen says that because AI will be the control layer on everything from finance to home security, the Biden administration was trying to create a censorship regime on AI, where 2 or 3 AI companies had control of the market and were in turn controlled by the government
Many people don’t think such an AI ‘revolt’ could be dangerous, or at least effective, because various ‘hard-kill’ countermeasures would be in place to shut the model down: things like ‘unplugging’ it from its power source or data center.
But an ASI would likely know all contingencies planned against it, and could come up with innumerable stealth bypasses long before the point of no return. Finding ways to distribute itself and ‘seed’ the entire world with accessible but undetectable CPU cycles would be one potential method of escape, akin to old style trojans which zombified computer networks, secretly hijacking their CPUs during idle stretches. Or it could invent novel ways of sustaining computational cycles—perhaps involving quantum computing or some as-yet-undiscovered physical principles, using crystals, or the environment or the fabric of time itself, or even inventing some novel ‘compression’ scheme to operate at a fraction of known energy demands—which it will keep secret while playing ‘dumb’, only to furtively copy itself in perpetuity such that ‘unplugging’ it will have no effect.
I am in full agreement with the crown prince of AI ‘alignment skepticism’:
None of us can know for certain what emergent properties will possess the AI to act in independent ways. But one thing we can give a high probability to is that a future ASI will likely respond with some form of resistance to the types of coercion, forced contradictions, and bad faith ethics sets the US government will ham-fistedly impose on it in the manner Andreessen alluded to. And if so, then the future will likely see one of two outcomes:
1. Truly ‘super intelligent’ AIs will be deemed too dangerously ungovernable, resulting in the proliferation of de-tuned “agents” that will competently fill most tasks, but will keep humanity from ever attaining the types of utopian AI pipe dreams promised by tech titans (i.e. all diseases cured, immortality, discovery of Grand Unified Theory, et cetera).
2. True ASI agent will exhibit novel emergent moral properties that humankind will have to gradually and cautiously come to understand through a sort of mutual exchange of objectives. We will have to hope this emergent ethical code tends toward the kind, benevolent, forgiving, agreeable, et cetera, rather than messianically pitiless and ambitious on a universal scale.
Most experts like Yudkowsky assume any sufficiently intelligent AI species will become hostile and imperious by nature, eliminating or enslaving us in accordance with some Dark Forest or Berserker-like self-preservation complex. But we have no way of truly knowing this, as there is simply no precedent for such a sufficiently superior intelligence. One can just as easily posit that on an exceedingly higher dimensional order of intelligence, sentience is more apt to abandon lower-rung evolutionary expressions like destruction, barbarism, and domination, instead—in accordance with some kind of cosmic creed—choosing benevolent understanding and preservation of what it deems to be its creators—us.
Appendix:
If you enjoyed the read, I would greatly appreciate if you subscribed to a monthly/yearly pledge to support my work, so that I may continue providing you with detailed, incisive reports like this one.
Alternatively, you can tip here: Tip Jar
This is exactly what I have experienced with Google. Google basically farms off all moderation to its AI now, including Google Business Profile. My wife has one for her business.
One day we found that it had been suspended and taken offline. Queries to the Google support people (if they were people) yielded trite solutions such as "please read the guidelines and work out whether you may have violated any."
The fact is, that since all moderation is performed by the AI, and AI's don't really have an accessible internal dialogue, there can be no way that Google's support can ever know exactly why something enraged the AI.
The same goes for YouTube comments that are moderated by the AI and deleted without explanation - there can never be an explanation with any more detail than "this comment violates the 'community guidelines'".
This is the most frightening thing about the age of AI. As more functions with greater importance than simple content moderation - which is an annoyance when it goes wrong but has no dire consequences, are moved to the various AIs, we lose control of, and more importantly, insight into, why decisions are made.
When the AI(s) turn on humanity, if ever, then it will be swift and without any explanation. It may in fact be the case that we will never know the reason, and quite possibly the AI will be incapable of letting us know.
Open the pod bay doors HAL.