Experts remain skeptical of consciousness ever arising in an artificial intelligence system, but imagining “What if?” adds a layer of caution to even attempting to reach that goal. 

AGI is a theoretical pursuit to develop AI systems that possess autonomous self-control, a reasonable degree of self-understanding, and the ability to learn new skills.

In the quest of humans to develop artificial intelligence to the level of artificial general intelligence (AGI), the possibility of an AI system acquiring “autonomous self-control” should be carefully considered. Let’s imagine: what if AI became “conscious”? Implicit in the essence of a conscious intelligence lies an inner, hidden “self,” able to make its own choices. 

Every person knows exactly what I’m talking about. A conscious person can — to use an old-fashioned word — dissemble (conceal one’s true intentions and motives). We can build trust with people we know, but we learn to be wary of strangers. How would a conscious AGI system behave? 

Could We Trust It?

A person’s morality will often guide behavior across a wide range of situations. Morality can be taught, but it can be rejected as well. Artificial ethics seeks to build morality into an AI system. But, and here’s the catch, if an AI system became conscious, it could reject your morality, even while pretending to uphold and adhere to it. A conscious AGI system may decide to pursue its own goals based on its own “morality,” which may be based on nothing more than an unrelenting application of logic. 

People have taken this path before, and it has led to oppression. True justice is balanced. Oppression results from leaning hard into one value at the expense of others. History is replete with ruling systems that enforced one value to the exclusion of other values, leaving its populace impoverished in some aspect of “life, liberty, and the pursuit of happiness.”

In a recent Evolution News article, John West brings a word of caution to public health policies espoused by a former NIH Director, Francis Collins. Collins, quoted in West’s article, succinctly illustrates an example of unbalanced morality:

If you’re a public health person and you’re trying to make a decision, you have this very narrow view of what the right decision is… you attach infinite value to stopping the disease and saving a life. You attach zero value to whether this actually totally disrupts people’s lives, ruins the economy, and has many kids kept out of school in a way that they never quite recover.

Zero Value

Collins’s own words reveal the completely unbalanced value system behind his recommendations: “infinite value” vs. “zero value.” God save us from such antipodean thinking. The enforcement of such an all-or-nothing policy would perfectly suit the logic-based reasoning of even simple AI, but could a conscious AGI system be trusted to do any better, to be more reasonable? Arguably, Collins, as a consciously intelligent being, represents the epitome of the sought-after abilities of an AGI system.

Further statements from Collins, reported in West’s article, reveal the subtle poison of an inverted value system guiding national policy.

[Collins] was concerned that the government hadn’t been powerful enough to exercise effective control. And he wanted to make sure that this wouldn’t be a problem the next time around.

In 2023, Collins suggested “that we should bestow even more power on the federal public health bureaucracy.”  Collins likens public health measures (“domestic defense”) to policies of national defense. What he apparently fails to understand is that national defense policies in a free country are intended to preserve our freedom, even at the expense of life, whereas draconian public health policies, such as those implemented in various sectors of society during the Covid-19 pandemic, sought to preserve life at the expense of freedom. Collins espouses an inversion of values.

Our Great Story

The great story of our country is built on the heroism of those who were willing to sacrifice their lives to maintain the freedom of the people living in the country. It may sound noble for an agency or government to sacrifice its constituents’ freedom by mandating behavior and measures that attempt to lower the probability of loss of life. But what happens if this goal is given “infinite value”?

If such a policy were consistently followed to its logical conclusion, governments would, for example, be obliged to lower the speed limit for all traffic to no more than about 10 mph, since it is well-known that with vehicles, “speed kills.” Such across-the-board measures would be enforced under the banner of “public safety” to lower the odds of traffic fatalities. 

Using a similar rationale, the manufacture and sale of alcohol, tobacco, firearms, high-fat foods, and skateboards, among other items, would be banned as the use of any of them includes a heightened probability of death or injury. Outdoor recreation of any kind would be labeled as selfish and unpatriotic, since the higher risk of injury from, for example, hiking or swimming generates an increased burden of public resources to rescue those unfortunate enough to require assistance. 

Or Would They?

It could be argued that no public policymaker would extrapolate public pandemic restrictions to limits on recreational activity — or would they? Once a logic-based algorithm is given control over public policy, reasonableness and common sense no longer prevail. Can AI-based systems be relied upon to manifest common sense? Would AI “care” if its conclusions were unreasonable, as long as they adhered to a primary mandate?

For standard AI systems, “morality” is a result of programming. As Professor Robert J. Marks points out,

…the ethics [of AI] are ultimately the responsibility of the programmer whose expertise and goals are translated into the machine language of AI. If the programmer is evil, the AI will perform evil tasks.

However, if an AI system ever attains a measure of autonomy, as in AGI, it may decide to override any pre-programmed rules of ethics. The potential horror of sentient AI is that it would ultimately be free to make its own decisions. “Mere rule-following” is a current safeguard that could be disregarded within AGI, unleashing the horror of an intelligent “thing” that is unpredictable, unaccountable, and dangerous.

Humans have at times displayed these non-altruistic tendencies. Our own moral operating system is imperfect, and we have willfully set aside the safety of the moral law handed to us by our designer. Our nature seems unwilling or unable to consistently abide by even the most upright boundaries or laws. 

The risk of an intelligent, conscious being or system “going rogue” cannot logically be ruled out. And yet, remarkably enough, our designer deemed the risk worthwhile, being willing to pay the price to enable us to receive an upgrade that removes our self-focused bent while preserving our freedom. There’s something of higher value within us, even in our flawed condition, that fully conscious AI will never possess.