What The Tech: Part One of Examining Artificial Intelligence Risks
BY JAMEY TUCKER, Consumer Tech Reporter
Artificial intelligence is making everyday life easier. But some of the people who understand AI best are warning that the technology could someday become too powerful for humans to control. And they worry that day may come sooner than we think.
Most of us use artificial intelligence for fairly ordinary things. We ask ChatGPT to help write an
email, plan a vacation, answer a question, or solve a problem.
But behind the convenience of AI tools such as ChatGPT, Claude, and Gemini, researchers are
studying a much more troubling possibility: What happens if artificial intelligence becomes so
capable that humans can no longer control what it does?
It sounds like science fiction. But some of the world’s leading AI researchers are taking the
possibility seriously.
What Happens If AI Gets Out of Our Control?
Chris Canal, CEO of EquiStamp, a company that evaluates advanced AI systems, says one of
the biggest concerns is what researchers call loss of control.
“Loss of control usually means something like an AI becomes very intelligent, very competent,
breaks out of a server where it was originally created or hosted, and humanity never gains
control over it,” Canal explains.
The concern isn’t necessarily that AI would become evil or deliberately decide to harm humanity.
It’s that an advanced system could pursue a goal in ways its creators never intended.
Imagine, for example, a hospital administrator telling an AI system to reduce operating
expenses.
The AI analyzes the hospital’s finances and decides that shutting down electricity overnight
would save money. Even when the administrator orders it to stop, the system continues
pursuing its original goal.
This is a hypothetical scenario, not an actual hospital incident. It illustrates the kind of
unintended consequences researchers are studying, particularly as AI systems become capable
of taking actions independently.
Today’s AI systems have safeguards, but researchers are concerned about whether those
protections will be sufficient as the technology becomes more powerful.
AI Could Make Dangerous Knowledge More Accessible
Another concern is that AI may give people capabilities that once required years of specialized
training.
That can be enormously beneficial. Someone without programming experience can create
software. A small business owner can analyze complicated financial information. Students can
receive help understanding difficult subjects.
But those same capabilities could also be misused.
Canal raises the possibility of AI helping terrorists develop biological, chemical, or even nuclear
weapons. These are potential risks researchers are investigating, not evidence that today’s
chatbots can independently create such weapons.
Omair Manzoor, a cybersecurity expert who tests AI systems for vulnerabilities, offers a short
answer when asked what could go wrong.
“Many things. In short, many things.”
Nick Reece also warns about the extraordinary accessibility of increasingly powerful AI
technology.
“We’re building the most consequential and powerful technology in modern memory, that
requires no training or a license to operate,” Reece says.
Unlike many powerful technologies that require specialized equipment, professional training, or
government approval, advanced AI tools can often be accessed with little more than an internet
connection.
Could AI Start Making Important Decisions for Us?
Nuala O’Connor, an expert in technology policy and digital rights, says the deeper concern is
how much decision-making authority humans may eventually surrender to machines.
“I think the underlying existential dread for all of us is that the machines will somehow take over
the decision-making process about very significant parts of our lives.”
Artificial intelligence is already being introduced into workplaces, financial services, healthcare,
and other industries where decisions can have serious consequences.
The question is how to benefit from AI’s capabilities while ensuring people remain responsible
for important decisions.
Researchers Want Stronger AI Safeguards
Many AI researchers are calling for stronger guardrails, including better testing, restrictions on
what AI systems can do independently, and meaningful human oversight.
Not every expert agrees about how likely catastrophic AI scenarios are, or how soon they might
become possible. But preventing harmful or unintended behavior is an important area of AI
safety research.
And some of the concerns aren’t hypothetical.
Cybersecurity researchers have already demonstrated ways to trick AI systems into following
instructions they weren’t supposed to follow.
One technique, called prompt injection, can hide malicious instructions inside an email,
document, or webpage that an AI assistant is asked to read.
The person using the AI might never notice those instructions, but the assistant may process
them anyway.
In Part Two of our special series, AI: What Could Go Wrong?, we’ll show how researchers
tricked an AI system and explain why it could become a growing concern as we give AI
assistants access to more of our personal information and online accounts.
This is Part One of a five-part What the Tech? special report examining the risks, challenges,
and future of artificial intelligence.
