AI researcher Jacob Coxon resigned from Anthropic last week, warning that people building the technology believe it could “kill us all” by the end of the decade.
Anthropic researcher Evan Hubinger subsequently said he believed there was a greater than 10% chance of AI killing all humans within the next decade.
Former Google DeepMind researcher Bilal Chughtai has now added his voice to the warnings, while Anthropic CEO Dario Amodei has called for the industry to slow the development of increasingly powerful AI systems.
However, Dr Alison Kahn argues we should worry not only about whether AI could one day become smarter than humans, but about how much power we are already giving machines that do not understand the world as humans do.
The Loughborough University academic says AI can produce remarkably convincing answers while lacking important forms of human understanding that come from lived experience, relationships, emotions and having a body.
She said: “The danger becomes particularly acute when governments, businesses and other institutions treat those outputs as genuine understanding and give AI authority over decisions affecting people's lives.”
In her paper, Consummatum Est: On the Faustian Laboratory, the Hubris of AI’s Architects, and the Humanity They Forgot to Ask, Dr Kahn uses Christopher Marlowe’s Doctor Faustus to explain the problem.
Faustus is a scholar who trades his soul for extraordinary knowledge and power. AI has frequently been described as a similar “Faustian bargain”.
However, Dr Kahn argues we have been comparing AI to the wrong character.
AI, she says, is better understood as Mephistopheles, the demon who provides Faustus with the power he desires, while humanity collectively takes the role of Faustus by embracing AI without fully considering what may be sacrificed in return.
The paper also draws on Shakespeare’s King Lear. Lear asks his daughters to demonstrate how much they love him so he can rank them.
His mistake, says Dr Kahn, is assuming different kinds of love can be placed on a single scale.
She said: “AI systems similarly turn complicated choices into rankings to determine the “best” outcome. But values such as love, fairness, compassion and loyalty cannot always be neatly compared.
“This makes the humanities increasingly important. Philosophy, literature and other humanities disciplines can help society examine questions about morality, meaning and human experience that cannot simply be reduced to data.
“The central warning is therefore different from predictions of a future AI apocalypse: the danger is not only what AI might eventually become, but what humans may choose to let it decide before it ever gets there.”
ENDS