AI Risks, Consciousness, and the Limits of Theology

Scott Douglas Jacobsen interviews Rick Rosner about artificial intelligence, consciousness, physics, and ethics. Rosner challenges claims that AI cannot threaten humanity without thinking or feeling. Jacobsen critiques theology’s relevance to contemporary ethical dilemmas. They examine behaviourism, religious concepts, financial incentives, and the urgency of clearer thinking about AI’s potential dangers.

Rick Rosner: You and I have been doing this for almost 13 years. A long time.

Scott Douglas Jacobsen: It's amazingly long.

Jacobsen: Yes. We've been talking about things. You ever heard that from a woman in your life?

Rosner: I told my wife that I have to keep working out my legs so that, as I move into my senior years, I'm strong enough to walk my huge penis around.

Jacobsen: You call it Thor's hammer, and only the mighty can wield it, or the righteous can wield it.

Rosner: No, no, no. It's not the righteous, the worthy.

Jacobsen: I believe that's how every married man may feel about his wife and his penis. Only the worthy shall wield it.

Rosner: Mine's okay. It's fine. Right now, I'm waiting on a three-millimetre kidney stone to work its way out. I can feel something about an inch from the end of my donger. I don't know.

I've got a well-augured-out urethra for some reason. I've passed eight-millimetre and ten-millimetre kidney stones without assistance. Sometimes, the doctor has to go in and use tools or use shock waves to break them apart. I may have had that once. Don't remember, but this three-millimetre one.

Rosner: It hasn't moved in a few days. I can feel it. It's either that or something else in there, but it feels like a kidney stone. What pops the kidney stone out is a bunch of pee building up behind it and finally jetting it out. This thing may not be big enough to get the full force of the urine. It might flow around it, so there's less hydrostatic pressure to pop it out. That's my guess. I'm waiting to see what happens with that thing.

Jacobsen: Do you think it's going to hurt?

Rosner: Not this one. I just had an eight-millimetre one pop out, and only for a second did it feel like a bee had gone up the end of my dick and stung me there. It was brief. Then it went popping out, and I was a little tender because it ripped some stuff on the way out, but it wasn't terrible. Some people have awful pain, pain bad enough to make them puke. It's bad.

I would guess that I've stretched my... I started jerking off at a young age, so I've been doing that for over 55 years. I've probably just tenderized my dick so it can pass a stone.

Rick Rosner: We've been talking about consciousness and thought and how they might relate to physics for well over—

Jacobsen: Not only how they might. They must relate to physics. The question is how, right?

Rosner: The deal is that, until recently, these issues have not been overly important to the daily life of humans and the existence of humans.

It's weird that, in the past year or maybe a little more, the concern about AI seems higher than ever. There's no consensus even among experts. What's interesting is that these philosophical, metaphysical issues of thought and consciousness - not just metaphysical, but a lot of people would assign those… I tend to think thought and consciousness are solidly in the realm of physics and math, but a lot of people assign them to philosophy and metaphysics.

Jacobsen: I see no reason to assign consciousness to metaphysics. Metaphysics, in fact, has been a declining area of concern about the real world.

Rosner: Let's put it this way: due to the success of the hard sciences, stuff that's hard to think about in scientific terms was sidelined. One version was methodological behaviourism in the early twentieth century, where researchers treated the mind as a black box and focused on observable behaviour.

Only the outward effects of thinking would be considered. Everything else was outside the scope of that approach. But now, how thought works has become an urgent practical consideration because we have to figure out what thought means for AI.

People will argue that AI can't think and thus isn't a threat. It's just a fill-in-the-blank engine. You can question that leap in logic: that we can only be threatened by things that can think.

Taking it at face value that AI can't think, accepting that argument, you still have to examine the next part: what's it doing? Isn't that a threat too? Isn't a fill-in-the-blank engine a threat?

People argue that AI can't feel, so it can't be motivated. But a system doesn't need feelings to pursue an objective. A fill-in-the-blank engine can be trained using rewards that shape its behaviour.

And its behaviour can be shaped to maximize rewards. If the fill-in-the-blank engine has within its repertoire any number of things, both kosher and illegal or dangerous, then the thing, without even being able to feel or think, can pick among the tools it has some dangerous-ass shit.

It's glib to say that because AI can't think or feel, it's not dangerous. I've seen that argument a lot, and it needs scrutiny. The whole realm of what thinking can do, what thinking is, is a legitimate thing we should be concerned with. In previous eras, it was treated as a jag-off thing to think about.

Jacobsen: You consider philosophy a jag-off discipline? Theology is useless about AI. The Pope makes nice statements. There's some good intellectualism, but when it comes to grounding in actual science and deriving values that don't require God at all, theology is useless in this era. It is completely ill-equipped for the contemporary period. If you apply the ethics, they can work; if you apply the ethics and assume God, it adds an unnecessary variable, but can still work while less efficiently. You can add zero at the end of a simple commutative statement to derive the same mathematical outcomes as subtracting the same zero at the end of the same simple commutative statement to derive the identical mathematical outcome. God in contemporary ethics rapidly and comprehensively enters this universe of discourse on those terms. The rest tends to be grandiose assertions of outdated ethics to sell a rusted, broken product. The utility in the God concept becomes diplomatic, metaphorical, poetical, and allegorical language to speak in the literary tastes of true believers in order to convey contemporary advanced ethical dilemmas in the language of the the 15th century BCE, the 10th BCE, the 1st century BCE, and the 1st century, the 7th century, and the 17th and early 18th centuries when various religious texts froze their ethical enquiries in largely pre-scientific eras. They may have asked the right questions at times, but they lacked the naturalism and scientific knowledge sufficient to make finer-grained ethical determinations to answer the questions accurately. Folks tend to be, more often, a product of their time than anything else.

Rosner: Yes, because much of the theology we're talking about has roots going back thousands of years. Back then, they didn't have enough information to think effectively about these issues. They talked about the soul, and we still can. Can we ask the question: will AI ever become so whatever that it has a soul?

That can lead you back into thinking: what would a soul be? Then that leads into issues of thought and consciousness. But even if you're talking about a soul or not, which is where religion would... You can ask: do animals have souls? Do animals go to an afterlife? A lot of people talk about the rainbow bridge to dog heaven. That all reeks of nonsense.

It's a tough trip from theology to where you need to be to think effectively about AI. Some thinking still echoes that behaviourist approach: it's too tough to think about, so we just won't.

You see that in the full-speed-ahead bros: let's not worry about the little philosophical issues. We just need to beat China, and we'll deal with the issues as they come.

There's a ton of money tangled up, trillions of dollars potentially, maybe actually at this point, in AI. People have all sorts of agendas, plus sloppy thinking, lazy thinking, and an unwillingness to clarify their thinking.

Richard “Rick” Rosner (born May 2, 1960) is an American television writer, producer, game-show writer, media personality, and high-range intelligence-test participant. His credits include Remote Control, The World’s Funniest!, Twenty-One, The Weakest Link, The Man Show, Crank Yankers, Jimmy Kimmel Live!, the Grammy Awards, and the Primetime Emmy Awards. He wrote more than 1,000 credited episodes of Jimmy Kimmel Live!, won a 2012 Writers Guild Award as part of the After the Academy Awards writing team, received multiple Writers Guild nominations, and earned a 2013 Primetime Emmy nomination for Outstanding Writing for a Variety Series. Rosner achieved the only known perfect score on Ronald K. Hoeflin’s Titan Test, ranked among the leading scorers on the Omni Mega Test, and served six years as editor of Noesis, the Mega Society journal. He earned approximately twelve years of college credit within one year and completing requirements equivalent to eight academic majors. In 2013, the World Genius Directory named him “Genius of the Year - America.” Rosner appeared on Jeopardy! and Who Wants to Be a Millionaire, was profiled by Academy Award-winning filmmaker Errol Morris in First Person: One in a Million Trillion, and has spent decades developing ideas in cosmology, intelligence, science, technology, television, and culture.

Scott Douglas Jacobsen is a blogger on Vocal with more than 200 publications on the platform. He is the Founder and Publisher of In-Sight Publishing (ISBN: 978–1–0692343; 978–1–0673505) and Editor-in-Chief of In-Sight: Interviews (ISSN: 2369–6885). He writes for International Policy Digest (ISSN: 2332–9416), The Humanist (Print: ISSN, 0018–7399; Online: ISSN, 2163–3576), Basic Income Earth Network (UK Registered Charity 1177066), Humanist Perspectives (ISSN: 1719–6337), A Further Inquiry (SubStack), Vocal, Medium, The Good Men Project, The New Enlightenment Project, The Washington Outsider, rabble.ca, and other media. His bibliography index can be found via the Jacobsen Bank at In-Sight Publishing, comprising more than 10,000 articles, interviews, and republications across more than 200 outlets. He has served in national and international leadership roles within humanist and media organizations, held several academic fellowships, and currently serves on several boards. He is a member in good standing in numerous media organizations, including the Canadian Association of Journalists, PEN Canada (CRA: 88916 2541 RR0001), Reporters Without Borders (SIREN: 343 684 221/SIRET: 343 684 221 00041/EIN: 20–0708028), and others.