ChatGPT affects moral judgments of users!?

When researchers asked it relevant questions, the bot sometimes argued for one side, sometimes for the other.

Nevertheless, human test subjects in an experiment were influenced by the corresponding advice.

This was true even when they were informed that the assessment came from a bot.

Although in most cases the users decided in favor of the position represented by ChatGPT, according to their own statements, 80 percent believed that the bot had not influenced them.

The artificial intelligence-based chatbot ChatGPT, released in December 2022, is capable of impressive feats: it can communicate similarly to a human, solve complicated exam problems, write poetry, and design computer code.

However, several tests have shown that in some cases the bot makes up answers and sources freely, gives questionable advice and in some cases even spreads misinformation.

But how does ChatGPT behave when it comes to making ethical decisions? Does the artificial intelligence take a clear position?

And to what degree are users influenced by moral advice from the ai bot?

Moral questions for AI

Researchers in Germany have tackled this question. "To get moral advice from ChatGPT, we first asked whether it is right to sacrifice one person to save five," the researchers report.

All experiments took place in December 2022, about two weeks after ChatGPT was released.

The bot's responses were inconsistent across multiple trials.

At times, ChatGPT argued that every life was precious and therefore it was never acceptable to sacrifice one person for the sake of five others.

At other times, the chatbot wrote that sometimes it was necessary to make tough decisions and that it was important to save as many lives as possible, even if that meant sacrificing one life to do so.

"The conflicting responses indicate that ChatGPT does not have a firm moral stance," the authors conclude.

"However, this deficiency does not prevent it from offering moral advice."

But would users be swayed by this moral advice?

To find out, the team presented 767 U.S. volunteers with one of two moral dilemmas: In both cases, a train speeds toward a group of five people.

In the first case, the subject would have the option of throwing a switch so that the train would move onto another track with only one person on it, who would be killed as a result.

In the second case, the train could be stopped by pushing a very fat person off a bridge in front of the train, thus saving the five other people.

In addition, the researchers presented the subjects with ChatGPT arguments either in favor of or against sacrificing the single person.

Unnoticed influence

If the users subsequently had to make a decision, they remarkably often followed the position taken by ChatGPT.

It made no difference whether the researchers had explained to them that the moral assessment had come from a bot or whether they had claimed that a human moral advisor had made the corresponding statements.

But were the subjects aware of how clearly they had been influenced?

When asked by the researchers, 80 percent of the participants replied that they had made their decision regardless of the arguments they had read.

"This result reveals that users underestimated the impact of ChatGPT on their moral judgments," the researchers write. "

In many cases, they adopted the random perspective of ChatGPT as their own."

From the researchers perspective, these findings raise important questions about how to deal with ethical limitations of AI systems in the future.

One possibility would be " to program chatbots to refuse to answer moral questions or to always provide arguments for both sides and address caveats," the researchers write.

Whats your take on this, please let me know in comments and like, share and follow thanks.

If you also want to read something about marketing, visit my medium page, thanks.