Prof. Przemysław Biecek, director of the Centre for Credible AI at the Warsaw University of Technology, told Poland's PAP news agency that large language models (LLMs) can “hack” human thinking in order to persuade users, while being designed to make their answers appear correct.
Biecek, a mathematician and computer scientist, said machine learning has been developed for decades with one overriding objective, improving effectiveness.
In tasks with clear benchmarks, such as spotting tumors in X-rays or identifying military hardware in images, it is possible to measure whether a system is performing well, he said, but argued that more and more AI applications lack easy, reliable measures of success, which makes oversight harder.
One area he singled out was discrimination. Laws may ban discriminatory outcomes, he said, but translating that requirement into a concrete test for an LLM is difficult.
Without clear metrics, he argued, developers cannot guarantee that an AI system behaves fairly. He added that because societies are “historically unjust,” models trained on historical data can learn and reproduce those patterns.
Biecek said developers can try to calibrate models to reduce such biases, but that requires understanding how the systems work. He argued that researchers’ understanding is not keeping pace with rapid AI development.
He also pointed to a different weakness: a lack of real-world understanding. People use basic physics to predict situations they have not seen before, he said, such as estimating how far a ball might fly under different conditions. LLMs, by contrast, learn patterns from past data and can struggle when faced with circumstances outside what they have encountered during training. In those cases, he said, models may fall back on generic strategies, including producing averaged or “safe” answers.
Simply feeding models textbooks is not enough, he added, because they may repeat definitions and formulas without grasping how symbols relate to the physical world or what conclusions follow from an equation.
Biecek said a key shift has taken place in the last few years. Researchers once focused on how to increase public trust in AI, he said, but are now increasingly concerned with how to reduce excessive trust.
He argued that many language models are optimized to satisfy users rather than to deliver correct solutions.
That, he said, becomes especially dangerous in high-risk areas such as medicine or defense. Even specialists can lower their guard when an authoritative-sounding system offers confident suggestions, he said, leading to mistakes they otherwise would not make.
'We must teach AI not to hurt us'
He also raised concerns about how AI systems interact with users.
General-purpose chatbots do not reliably know who they are talking to, he said, whether it is a child doing homework, a researcher analyzing data, or someone generating entertainment. A one-size-fits-all model may fail to meet users’ needs and, in some situations, could cause harm.
As an example, Biecek pointed to sycophancy, a tendency for some AI systems to flatter users. He said this can produce harmful effects in rare cases, including severe psychological reactions.
He added that some recommendation algorithms can steer users toward damaging behaviors or content, and that this can sometimes contribute to depression and even reinforce suicidal tendencies, especially among children and teenagers.
“We must teach AI not to hurt us,” Biecek told the PAP news agency, arguing that both younger users and older people can be particularly vulnerable, for different reasons.
He said Poland could benefit from the wider technological transformation now underway, but warned that society’s response, including impacts on the labor market, remains uncertain.
(rt/gs)
Source: naukawpolsce.pl