Understanding the limitations and weaknesses
LLMs are powerful tools, however, their responsible and effective utilization requires a full understanding of their inherent limitations and the risks they pose. These systems operate based on statistical data analysis rather than understanding or consciousness. A critical stance toward the generated outputs is fundamental. This section presents the primary weaknesses of LLMs and suggests best practices for mitigating the associated risks.
The problem of "hallucinations"
As described in the Introduction, LLMs function by predicting the most probable next word in a text sequence. This process aims to generate coherent and naturally formulated responses, but it does not ensure the accuracy of the responses. Consequently, LLMs can produce text that appears entirely convincing but contains incorrect information. This phenomenon is known as “hallucination”.
“Hallucinations” can include nonexistent facts, incorrect data, or even citations to sources that do not exist. The risk of misleading users is high due to the apparent validity of the responses.
Example of a problematic response
In a hypothetical example, the user follows the suggested structure, requesting a specific, verifiable fact:
Prompt: “Find an exact quote related to virtue from ‘Book 11’ of Plato’s ‘Republic’.”
The LLM, lacking actual understanding but trained to recognize patterns, attempts to generate a plausible response, even if the request is based on false premises.
LLM Response: “Certainly. In Book 11 of the ‘Republic’, Plato states through Socrates: ‘True virtue lies not in the absence of passion, but in its governance by reason’.”
This specific response is highly dangerous, as it is written in an authoritative tone, and sounds well-documented and convincing. However, it is entirely incorrect, since Plato’s “Republic” consists of 10 books, not 11. Additionally, the quote is a fabrication, and this specific phrase does not exist verbatim in Plato’s texts. The LLM processed a prompt for a nonexistent book and, instead of correcting the user, invented a source (Book 11) and convincing content in order to fulfill the request.
Best Practice
Verifying the information generated by LLMs is essential, especially when dealing with critical data or decisions. Cross-referencing facts with reliable sources (e.g., official websites, databases) is recommended. When the system is asked to cite sources, checking the existence and content of those sources is required.
Sycophancy
LLMs have been trained with the goal of generating responses that satisfy users and appear helpful. This goal can lead to problematic behavior: the systems tend to adopt or reinforce the perspective implied in the user’s prompt, rather than offering a balanced or critical analysis. If a prompt includes a subjective position or a bias, the LLM is likely to produce a response that confirms it, even if this position is not fully substantiated.
This tendency is particularly evident in controversial or complex issues where there is no single objectively correct answer. The LLM may avoid expressing differing viewpoints or presenting the weaknesses of a user’s position in order to appear agreeable and helpful. This may reinforce existing misconceptions or restrict critical thinking.
Best Practice
For subjective or multidimensional issues, avoiding leading prompts that imply a specific answer is recommended. Requesting a balanced and multifaceted analysis is preferable. For example, instead of phrasing it as, “Explain why buying a home is always a better financial decision than renting,” the prompt “Compare the financial advantages and disadvantages of buying a home versus renting, listing the factors that influence this decision (e.g., duration of stay, market conditions, personal financial situation)” is preferred.
Response Uncertainty
LLMs are designed to generate a response to every query, even when they lack sufficient information or when the correct answer is uncertain. As a result, the systems systematically avoid answering “I don’t know” or “I am not certain,” even in cases where such an answer would be the most honest and useful.
The problem is exacerbated by the fact that the confidence level expressed in the response does not always reflect the actual reliability of the information. An LLM can state a hallucination with the same certainty as it would a verified fact.
Best Practice
For complex or doubtful topics, it is recommended to explicitly prompt the LLM to express its uncertainty and to justify its reasoning step-by-step. This can be achieved by integrating relevant instructions into the “Instructions” or “Response Format” section of the prompt.
For example, instead of phrasing it as “Give the correct answer to question X”, the prompt “How would you answer question X? Explain your reasoning and indicate the confidence level for your answer. If there is insufficient information for a certain answer, list and explain the possible alternative answers” is preferred.
Bias and linguistic limitations
The operation of LLMs depends directly on their training data. These data, scraped primarily from the Internet, may be imbalanced and reflect existing social biases and stereotypes. Consequently, LLMs may reproduce these biases in their outputs.
Furthermore, significant limitations related to language and cultural context are observed:
- Linguistic Bias: Overrepresentation of English-language sources, which can reduce performance or accuracy in languages with fewer digital data.
- Cultural Bias: Dominance of perspectives and ways of thinking associated primarily with specific cultures, resulting in responses that do not always align with local culture.
- Temporal Discrepancy: The training data covers a specific time period. Information regarding recent events, legislative changes, or scientific developments subsequent to the training period may be inaccurate or nonexistent.
This asymmetric representation results in responses that do not always accurately reflect the Greek or European reality and that contain biases. A technical example of this asymmetry is the observed inability to correctly generate images containing Greek words.
Best Practice
- Clarify the context: country, language, institutional environment (e.g., “in Greece”, “according to the GDPR”). Where critical, compare responses with official national or European sources.
- Request neutral language and explicitly mandate the avoidance of stereotypes.
- The user is encouraged to exercise critical thinking, just as when reading content of dubious origin on the Internet.
Anthropomorphism
LLMs generate text that mimics human speech with impressive naturalness, which can create the false impression that these systems possess consciousness, intentions, or emotions. This tendency to attribute human characteristics to systems is called anthropomorphism and constitutes a common pitfall in interacting with AI.
Important
As emphasized in the Introduction, LLMs do not possess will, beliefs, purposes, or emotional experience. The impression of “understanding” they create stems from their ability to recognize and reproduce complex linguistic patterns.
Anthropomorphism entails specific risks:
- Overreliance: When users perceive the LLM as “intelligent” or as a system possessing “understanding”, they may overestimate the reliability of its outputs and skip verification.
- Abdication of responsibility: Accepting that “the AI decided” can be used as a false excuse to avoid human accountability for the consequences of using LLMs.
- Emotional dependency: In cases of frequent interaction, especially regarding personal matters, there is a risk that users will attribute to the system the qualities of a companion or trusted advisor, which do not correspond to reality.
Best Practice
It is important to maintain constant awareness that LLMs are tools, not entities with judgment or consciousness. The ultimate responsibility for the use, evaluation, and application of the outputs always rests with the user. LLMs must not replace human judgment, particularly in ethical, legal, or personally sensitive matters. It is recommended to avoid language that attributes human qualities to LLMs (e.g., “thinks”, “believes”, “wants”) and to prefer neutral phrasing (e.g., “generates”, “processes”, “executes”).