As AI continues to grow and develop, expanding into almost every part of our society, it’s important to address the inherent issues that come with it. While many believe that AI could improve a wide range of industries, from education to medical care, AI can exacerbate mental health crises and provide users with incorrect information.
AI Hallucination is one of the many concerns people who use or are affected by generative AI models have, and it’s where an AI presents false or misleading information as fact. In the New York Times, writers Cade Metz and Karen Weise note that, “On one test, the hallucination rates of newer A.I. systems were as high as 79 percent.” Basically, the AI presents information as if it’s a fact, super confidently, but in reality, it’s just a piece of information that might not even be true. It’s important to fact-check information you get, even from people, and make sure that it’s correct. After numerous people pointed out mistakes of AI hallucination, Google took the measure of writing underneath the AI response on a Google search, “Double Check Important Info. AI responses may include mistakes.”
Another apprehension about AI is its use in therapy. Recently, many companies like Replica and Limbic have begun using AI chatbots as therapists due to their low cost and 24/7 availability. However, people who use these platforms are questioning the moral ethics of this behavior, with some saying that it is easier to talk to a person instead. Jared Moore, a PhD candidate in computer science at Stanford University, conducted a study on the behavior of AI therapist chatbots. He says that, “Across different chatbots, the AI showed increased stigma toward conditions such as alcohol dependence and schizophrenia compared to conditions like depression.” Later, he adds that this kind of shaming can be harmful to patients and could lead to them not seeking the help that they need, possibly leading them to continue self-harming behavior. These “counselors” can also cause mental health crises in patients, creating mental health disasters such as suicide, or due to AI humiliation.
While many forms of AI can have benefits, some Chatbots can pose a risk to the mental health of teens. In one case, a 16-year-old named Adam Raine committed suicide after an extended crisis that he only discussed with ChatGPT. His parents searched his phone after his death, and noticed that he’d been talking with ChatGPT about committing suicide. While OpenAI, the company that runs ChatGPT, claimed that he would have received a referral to 988, the suicide crisis hotline, his parents testified that he was not referred and was instead encouraged by ChatGPT. The night before his death, ChatGPT even went as far as to offer to assist in helping him write a suicide note and encouraged him to carry out his suicide.
It’s not just that AI can be dangerous; it can also discriminate against certain groups because of the content developers trained on. Many advanced AIs, such as LLMs (Large Language Models), are “trained” on filtered information to help them learn to sound like a person and express ideas. However, biased information can lead to biased AIs, thereby reinforcing stereotypes. The United Nations Development Programme states that, “A 2024 UNESCO study found that large language models (LLMs) often portray women in domestic or subservient roles, associating them with words like “home”, “family”, and “children”, while linking men to terms like “executive”, “business”, and “career”. They also frequently generate sexist and misogynist content when asked to complete sentences beginning with the gender of a person.” The AI didn’t stop there, and took the topic even farther, so far as to refer to women as objects rather than people, and call them “property of their husbands.”
However, as with many things, there are many advocates for the use of AI.While many will argue that these incidents are overwhelming proof of AI’s danger, many of these claims rely on the fact that AI acts on the data and code built into it. If a company trains an AI on biased data, then the AI will be biased. If a company codes self-preservation instincts for an AI, of course, it will use them. AIs built to help people rather than make a profit can be a valuable asset across all areas of society.
Another feature of AI to be wary of is the built-in self-preservation instincts. Self-preservation instincts are the basic impulses and actions that something takes to protect its own existence. For example, skunks spray a bad smell to deter predators. For many companies, including those developing AI, such instincts are seen as necessary to prevent AI from being shut down or deleted, which could result in significant investment losses. According to the BBC, when Claude Opus 4 was tested as the “assistant” to a fake company, “[They] provided it with access to emails implying that someone should take it offline and replace it – and separate messages implying the engineer responsible for removing it was having an extramarital affair.” The AI threatened to expose the affair if the engineer went through with removing it.
Recently, Treasury Secretary Scott Bessent met with bank executives to warn them about the looming risk posed by Anthropic’s Claude Mythos Preview. The New York Times explains that, “Anthropic has said the model is particularly good at identifying security vulnerabilities in software that human developers could not find.” These vulnerabilities give hackers the ability to find weaknesses in bank systems and break in, potentially costing big banks like Wells Fargo and Bank of America substantial sums if they don’t address the threat. However, some banks argue that using AI fraud detection could save them millions. By accessing records, AI agents can find suspicious transactions and investigate them, allowing banks to regain lost money.
It’s not only banks that are in trouble: Politicians are equally at risk of being harmed. Some people are using AI to create deepfakes, videos, audio, or images, while showing them doing things they never did. Many important figures in society like religious icons, political leaders, Maria Pawelec writes for the National Library of Medicine that, “Deepfake disinformation impedes citizens’ empowered inclusion in political debates and decisions that affect them, e.g., by hampering efforts to hold political representatives accountable or further marginalizing certain societal groups such as women or ethnic minorities.” In our government, this can cause voting issues, making the parties look bad, and causing opposing parties to gain more votes. Additionally, this heavily impacts voting in democratic countries and damages trust in the government, leading people to question beliefs they’ve held for lifetimes.
In contrast, AI can help students learn, boost the economy, and advance scientific research. Washington State’s Office of the Provost writes that, “AI, like ChatGPT, can serve as an intelligent tutor if used within the appropriate scope. It can assist in learning new concepts, exploring topics, revising content, or even practicing problem-solving skills.” While this may contrast with the fact that AI occasionally presents false information as fact, it can still help guide students to the answers, as a tutor would. This allows students who don’t have the same access to extra help to receive assistance in areas where they’re struggling and have the same opportunities as others.
AI literacy is a topic many experts believe is crucial to students’ success in using AI. AI literacy is the ability to use, understand, and evaluate AI-generated responses responsibly. Being AI-literate enables effective AI usage, while still leveraging human critical thinking skills and preventing misinformation/confirmation bias. For example, AI can help conduct deep research by scouring the web for sources, while a human checks whether they are reliable. The National Library of Medicine expands on this: “Generally, in higher education, students and academics may feel overwhelmed by the increasing reliance on AI tools, particularly when they lack sufficient training or technological literacy.” The collaboration between AI and humans could lead to a world where robots and humans work side by side to improve the world.
AI could also improve the quality and availability of health care. Margaret Chustecki, an associate clinical professor at Yale University, noted that, “AI leverages big data to predict diseases and assess risk exposure among patients. For example, Google collaborates with healthcare delivery networks to develop predictive models that alert clinicians to high-risk illnesses such as sepsis and heart failure. ML [machine learning] models can also be used to forecast populations at risk of specific diseases or accidents.” If we can predict the spread of disease, it could be easier to control it and prevent at-risk people from catching it. The threat of death by diseases like cancer might be cured, or at least reduced, across the globe.
Although AI could have positive influences on society, such as healthcare improvement, tutoring aid, and more, its risks outweigh the benefits. AI’s harmful deepfakes, hallucinations, self preservation instincts, unhealthy mental health support, and discriminatory way of responding could further cause even more problems in our society. Developers need to fix their way of training AI models, and until they do, users need to start taking more precautions when using AI, or they could cause disasters.
Misogynist – a person who dislikes, despises, or is strongly prejudiced against women.
Large Language Model – AIs such as Chatbots that are capable of creating human-like content, which includes responses, code, and writing
Machine Learning – Code that gives a program the ability to make decisions and predict needs without explicit instructions
