What Is an AI Hallucination and How Does It Impact Business Leaders?

How to identify and avoid hallucinations

As more organisations turn to generative AI tools such as large language models (LLMs), they must understand the risks. One of the risks that come with LLMs is AI hallucinations.

According to Gartner, AI hallucinations are completely fabricated outputs of LLMs. While they represent false information, the LLM presents the false facts with confidence and authority.

Tong Zhang, senior director analyst at Gartner spoke during the Data and Analytics Summit in Sydney on how AI hallucinations are a key risk for business.

He said looking at AI hallucinations from an enterprise perspective, users and consumers are looking for accurate information so if you let the LLM to hallucinate there is no point in using it.

“This is the first risk that we need to take control of,” he said.

This can be done through content filtering and asking a question in a clear, concise way to ensure there is no inaccurate data sent to the models.

Some of the most common AI hallucinations that can happen according to Zhang include partially true outputs that are wrong on important details, completely fabricated outputs, and training data bias resulting in biased outputs.

“Large language models (LLMs) often have a “knowledge” cut off, so information may be out of date,” Zhang added.

Hallucinations are already being played out in real life, Zhang uses an example of a New York attorney who used a generative AI-based chatbot to do his legal research.

“The federal judge overseeing the suit noted that six of the precedents quoted in his brief were bogus. It turns out that not only did the chatbot make them up, it even stipulated they were available in major legal databases,” he explained.

How can AI hallucinations be avoided?

To avoid AI hallucinations within the workplace, Zhang said business leaders can leverage internal knowledge to provide an accurate LLM.

“This means finding the exact answer from an internal knowledge base and providing the right answer to AI only for conversation,” he explained.

“They can also do prompt engineering to control the input and output with content filter and human validation.”

Zhang added other ways to avoid hallucinations are to ingest business knowledge into AI via fine-tuning, or to designate a red team to do adversarial attacks for those models.

For developers building and interacting with AI models, Zhang recommends that developers should focus on using better sample data for the fine-tuning of foundation models, as well as building content filters for input and output layers.

“It’s also important to collect human feedback to continuously optimise the model,” he added.

While hallucinations may be a risk for businesses when implementing LLMs within their business processes, Zhang said as ChatGPT continues to evolve, the percentage of hallucinations appears to be decreasing.

“For example, in both March and August, I asked ChatGPT to write me a biography. Back in March, it produced a version with a lot of incorrect information. However, in August it doesn’t seem as willing to answer something it doesn’t know,” he said.

By Athina Mallis

Source: ITnews