Skip to main content

Minimize Hallucinations by Updating LLMs with RAG



Large Language Models can be updated by using Retrieval Augmented Generation of High-Quality Data like Journals. This should be done on topics that the user asks using Gen AI. Unnecessary data that no one cares about need not be updated. This updated LLM fragment should be marked as high quality with more weightage given to it. If there is time-bound data, stale data should be marked, and RAG should be performed again every time or whenever necessary.

Problem with Present Large Language Models

The problem with present LLMs is in the frantic attempt to train LLMs, maximum internet was covered without considering the quality of the data in it. We need to rectify it. But it is impossible to start from the beginning and correct everything. Also, there is no need to do it. We can concentrate on the data that is most needed.

Hallucinations

Hallucination does not come from somewhere. It is the improper data fed into the LLMs during the training of the model. If we need to minimize hallucinations or make it negligible, we need to feed good data into the system. This time we can use high-quality data like journals, most page ranked (above a threshold) webpages, authoritative books, articles or books written by credible authors, and news articles, all that is of high quality.

Updating Large Language Models

We can update LLMs when a user asks for data, with the help of Retrieval Augmented Generation (RAG). Normally RAG gets information from the high-quality source or documents of an organization and augments it with the output of the GenAI. We can use RAG to get high-quality documents from reliable sources mentioned above and update the LLM itself. Obviously, We should protect the private documents of the organization, which are not in the public domain. This way LLMs will be of good quality data as time passes when more and more people start using LLMs.

Weightage and Prioritizing Organizations

The Organizations which use the services of LLMs in a large way should be given priority. Because they form a big chunk of the user base and may point to the important data that needs to be updated by RAG. The high-quality data obtained by RAG needs to be updated in LLMs and given high weightage, as it is of high quality. The data that no one asks or asked by a minuscule number of free users, need not be updated at all. Free users should ask for a considerable amount of time, only then RAG comes in and updates the LLM.

Time Bound Data

We need to find the time-bound data in the LLMs and always update with the latest information available with the help of RAG. When data becomes stale, it should be marked stale. Or recorded as old data, which may be required by the user too. Time-bound data are data that are usually available on news media. News media is the biggest source to identify stale data. We can also provide the information's last date/time updated to the user if he/she demands it, explicitly in a prompt.


Photo Courtesy: miro.medium.com

Support Us: https://supportusbydeepak.blogspot.com

Labels:
Location: Chennai, Tamil Nadu, India

Comments

Post a Comment

Popular posts from this blog

Hallucinations of AI and the quality of Inputs

  In this article, we discuss why Generative AI hallucinates. Is it based on the quality of training data? How can we get output that has minimum hallucination? Is Artificial Intelligence biased? How to Mitigate Bias in Generative AI? We discuss these in this article. Introduction You should understand first how AI Models work. AI takes input from innumerable sources of information such as web pages on the World Wide Web, Books, Journals, Articles, and so on. These data are stored as nodes in Neural Networks. Generative AI just predicts the next most probable word. The next Node connection path it should take to churn out the answer to your question. That means no data scientist can ever examine or predict the output of Generative AI. It is just connections between words. How will you examine connections between words?  Hallucinations of Generative AI It is a common misbelief that AI Large Language Models hallucinate and give out non-existent information. It does not make up a...
Post a Comment
Read more

AI learning methods other than Brute Force

  In this article, we discuss the various learning methods by which Artificial Intelligence can learn. We discuss methods like learning a subject completely by branching out to different topics and learning a subject starting from a high-quality source and then to a low-quality source of data available on the internet. Introduction We know that Generative AI tools like ChatGPT and Bard already have learned a good part of their knowledge by brute force. We are suggesting a few methods by which it can learn otherwise. These methods should help it to optimize itself, if not learn from scratch. Re-Inforced learning by adding weights ChatGPT and Bard should add weights to their important sources of information. These sources could be the most visited websites, the most referred websites, acclaimed books from renowned authors, research papers or articles from well-known magazines, Articles, and even the most watched YouTube Channels. These sources need to be given weights and they should...
Post a Comment
Read more