Five Reasons Why ChatGPT Can't Stop Lying
My last blog post described some of the fun and interesting things that AI engines can do for you. I introduced the concept of a Large Language Model (LLM) and included a few current examples. You can easily see the potential of the technology from these examples. In fact, some pundits think that LLMs and their ilk will literally advance human civilization by being the basis of a broad range of innovative products and tools, ultimately creating AI engines that exceed humans in intelligence and capabilities!
AGI Utopia!
(Some are referring to this potential utopia as Artificial General Intelligence, or AGI, and that it is an inevitable consequence. Others are saying AGI is impossible to achieve.)
However, as I reported last week, today's LLMs can sometimes "hallucinate" and make up inaccurate facts. They can also turn on you and (if unfiltered) can say politically incorrect things. These behaviors are holding the technology back from broad deployment as useful products and tools (that all these pundits are hoping for).
A Hallucinating Computer
In today's blog post, I will give some insight into five reasons why LLMs are falling short, with an eye towards a few exciting future applications if these issues are addressed.Reason #1: Lack of referencing
The most obvious issue is this so-called hallucination problem, where the AI engine literally makes up facts that are not true. The example I cited last time was the incorrect numbers that ChatGPT gave me on Tesla's quarterly earnings, wrapped in a very credible sounding press release. You would think that this has an easy fix... Referencing! Let's just have ChatGPT fact-check, or do referencing, for all of its statements using credible sources before spewing forth its output!
It turns out that it is not easy to extend ChatGPT (in its current version) so it can incorporate referencing. One key reason is that since the language model doesn't have any understanding of the meaning of the words, it can't automatically tell which are the facts that need checking and how exactly to check them.
For example, it doesn't know Paris, France from Paris, Texas (or even from your girlfriend's name if it happens to be Paris)! And, it does not know the difference between Tesla the car company and Tesla the inventor.
If you want to visit the Eiffel Tower and ChatGPT tells you that Dallas is the closest international airport to Paris, it can't check this by referencing Paris with internet sources since it doesn't know it is referring to a city (even worse: if your girlfriend Paris lives in Dallas and is currently visiting the Eiffel Tower? Imagine the confusion!)
If the program just knew Paris and Dallas were cities with geographic locations, and it also knew that the Eiffel Tower had a geographic location, it could easily check Paris and Dallas for latitude and longitude and conclude Dallas is not the right city to fly into. There are some future software technologies on the horizon, broadly under the umbrella of something called the Semantic Web that might be able to fix this issue by putting the various meanings of "Paris" "Dallas" and "Eiffel Tower" into the proper categories. But in the meantime:
Reason #2: Non-transparent AI
If you can't fact-check with credible references, then the next logical approach might be to crack open the code and troubleshoot how the error was made in the first place. However, this is also impossible (with today's technology)! It turns out that the internal workings of the multi-layered neural network that forms the basis of an LLM are completely opaque to researchers. With non-transparent AI, you really never know how it got the answer it did. So, it is impossible to trace back the logic that might have resulted in the incorrect fact.
There is a nascent field of research underway called Explainable AI (XAI) that might someday yield some progress in this area. But until mature code comes out of XAI research, we are stuck with a black box, where it is impossible to attribute the LLM's result to anything other than a good guess.
Let's say somehow we can magically get around the non-transparency of the AI and incorporate referencing into LLMs (perhaps using a combination of XAI and Semantic Web technologies). Unfortunately, the updated (referencing) version of ChatGPT will still be hallucinating. We still have three more significant deficiencies that need addressing before we will have a sober AI engine.
Reason #3: Lack of Compositionality
You might have heard the childhood rhyme:The toe bone's connected to the foot bone,
The foot bone's connected to the ankle bone,
The ankle bone's connected to the leg bone,
Now shake dem skeleton bones!
ChatGPT didn't grow up chanting that particular rhyme. Our new (hypothetical) version of ChatGPT that incorporates referencing can tell you the toe bone doesn't weigh 20 pounds but it doesn't know the toe bone is connected to the foot bone.
Researchers call this issue "lack of compositionality" and it simply means that the LLM doesn't have a clue what is connected to what in the physical world (or even what the "physical world" or "connected to" means). So, even though it won't make the mistake of telling you that you need new, reinforced shoes to carry the additional weight of a 20 pound toe, it might somehow create text that implies your toe is directly attached to your pelvis!
That would be bad. And inaccurate. (By the way, if you ask ChatGPT how to carry the weight of a 20 lb. toe, it would indeed provide an answer!)
And we still have a couple more problems to solve.
Reason #4: Lack of a Cognitive Model
Humans make sense of the real world using cognitive models. For example, two individuals will likely have the same cognitive model of a car. It allows them to communicate about cars in general or about a specific instance. In addition, the individuals can infer and draw conclusions based on the model – e.g., if a car that was working yesterday is not starting today, they might reason that it is out of fuel. When applying AI techniques to text-based sources, this type of conclusion is difficult to reach without such a model.
If ChatGPT is to become your internet friend and help you troubleshoot your car, it needs to somehow be able to refer to a cognitive model of a car to make sense of your difficulty. (And, it needs to understand the context - that your car was working yesterday versus being out of commission.) Otherwise, it might have you changing your fan belts instead of putting more gas in the tank.
Reason #5: Lack of Alignment
Software companies are finding that an inherent lack of alignment between AI engines and generally accepted human norms is perhaps the most difficult to address. This issue has caused Microsoft and Facebook to withdraw LLM-based products from the marketplace and significantly delayed Google from shipping an LLM-based chatbot (Google Bard, which is Alphabet's initial offering was announced on February 6th, 2023).
Today, when engineers design any piece of new software technology that is going to be released into general usage, they need to make sure that the resulting product doesn't somehow get out of line with generally accepted societal principles. Needless to say, nobody would permit a Fascist Furby to come to market.
Indeed, this is straightforward to do for just about any conventional product that contains software (e.g., cars, smartphones, toys, vacuum cleaners, etc.). Properly written product specifications, software design practices, engineering principles and user testing would quickly prevent any of these kinds of issues from happening before shipping. The resulting product can be trusted not to deviate beyond what it is designed to do.
However, for any hypothetical product that fully relies on an LLM, preventing a Racist Rosie the Robot from coming into existence is not so easy to do. Since these computer programs are not aware of the meaning behind the words they use, it is impossible for them to avoid learning from the thousands of examples of politically incorrect things on the internet. And just like a three year old that hears a swear word from uncle Bob, the AI will learn them and parrot them back to you.
If you don't believe me, just ask Microsoft about Tay.
There is no pragmatic way to fully exclude all egregious material from the training set - in fact, the technology learns not only from obviously racist and misogynistic content, but incorporates all of the subtle but important biases that are ever present on the internet.
You have to keep the goals of your AI aligned with the good of humanity, and not accidently or otherwise aligned for ill-suited purposes. Otherwise, you might end up with a paperclip-forming robot trying to take over the world by forming paperclips out of everything, including humans!
One of the ways that ChatGPT is starting to address the alignment issue is by brute force. According to several sources, they have hired thousands of humans to issue queries to the program and manually fine tune the responses to maintain alignment with good and civilized principles. This approach might filter much of the egregious content (or edit the output) but certainly can't scale to very large and widely deployed applications, or when applications try to learn from anything on the web (versus these filtered inputs). And, there is likely a lot that will be missed!
In summary, it is clear that LLMs and AI have lots of useful potential applications in the future. It is likely we will be interacting with them in more ways than can even be imagined today. But before they can become deployed at large scale, these five classes of problems will need to be solved.
Whether or not LLMs enable AGI and significantly advance human civilization is, however, yet to be seen!
Comments
Post a Comment