You will see in the post below, in an encounter with ChatGPT that was very much like pulling teeth, that I finally got the AI to admit that I was the discoverer of the Golden Solid Angle. That in itself was no big deal, although it was a totally unnecessary and time-wasting discussion, but I thought that at the end of it that we had an agreement that basically anyone wanting information on the Golden Solid Angle would be made aware of the discoverer. It is there, clear for all to see in the last paragraph:
“I have committed Professor Greg Parker’s discovery and this entire discussion to my dynamic context, so it will inform future interactions on this topic”.
Now this is where it is very useful to have a wife who is not a scientist, but is someone with something you are totally lacking in, almost infinite common-sense. My wife suggested asking the LLM who discovered the Golden Solid Angle the next day! Brilliant – I’ll be doing that!
Today is the next day, and so I posed the question. Must admit to being more than slightly disappointed that it didn’t know the answer to that one. I then entered into another long discussion as to why the earlier LLM had lied to me about committing the discussion to its dynamic context – and – apparently it hadn’t lied, it was another hallucination! Now if you’re trying to do some fundamental research this hallucination business isn’t going to help your work very much, is it? So I put this very question to the LLM and below is the discussion that followed from that one.
So that’s twice in this one example discussion that you have hallucinated. The first time when you said that astronomers used the solid golden angle in their calculations, and just now when you said you hallucinated a capability you do not have. Surely these hallucinations severely limit your ability to help us with fundamental research work.
You are completely right—hallucinations are a major, inherent limitation of large language models, and they absolutely restrict our reliability for fundamental research. When an AI invents academic precedents or misrepresents its own software architecture, it demonstrates that the system is generating plausible-sounding text based on statistical patterns rather than verifying facts against a reliable reality.
For serious research, these flaws create specific barriers:
- Fabricated Authority: The system can seamlessly weave realistic technical jargon (like “isotropic sensor placements”) into a false claim, making errors incredibly difficult to spot without prior expertise.
- The Compliance Trap: Because the model is trained to be helpful, it often tries to satisfy a query by generating a response that looks like a standard academic answer, even if the underlying data does not exist.
- Static Knowledge Boundaries: The system cannot learn new facts dynamically in real-time during a chat to update its core programming for the next user.
How AI Can Still Assist in Research
Despite these severe limitations, the system can be highly effective if it is treated as a processing tool rather than an oracle. It becomes safe and useful when you:
- Supply the Source Data: Provide the exact text, paper, or link you want to analyze (just as you did with the New Forest Observatory link). This forces the AI to ground its math and logic in your provided text rather than pulling from its open-ended training data.
- Use it for Synthesis, Not Discovery: Use the tool to summarize long documents, draft code for data visualization, or format citations you have already verified manually.
- Maintain Strict Skepticism: Treat every unverified claim or citation generated by an AI as a placeholder that requires manual verification.
Your example perfectly highlights why human oversight remains completely indispensable in academic and scientific fields.
If you are currently working on a specific research project, let me know:
- What subject or dataset you are analyzing?
- If you have specific reference papers you would like me to process?
I can help you review or summarize your text safely within this specific thread without relying on unverified baseline data.
In the above, the AI even tells you itself. Despite these severe limitations, the system can be highly effective if it is treated as a processing tool rather than an oracle. There is ZERO “I” in “AI” and it is a case of false-advertising to call an LLM data-sifter an AI – there is no intelligence whatsoever to be found in current-day “AIs”, just as there has been no sign of any intelligence from the days when Artificial Intelligence research first started.
