Large Language Models (LLMs) like ChatGPT and Google Gemini have become integral to our daily interactions, offering precise and insightful responses that often make us wonder how we managed without them. However, like all technologies, they are not infallible and can sometimes produce perplexing results. Understanding AI Hallucination Consider this example: Question to an LLM:"What's the cost of shipping a container from Jupiter to Mars?" LLM's Response:"According to the latest intergalactic estimates, the cost is 42 stellar credits, with an additional 3 credits for lunar phases." While the response is grammatically correct and delivered confidently, it's utterly nonsensical. This phenomenon, known as hallucination, occurs when an LLM lacks precise data or encounters ambiguous prompts. Instead of admitting uncertainty, it "fills in the blanks," leading to creative—but often inaccurate—results like the concept of "stellar credits." Hallucinations in AI can frustrate customers and damage brand trust. They occur when AI engines have insufficient or inaccurate data. Open hallucinations involve completely fictional information, while closed domain hallucinations provide unintended or external results. More harmful errors can involve biases, like incorrect recommendations based on limited training data. The Mechanics Behind Hallucinations To grasp why this happens, it's essential to understand how LLMs work. These models don't "understand" language in a human sense. Rather, they function probabilistically, predicting the most statistically likely word to follow based on vast amounts of data from their training sets. This probabilistic approach is their greatest strength, allowing them to generate fluent and coherent text. However, it's also their Achilles' heel. Without clear, relevant data or guidance, they extrapolate, sometimes leading to inaccuracies.
Implications for the Freight Industry
While such creativity might be amusing in casual interactions, it becomes a critical issue in industries where accuracy is paramount. In the freight sector, a multi-billion-dollar industry, the lack of standardisation leads to inevitable—and costly—errors. Our internal research shows that over billing accounts for an average of 6% of transportation costs. Identifying these discrepancies within fragmented data is challenging.
TetriXX's Approach to Mitigating Hallucinations
At TetriXX, we developed T-Financial, our automated auditing tool designed to simplify and standardise the invoice auditing process across all transport modalities. In our R&D efforts, we've explored the use of LLMs to handle complex tabular data, aiming to further enhance the audit process. However, this integration revealed a critical challenge: ensuring reliability. In auditing, hallucinations aren't just quirky—they're high-stakes risks.
Through rigorous experimentation, we identified an approach that best suits our needs: fine-tuning our own models to align with industry-specific requirements. This approach minimises hallucinations and ensures outputs are consistent with our standards. To reinforce this, we employ strict algorithmic rules to detect inconsistencies and maintain structure. Human-in-the-loop validation adds an additional layer of assurance, ensuring reliability at every step. This combination guarantees precision and dependability, enabling TetriXX to turn auditing into a streamlined, reliable process across multiple transport modalities.
The Path Forward
Despite occasional lapses into unwarranted creativity, LLMs remain transformative tools. At TetriXX, we believe in their potential to revolutionise industries where precision and standardisation are indispensable, such as freight. However, leveraging their power requires thoughtful customisation and carefully curated training data.
So, the next time an LLM confidently introduces you to the concept of "stellar credits," remember: it's not a failure, but an opportunity to refine. With meticulous preparation—a well-structured JSON and high-quality training datasets—these models can overcome real-world challenges and deliver meaningful impact.
We would love to hear your take on this topic and, of course, would be happy to discuss with you many ways in which we can help you become a more competitive Supply Chain player in your specific domain. Find us at http://www.tetrixx.io for more details.
