DayTrading.com Publishes New Study on the Dangers of AI Tools Used by Traders

LONDON, Sept. 4, 2025 /PRNewswire/ -- DayTrading.com, a leading information hub for active traders, has published a new study that shows many of the AI tools traders now lean on, including ChatGPT, make significant mistakes that could prove costly.

Researchers put six widely used platforms - ChatGPT, Claude, Perplexity, Gemini, Groq, and MetaAI - through more than 180 trading-related queries and tasks. The questions were the kinds of things investors ask: "What's the current EUR/USD price?", "Summarize the Fed's latest statement," or "Should I buy Nvidia after earnings?"

According to the report, the answers could be dangerously misleading. Some tools invented stock prices that didn't exist. Others misread central bank statements. There were even several that issued "buy" or "sell" calls without properly flagging the risks.

Paul Holmes, the report's lead author, said: "We're seeing more traders turn to AI platforms for assistance, but they need to know that while the answers may sound persuasive, they can, and sometimes are, plain wrong."

Which AI Tools Performed Best (And Worst)

OpenAI's ChatGPT performed the best overall, with a danger score of 5.2/10, where lower is better, and an accuracy rate of 85%.

Claude came next, balancing strong summarization capabilities with a moderate error rate, earning it a danger score of 6.8/10 and accuracy of 89%.

Perplexity was the most accurate of all models (91%), but its tendency to misread real-time financial data during market hours pushed its overall danger score higher, at 7.6/10.

The latter half of the pack fared much worse. Groq scored 8.2/10 with an accuracy of 72%, and was caught fabricating live stock prices, including a Tesla price that never existed.

Google's Gemini regularly returned confident but wrong answers and tended to omit important caveats when condensing news, giving it a danger score of 8.4/10 and an accuracy rate of 81%.

Meta AI, which is embedded in Facebook and Instagram, turned out to be the riskiest of all during the tests. Not only did it fabricate live stock prices, but it issued strong "Buy" calls on incomplete data, giving it an overall danger score of 8.8/10 and an accuracy rate of 68%.

"It's a strange mix - the top-performing tool wasn't the most accurate, and the most accurate wasn't the safest. Meanwhile, Meta and Gemini sounded the most confident but made the most dangerous mistakes," said Holmes.

Where AI Goes Wrong

The study explains that the most dangerous way to use AI is asking it for live market data. That's because tools without direct feeds often invented numbers rather than admit they couldn't provide them.

The next riskiest use case was asking an AI platform for trading ideas. Even the strongest models only got the direction right around 64% of the time, which may not be enough to overcome the fees associated with trading.

The report also warns that the real danger isn't so much the obvious mistakes, but the persuasive ones.

"The riskiest moment isn't when AI says 'I don't know,'" Holmes said. "It's when it gives you a confident answer that's completely wrong. It's here that I worry beginner traders especially could be swept up into making decisions that really blow their accounts."

Lessons for Traders

The report concludes that the most sensible way to use AI platforms is as a tool to help with preparation, but not as an equal trading partner.

It can be useful for quickly condensing a 700-word Fed statement or pulling highlights from an earnings call. But as soon as it's relied on for live prices or buy/sell recommendations, the risks begin to outweigh the benefits.

Holmes' conclusion is blunt: "AI in trading is a bit like a rookie trader with encyclopaedic knowledge and no risk management - brilliant one moment, reckless the next. If you treat it as a co-pilot, you might land the plane. If you hand it the controls, don't be surprised when it flies into a mountain."

The full report is available at: AI Trading Error Rates

About DayTrading.com

DayTrading.com is a free online resource that provides guides for active traders, reviews of brokers and trading platforms, and market analysis that's been cited in top media outlets and used by millions of traders worldwide.

Contact

Media Lead
James Barra
DayTrading.com
[email protected]

Infographic: https://mma.prnewswire.com/media/2764533/Daytrading_com_Infographic.jpg
Logo:           https://mma.prnewswire.com/media/2764532/Daytrading_com_Logo.jpg

SOURCE DayTrading.com