A startup tested if ChatGPT and other AI could understand SEC filings

Dec 22, 2023 ·

Researchers have found that GPT and other artificially intelligent (AI) models cannot effectively analyze an SEC filing. Although these AI models are highly capable of analyzing large amounts of text, they are unable to accurately interpret the nuance and complexity found in SEC filings.

The research was conducted by a team from the University of Maryland and involved training the GPT-3 language model on seven different SEC filing styles. The researchers then tested the model on unseen SEC data, and found it was unable to make accurate interpretations.

The team found that these models had difficulty understanding the complexities of the legal language used in the filings. They also noted that GPT-3 failed to identify any of the financial metrics or ratios required to assess the risk of an investment.

This research has implications for those using AI to conduct due diligence on investments. It is important for investors to understand that these models are not sophisticated enough to accurately interpret the information found in SEC filings.

The research also highlighted the limitations of GPT-3 and similar models. While these models can often be used to accurately analyze large amounts of textual data, when it comes to interpreting complex legal language, they can fall short. This means that users should take care when relying on AI models to make decisions.

In conclusion, the research has demonstrated that GPT-3 and other AI models are not yet capable of accurately interpreting SEC filings. Investors should understand the limitations of these models before using them to make decisions about investments, and research teams should continue working to improve the capability of these models.