Evaluating Large Language Models for Table Data Extraction from Annual Reports in PDF

This paper assesses which Large Language Model (LLM), ChatGPT 3.5 or Google Bard (Bard), performs better in extracting tabular data from annual reports of major publicly traded companies in Portable Document Format (PDF).Through an examination of 40 experiments with different difficulty levels, this study delves into the capabilities of these LLMs in processing and extracting financial information.The evaluation metrics, including relevance, accuracy, completeness, consistency, and context awareness reveal varying degrees of proficiency when handling complex financial data for both ChatGPT 3.5 and Bard.In our study, ChatGPT 3.5 demonstrated superior performance, particularly on more challenging questions.The findings offer valuable insights into the utility of LLMs in financial data extraction and provide recommendations for future research.