An automated information extraction system from the knowledge graph based annual financial reports

Background & motivation

The potential use cases for this research are explained below.

Alice receives her pension monthly. She is interested in investing it to gain the highest profit/earnings after tax. For this purpose, she has gained access to the publicly available financial reports of various banks. She is trying to find her query by looking at the values and terminologies in the annual financial reports.

Bob is interested in investing in the stock shares of the banks that offer the best cash dividend per share value. To get these insights, he is trying to search for different values from the annual reports.

Carol needs a loan on suitable terms to invest in her business. She is working with various financial reports of banks to search for the banks that give loans at low markup.

In these use cases, the customers are coming across with following challenges:

• The reports are mostly lengthy, detailed, and available in unstructured format and most of the contents may not be relevant for investors.

• The financial reports are in a human-readable form. Machines cannot understand these reports therefore, they are not queryable in an automated way.

• Each company/institution uses its terminologies due to the lack of standardization. This leads to a huge amount of semantic differences.

The current trends suggest using various recommendations through blogs, reviews, and posts on social media. There are two issues with such options:

• i)

  They are based on user experience which may or may not be the true reflection of actual data.

• ii)

  They may not contain relevant information as desired by the user.

Objectives

In order to resolve the issues listed above, an ontology-based information extraction (OBIE) is proposed in which the information extraction process heavily relies on ontologies and has the potential to assist the development of the semantic web (Wimalasuriya & Dou, 2010).

For adapting the system for users as mentioned in Use Cases, the competency questions are designed to show how the system will be used. The competency questions guide the development of the ontology to ensure that it addresses all the inquiries outlined within them. These competency questions are discussed in later subsections. We have used the ontology for matching with the terminologies available in the annual financial reports along with the relevant values of those terminologies. We have also used a regular expression to filter out the precise terminologies to get the relevant values of each terminology.

These reports are in the textual format, containing the financial information about the companies in the form of key terminologies. A Python-based web crawler is developed which is used to crawl the company’s website to search and retrieve the annual financial reports. The details are discussed in later subsections.

The annual financial report in a textual format is converted into the W3C-compliant CSV format by mapping with the ontology to extract the relevant data which helps in standardization. The relevant data is then transformed into the financial knowledge graph (Zehra et al., 2021) to represent it in a machine-readable format for querying competency questions. This research work is the extension of Zehra et al., (2021) with the following contributions:

• i)

  An automated information extraction system has been proposed.

• ii)

  The system has been applied to various banks to generate precision and recall values to effectively answer the competency questions.

• iii)

  The existing ontology in Zehra et al. (2021) has been extended with 3,849 triples and 730 concepts.

• iv)

  Transformation of extracted data to the knowledge graph.

In the proposed information extraction system, we attempted to achieve the maximum amount of precision in results along with the higher recall value. The generated results show that our system provides precise information extraction results while extracting the data from the annual financial reports by matching with the ontology. The details are mentioned in the results section.

The rest of the article is organized as follows. The literature review is available in the next section, followed by the methodology section which contains information regarding information extraction, competency questions, semantic differences, and different steps involved in the entire process, the results of the competency questions and queries are described in the results section, while the article ends with a conclusion, future work, and references.

Information gathering

The work in Sowunmi et al. (2018) proposed a framework, based on the semantic web, learners’ profiles, resources ontology, and a semantic web search engine. This framework is divided into two components: the general crawler and the RDF crawler, both of which are utilized for personalized information retrieval of educational content from the Internet. The search results are filtered based on semantics and user preferences using a reasoning inference engine. A ranking algorithm is employed to assign weights and rank the search results. The system and search management are accessed through a user interface.

The idea of representing the Hundred Schools of Thought by using a knowledge graph is presented by Wei & Liu (2019). Various tools and techniques are used to accomplish this task. Initially, an ontology generation process is initiated to specify the knowledge graph’s domain and relationships, which results in a better systematic and arranged presentation of the information on the Hundred Schools of Thought. A web crawler is used to collect the relevant data from the websites and utilize it to populate the knowledge graph. To ensure a complete database of knowledge about the Hundred Schools of Thought, the web crawler, after a certain period, scans and gathers data from certain websites. A graph database Neo4j is used for managing and storing knowledge graphs. Neo4j can effectively store, retrieve, and query graph data making it simple to navigate and explore the relationships between the schools, philosophers, and their beliefs. The proposed approach combines ontology development, web crawling, and the use of Neo4j as a knowledge graph database to build a comprehensive representation of the Hundred Schools of Thought. This knowledge graph is a helpful tool for understanding and learning about the conventional doctrines and philosophies of many schools.

An ontological method is presented by Hong, Lee & Yu (2019), for automated information gathering on flooring, ceiling materials, and interior wall finishing materials from four different product producers. The suggested method is using a web crawler to browse and gather pertinent data from construction websites that deal with building supplies. The classification of construction and material information is then particularly addressed by ontology. Data about construction materials may be collected and categorized automatically thanks to web crawling and ontology development.

An ontology-based web crawler (WCO) is introduced by Ibrahim & Yang (2019), for retrieving relevant information in the field of education. The WCO utilizes a crawler to gather information by evaluating the relevance between a user’s query phrases and facts within a specific domain’s reference ontology. To rank the crawled concept phrases, a similarity ranking method is employed. This approach enables the discovery of desired pages that may be overlooked by traditional web crawlers focused on educational content. The process begins with the construction of an ontology specific to the domain, which serves as a basis for comparing the user’s queries with the website’s content. The user’s input query is matched with the ontology and forwarded to the search engine module. This phase generates a collection of documents that are processed by the crawling system to verify the accuracy of the web pages. Only acceptable and high-quality web pages are parsed, and their content is compared with the ontology. If there is a match, the page is indexed; otherwise, it is disregarded.

The concept of gathering data from textile websites using web crawling and web scraping techniques is introduced by Muehlethaler & Albert (2021). The objective is to gather data from the websites of major retailers, such as clothing kinds, fibers, and colors. This approach of gathering data makes it easier to spot emerging trends, comprehend consumer wants, and examine consumption trends for various types of clothes. The authors create a web crawler and web scraper built on Python that is intended just for collecting information from store websites serving the textiles sector.

A learning-based focused web crawler algorithm is presented by Kumar & Aggarwal (2021), to increase the effectiveness of web crawlers. The method incorporates a modified crawler policy that helps the web crawler operate more effectively. Based on how often they are changed, the URLs are divided into static, frequent, and often modified categories. Using a k-nearest neighbors (KNN) classifier, it is possible to swiftly separate the relevant and irrelevant URLs. To further boost the effectiveness of the web crawler, the program also suggests ontology-based searching and semantic learning of keywords.

The idea of gathering unstructured data from multiple e-commerce websites is introduced by Thomas & Mathur (2019). The goal is to gather product information to provide ratings and outcomes based on this data. For web crawling, the Scrapy framework is used, enabling effective and organized data extraction from the targeted e-commerce websites. The goal of the study is to provide meaningful ratings and findings by gathering and analyzing unstructured data in order to offer insightful information on product specifics.

The Spacy and BERT machine learning frameworks are suggested by Chantrapornchai & Tunsakul (2021), to determine a hotel’s location and accommodations for travelers. Through the use of crawlers, the project gathers data on lodging, dining, shopping, and travel. The gathered data is organized and structured using ontology. To evaluate and compare the data, the machine learning frameworks Spacy and BERT are used. These frameworks make it possible to analyze and handle the information gathered, which helps the tourist sector choose appropriate areas and lodging options.

A framework is presented by Alexandrescu (2018), which is used to gather information from various online stores, particularly those that offer board and card games. The framework is developed by using Java as a programming language, and MongoDB as a database. The Google Cloud Platform, which provides scalability and reliability for data collection and archiving, is also used to build up the framework. Using board and card games as its primary domain, the framework is designed to crawl and retrieve pertinent data from various online retailer’s websites. Various online scraping techniques are used to extract information regarding product specifications, costs, user reviews, and other relevant data from different websites. For further processing and analysis, the retrieved data is subsequently kept in a MongoDB database. By utilizing Java and MongoDB the system is deployed on the Google Cloud Platform, the framework provides a dependable and scalable solution for obtaining and managing data from various shopping websites related to board and card games.

An automated method for creating datasets is introduced by Jiménez-Ruiz et al. (2020), which matches tabular data with knowledge graphs. Classes, instances, and their characteristics are extracted using this approach. Then, using SPARQL, a set of queries is created for each class. A refining phase is carried out to improve the correctness of the tables for matching purposes before the produced queries or tables are displayed in a tabular style. A smaller group of tables that are more difficult to match are then gathered.

The work in Selvalakshmi & Subramaniam (2018), describes the development of a novel semantic information retrieval system that uses feature selection and classification to improve similarity scores, along with a semantic information retrieval system based on ontology and Latent Dirichlet Allocation (LDA).

A semantic, ontology-enabled search model is discussed by Kyriakakis et al. (2019), which specifically focuses on the mapping and semantic annotation of bioinformatics resources. This search approach utilizes advanced ontologies like EDAM and SWO. Apache SOLR free-text annotator is employed for text parsing and term identification. Additionally, the model includes a speech recognition feature that enables search results based on spoken queries. The performance of the model is evaluated using the SEQanswers bioinformatics forum.

Another novel approach presented recently in Liu et al. (2023) proposes a novel focused crawler that addresses traditional web crawler challenges by integrating an enhanced tabu search algorithm with domain ontology and host information memory. The domain ontology, developed using the formal concept analysis (FCA) method for topic description, guides the crawler’s navigation. An improved tabu search (ITS) algorithm, along with host information memory, is employed to select the next hyperlink strategically. Additionally, a comprehensive priority evaluation method is designed to assess unvisited hyperlinks effectively, mitigating the issue of topic drifting. Experimental results in the tourism and rainstorm disaster domains demonstrate that FCITS_OH surpasses other focused crawling algorithms, enabling the collection of a greater quantity and higher quality of webpages.

Information extraction

OntoHuman is a toolchain designed to engage humans in the process of automatic information extraction and ontology enhancement (Opasjumruskit et al., 2022). It utilizes Ontology-Based Information Extraction (OBIE) to extract information in the format of key-value-unit tuples from PDF documents, guided by ontologies. The application of OntoHuman is versatile and applicable to documents across various engineering domains, offering an intuitive and collaborative approach to working with ontologies for users. The toolchain includes the Document Semantic Annotation Tool (DSAT) and continuously trained ontology (ConTrOn), a standalone application with a web API, which facilitates the automatic extraction of information from PDF documents based on ontologies.

An intelligent web data extraction system, created exclusively for the e-commerce industry, is suggested by Selvy et al. (2022). The data extraction is done by combining the You Only Look Once (YOLO), long short term memory (LSTM), and residual neural network (ResNet) models. The relevant data is extracted from e-commerce-related websites using these deep learning models. For object detection and identification the YOLO is adapted, the successive sequence modeling and analysis are done by using LSTM. Feature extraction from the images is performed through ResNet. The proposed solution intends to provide more reliable and effective data that is to be extracted from e-commerce websites by including these models, making it simpler to collect pertinent data for a number of e-commerce-related applications.

Teng, Day & Chiu (2022) introduce the Chinese financial information extraction system (CFIES) designed for constructing a financial knowledge graph (FinKG). This system is employed to recognize, dissect, forecast, or assess a company’s prospective financial worth using financial documents. The pertinent information is extracted from these financial documents and subsequently incorporated into the financial knowledge graph, which helps in finding the financial information and making better financial decisions.

An open-source framework called Knowledge Graph Exploration and Visualization (KGEV), leverages the Human Phenotype Ontology (HPO) to connect genes with clinical phenotypes and disease names (Peng et al., 2022). Utilizing neo4j as the graph database, the framework enables interactive exploration of knowledge graphs (KG), allowing users to search for specific entities and relationships across multiple data sources. The framework was applied to COVID-19 as a use case, demonstrating how the integration of data from diverse sources reveals various biomedical relationships, including genes, proteins, phenotypes, drugs, and diseases. The developed web application facilitates answering simple questions related to COVID-19.

One of the recent works in this area is presented by Chen et al. (2023) that introduces the article related to the creation of a knowledge graph known as the critical infrastructures protection knowledge graph (CIPKG). Its primary aim is to enhance defense mechanisms against emerging threats and bridge existing information gaps. This endeavor involves the development of two distinct ontologies: one centered around attack patterns, vulnerabilities, and threats for offensive purposes, and the other geared towards defense strategies. To ensure precise and effective information extraction, a combination of the BiLSTM and CRF model, in conjunction with the BERT model, is employed.

A framework called ANTON was presented in Hosseinkhani, Taherdoost & Keikhaee (2019), which leverages crime ontology to analyze cybercrime cases and assist in crime investigations using the ANT-based focused crawling technique. ANTON utilizes support vector machines (SVMs) and naive Bayesian AI methods for crime classification. The system consists of two key components: the crime ontology builder and the focused crawler. To address the match-making challenges in ant-based analysis, the ANTON methodology incorporates ontology-based analysis.

Knowledge-Enhanced Graph Inference (KEGI) is one of the recent works in Han & Wang (2024), that aimed at extracting knowledge from extensive text paragraphs. The solution introduces a framework for constructing knowledge graphs tailored to the industrial domain. This framework relies on knowledge-enhanced document-level entity and relation extraction, leveraging ontology and a bidirectional long short-term memory conditional random field (BiLSTM-CRF) framework. We utilize the SPFRDoc and SPOnto datasets to validate our approach. This solution exhibits promise for addressing fault diagnosis and facilitating intelligent question answering in real-world production scenarios.

ONTBOT, an ontology-based chatbot is presented by Vegesna, Jain & Porwal (2018), which is created exclusively for the e-commerce industry. The Python programming language is used to create the chatbot, which uses ontology-based methods for information extraction. The chatbot can recognize and reply to customer inquiries about e-commerce by utilizing ontology and extracting pertinent data to give precise and beneficial answers. Through natural language communication with the chatbot, ONTBOT aims to improve the user experience while seeking information and support in the e-commerce space.

The recent trends toward Question-Answering systems for the financial domain have seen the usage of the Large Language Model (LLM). Examples include Islam et al. (2023), Zhao et al. (2024), Srivastava, Malik & Ganu (2024) and Sarmah et al. (2023). However, initial results show that LLMs are still not very successful in capturing factual information and various semantic relations. Hence the results are not expected to be of high accuracy as shown by Sarmah et al. (2023) where, the semantic similarity, even after augmenting the LLMs with retrieval component, was not achieved above 60%.

Discussion

The aforementioned studies provide an overview of researchers’ efforts in proposing and constructing web crawlers for information gathering and extraction purposes. In general, ontology-based information extraction systems are predominantly developed to achieve domain-specific information extraction, with the aim of enhancing relevance. These domains encompass a wide range of areas such as education/e-learning, cybercrime, construction, garments websites, healthcare, and tourism. The listings are placed in Table 1, that shows a gap in the domain of finance as it does not gain much attention. Moreover, very few works have provided an initial discussion on financial knowledge graphs. As per the available literature, this work is the first attempt towards automated information extraction from an annual financial report using ontology and modeling the facts as graphs using knowledge graphs.

This work has the following contributions:

1. Financial ontology-based information extraction

2. Modeling relevant information as a financial knowledge graph (FKG).

3. Analyzing financial institution’s performance in an automated way.

An ontological representation has been employed to model the domain of Finance to extract relevant semantic information. This leads to the achievement of high recall and precision that will be discussed in the result section.

The financial knowledge graph will help in answering the semantic query from the annual financial reports of the banks. The process is discussed in the next section.

A financial knowledge graph is modeled for each bank. The integrated view of different banks will help in analyzing the banks’s performance through SPARQL-based graph queries.

Annual financial report retrieval

A Python-based focused web crawler is developed, which is dedicated to a particular target as specified in the query, used for retrieving and downloading the annual financial reports as per the defined target. Unlike conventional crawlers, focused crawlers do not crawl the whole web. Focused crawlers are primarily responsible for crawling only the areas of the web that are relevant to the specific topic (Khan & Sharma, 2016).

The focused crawler is developed using Python libraries including Requests, BeautifulSoup, and PdfFileReader. Python’s ‘re’ module has been used for regular expressions to retrieve the focused information from web pages. The system flow is as under: (i) Crawler initiates a request to the specified URL. (ii) The request ‘gets’ the HTML content. (iii) The crawler parses the content using the BeautifulSoup library. (iv) The parsed contents are evaluated using ‘re’ module for regular expressions to locate annual financial reports on the page. (v) The matched links are then extracted and added to a list.

The determination of the focus is one of the challenges as compared to the traditional web crawlers.This has been achieved by selectively retrieving only the links to annual financial reports available on the provided web page. This selective approach ensures that only relevant PDF files, typically containing annual financial reports, are captured, while disregarding unnecessary files and links. This efficiency saves time by eliminating the need to sift through irrelevant material, thereby streamlining the data retrieval process for further analysis and information extraction purposes.

Extending ontology and designing competency questions

The ontology in Zehra et al. (2021) is extended to model the domain of annual financial reports. The data pertaining to the vocabularies, entities, and terminologies associated with annual financial reports across diverse commercial banks is gathered. This collection aids in the formulation of properties and relationships among these elements. Through analysis and incorporation of these terminologies, classes and their relationships are established. Subsequently, these classes are organized into a taxonomic hierarchy, following the methodology outlined in Noy & McGuinness (2001). The model has been tested on various commercial banks including (MCB (mcb.com.pk), ABL (abl.com), HBL (hbl.com), UBL (ubldigital.com), ASKARI (askaribank.com), BAFL (bankalfalah.com), BAHL (bankalhabib.com), BISLAMI (bankislami.com.pk), FBL (faysalbank.com), MBL (meezanbank.com), NBP (nbp.com.pk) and SCB (sc.com/pk)).

The ontology is validated through the W3C ontology validator and the validation results are available online (Mohsin, 2024). The ontologist needs to determine the purpose of ontology development (Noy & McGuinness, 2001). For this purpose, the competency questions need to be determined by reviewing the various annual financial reports of the banks and their usage by the stakeholders. These questions are designed by keeping in mind the queries of the layman and investors related to the investment opportunities and the financial benefits offered by the different banks and financial institutions in their annual financial statements, also the design process of the competency questions involves a thorough examination and analysis of key factors associated with annual financial statements, drawing insights from information accessible on diverse financial websites (Investopedia, 2021).

The competency questions are mentioned below:

I. Competency questions

Q1. Does the bank offer an Internet banking facility?

Q2. Does the bank offer a mobile banking facility?

Q3. Which banks offer Internet banking facilities?

Q4. Which banks offer mobile banking?

Q5. Which banks announce dividend in the year 2020?

Q6. What is the cash dividend per share of the banks in 2020?

Q7. What are the earnings per share after tax of a particular bank in the year 2020?

Q8. What is the interest income earned by the bank in 2020 in billions?

Q9. What is the gross markup income of the bank in the year 2020 in billions?

Q10. What is the total number of accounts of different banks in the year 2020?

Q11. What is the total number of branches of different banks in the year 2020?

Q12. What is the total number of employees of different banks in the year 2020?

Q13. What is the total number of ATMs of different banks in the year 2020?

Q14. What is the total number of Internet banking customers of the bank in the year 2020?

Q15. What is the total number of mobile banking customers of different banks in the year 2020?

Q16. What is the total number of credit card customers of different banks in the year 2020?

Q17. What is the total number of Internet banking transactions of the bank in the year 2020?

Q18. What is the total number of mobile banking transactions of the bank in the year 2020?

Q19. What is the total volume of Internet banking transactions of the bank in the year 2020 in millions?

Q20. What is the total volume of mobile banking transactions of the bank in the year 2020 in millions?

The competency questions help in determining the domain. This leads to the generation of classes/subclasses, relationships and instances. Figures 4–6 show the ontology developed for information extraction. The figures show different classes, instances and their relationships. This figure models the facts as rules in the domain of financial reports using ontological representation. These facts help in extracting relevant details from the reports.

The following subsection details the information extraction from the financial reports. The system has been implemented on the physically available financial reports based on their complexities.

Due to the lack of common semantics, it has been found that each bank is using different terms for the same concept. The ontological representation helps in resolving the semantic differences using ‘same as’ relationships or equivalent classes. Examples of such are shown in Table 2.

Figure 7 shows the view of the ontology using the Protégé (protege.stanford.edu) panel. It shows different classes and their subclasses to supervise the process of information extraction. The ontology guides the information extraction phase in extracting relevant data. The OWL code and the view of the complete ontology are available online (Supplemental files).

Information extraction

The information extraction phase provides automation in the retrieval of annual financial reports from the online repository and extracting relevant data from the financial reports. A web crawler is developed in Python to search the annual reports from the repository which is processed with a Python programming script to retrieve the financial statements of the bank available in unstructured data format.

In the second task, the retrieved annual report (MCB Bank Limited, 2022) in Fig. 8 is transformed into a tabular format in an automated way. The data of the annual financial report 2020 of the bank (MCB Bank Limited, 2022) is converted into the W3C compliant CSV format file. The CSV file contains many relevant and non-relevant items. The relevant terminologies based on rules are retrieved using the ontology developed (Figs. 4–6). The relevant data is extracted from the annual financial report 2020 of the bank with the proposed ontology.

The different steps involved in the entire process are as under:

Step 1: The retrieval of the annual financial report from the online repository.

Step 2: Conversion of textual data into W3C compliant tabular data format

Step 3: Extraction of relevant data from the tabular data

Step 4: Modeling relevant data into the knowledge base using a financial knowledge graph

Step 5: Visualization for querying

Figure 9 shows the importing and querying view of the knowledge graph (using Neo4J software), of the bank for the year 2020. The answers retrieved to the queries are shown in the result section.

Financial knowledge graph

In this research, the knowledge graph is used as a data repository to store and query the results related to the different queries of the investors regarding the financial stability and market position of the bank, which includes the profit/income of the bank, the different latest banking facilities provided by the bank including online and mobile banking. The volume of online transactions can be used as a strong indicator, which shows the customer’s satisfaction and their confidence and trust regarding the online services of the particular bank. The knowledge graph serves as a structured representation of the bank’s data, enabling efficient storage, retrieval, and analysis of information.

It allows investors to gain insights into the bank’s financial performance, assess its competitive position in the market, and make informed decisions based on the available data. The information extracted from the annual financial reports of the financial institutes is modeled in the knowledge graph for querying and answering the competency questions. For storing the extracted data this article utilizes Neo4J software as a graph database, used for importing the extracted data, which is in the CSV data format, into a knowledge graph, the extracted data contains the terminologies and values of the annual financial report. The detailed knowledge graph is shown in Fig. 10.

The developed knowledge graph helps in answering intelligent queries beneficial in solving use cases presented in the initial subsection. It can provide relevant and reliable information to various information retrieval systems including question answering, recommender, and search engine systems. The next subsection details the performance of the knowledge graph developed by showing the results of implementation.