Knowledge graphs (KGs) are structured representations of diversified knowledge. They are widely used in various intelligent applications. In this article, we provide a comprehensive survey on the evolution of various types of knowledge graphs (i.e., static KGs, dynamic KGs, temporal KGs, and event KGs) and techniques for knowledge extraction and reasoning. Furthermore, we introduce the practical applications of different types of KGs, including a case study in financial analysis. Finally, we propose our perspective on the future directions of knowledge engineering, including the potential of combining the power of knowledge graphs and large language models (LLMs), and the evolution of knowledge extraction, reasoning, and representation.
Large language models (LLMs) have demonstrated great potential in natural language processing tasks within the financial domain. In this work, we present a Chinese Financial Generative Pre-trained Transformer framework, named CFGPT, which includes a dataset~(CFData) for pre-training and supervised fine-tuning, a financial LLM~(CFLLM) to adeptly manage financial texts, and a deployment framework~(CFAPP) designed to navigate real-world financial applications. The CFData comprising both a pre-training dataset and a supervised fine-tuning dataset, where the pre-training dataset collates Chinese financial data and analytics, alongside a smaller subset of general-purpose text with 584M documents and 141B tokens in total, and the supervised fine-tuning dataset is tailored for six distinct financial tasks, embodying various facets of financial analysis and decision-making with 1.5M instruction pairs and 1.5B tokens in total. The CFLLM, which is based on InternLM-7B to balance the model capability and size, is trained on CFData in two stage, continued pre-training and supervised fine-tuning. The CFAPP is centered on large language models (LLMs) and augmented with additional modules to ensure multifaceted functionality in real-world application. Our codes are released at https://github.com/TongjiFinLab/CFGPT.
Financial analysis is an important tool for evaluating company performance. Practitioners work to answer financial questions to make profitable investment decisions, and use advanced quantitative analyses to do so. As a result, Financial Question Answering (QA) is a question answering task that requires deep reasoning about numbers. Furthermore, it is unknown how well pre-trained language models can reason in the financial domain. The current state-of-the-art requires a retriever to collect relevant facts about the financial question from the text and a generator to produce a valid financial program and a final answer. However, recently large language models like GPT-3 have achieved state-of-the-art performance on wide variety of tasks with just a few shot examples. We run several experiments with GPT-3 and find that a separate retrieval model and logic engine continue to be essential components to achieving SOTA performance in this task, particularly due to the precise nature of financial questions and the complex information stored in financial documents. With this understanding, our refined prompt-engineering approach on GPT-3 achieves near SOTA accuracy without any fine-tuning.
In this article, we present a novel approach for the construction of an environment-friendly green portfolio using the ESG ratings, and application of the modern portfolio theory to present what we call as the ``green efficient frontier'' (wherein the environmental score is included as a third dimension to the traditional mean-variance framework). Based on the prevailing action levels and policies, as well as additional market information, scenario analyses and stress testing are conducted to anticipate the future performance of the green portfolio in varying circumstances. The performance of the green portfolio is evaluated against the market returns in order to highlight the importance of sustainable investing and recognizing climate risk as a significant risk factor in financial analysis.
Designing dynamic portfolio insurance strategies under market conditions switching between two or more regimes is a challenging task in financial economics. Recently, a promising approach employing the value-at-risk (VaR) measure to assign weights to risky and riskless assets has been proposed in [Jiang C., Ma Y. and An Y. "The effectiveness of the VaR-based portfolio insurance strategy: An empirical analysis" , International Review of Financial Analysis 18(4) (2009): 185-197]. In their study, the risky asset follows a geometric Brownian motion with constant drift and diffusion coefficients. In this paper, we first extend their idea to a regime-switching framework in which the expected return of the risky asset and its volatility depend on an unobservable Markovian term which describes the cyclical nature of asset returns in modern financial markets. We then analyze and compare the resulting VaR-based portfolio insurance (VBPI) strategy with the well-known constant proportion portfolio insurance (CPPI) strategy. In this respect, we employ a variety of performance evaluation criteria such as Sharpe, Omega and Kappa ratios to compare the two methods. Our results indicate that the CPPI strategy has a better risk-return tradeoff in most of the scenarios analyzed and maintains a relatively stable return profile for the resulting portfolio at the maturity.
Forecasting financial time series (FTS) is an essential field in finance and economics that anticipates market movements in financial markets. This paper investigates the accuracy of text mining and technical analyses in forecasting financial time series. It focuses on the S&P500 stock market index during the pandemic, which tracks the performance of the largest publicly traded companies in the US. The study compares two methods of forecasting the future price of the S&P500: text mining, which uses NLP techniques to extract meaningful insights from financial news, and technical analysis, which uses historical price and volume data to make predictions. The study examines the advantages and limitations of both methods and analyze their performance in predicting the S&P500. The FinBERT model outperforms other models in terms of S&P500 price prediction, as evidenced by its lower RMSE value, and has the potential to revolutionize financial analysis and prediction using financial news data. Keywords: ARIMA, BERT, FinBERT, Forecasting Financial Time Series, GARCH, LSTM, Technical Analysis, Text Mining JEL classifications: G4, C8
Form 10-K report is a financial report disclosing the annual financial state of a public company. It is an important evidence to conduct financial analysis, i.e., asset pricing, corporate finance. Practitioners and researchers are constantly designing algorithms to better conduct analysis on information in the Form 10-K report. The vast majority of previous works focus on quantitative data. With recent advancement on natural language processing (NLP), textual data in financial filing attracts more attention. However, to incorporate textual data for analyzing, Form 10-K Itemization is a necessary pre-process step. It aims to segment the whole document into several Item sections, where each Item section focuses on a specific financial aspect of the company. With the segmented Item sections, NLP techniques can directly apply on those Item sections related to downstream tasks. In this paper, we develop a Form 10-K Itemization system which can automatically segment all the Item sections in 10-K documents. The system is both effective and efficient. It reaches a retrieval rate of 93%.
This paper presents a novel approach for explainability in financial analysis by deriving financially-explainable statistical relationships through aspect-based sentiment analysis, Pearson correlation, Granger causality & uncertainty coefficient. The proposed methodology involves constructing an aspect list from financial literature and applying aspect-based sentiment analysis on social media text to compute sentiment scores for each aspect. Pearson correlation is then applied to uncover financially explainable relationships between aspect sentiment scores and stock prices. Findings for derived relationships are made robust by applying Granger causality to determine the forecasting ability of each aspect sentiment score for stock prices. Finally, an added layer of interpretability is added by evaluating uncertainty coefficient scores between aspect sentiment scores and stock prices. This allows us to determine the aspects whose sentiment scores are most statistically significant for stock prices. Relative to other methods, our approach provides a more informative and accurate understanding of the relationship between sentiment analysis and stock prices. Specifically, this methodology enables an interpretation of the statistical relationship between aspect-based sentiment scores and stock prices, which offers explainability to AI-driven financial decision-making.
Long-form numerical reasoning in financial analysis aims to generate a reasoning program to calculate the correct answer for a given question. Previous work followed a retriever-generator framework, where the retriever selects key facts from a long-form document, and the generator generates a reasoning program based on retrieved facts. However, they treated all facts equally without considering the different contributions of facts with and without numbers. Meanwhile, the program consistency were ignored under supervised training, resulting in lower training accuracy and diversity. To solve these problems, we proposed APOLLO to improve the long-form numerical reasoning framework. For the retriever, we adopt a number-aware negative sampling strategy to enable the retriever to be more discriminative on key numerical facts. For the generator, we design consistency-based reinforcement learning and target program augmentation strategy based on the consistency of program execution results. Experimental results on the FinQA and ConvFinQA leaderboard verify the effectiveness of our proposed method, achieving the new state-of-the-art.
Rising usage of deep neural networks to perform decision making in critical applications like medical diagnosis and financial analysis have raised concerns regarding their reliability and trustworthiness. As automated systems become more mainstream, it is important their decisions be transparent, reliable and understandable by humans for better trust and confidence. To this effect, concept-based models such as Concept Bottleneck Models (CBMs) and Self-Explaining Neural Networks (SENN) have been proposed which constrain the latent space of a model to represent high level concepts easily understood by domain experts in the field. Although concept-based models promise a good approach to both increasing explainability and reliability, it is yet to be shown if they demonstrate robustness and output consistent concepts under systematic perturbations to their inputs. To better understand performance of concept-based models on curated malicious samples, in this paper, we aim to study their robustness to adversarial perturbations, which are also known as the imperceptible changes to the input data that are crafted by an attacker to fool a well-learned concept-based model. Specifically, we first propose and analyze different malicious attacks to evaluate the security vulnerability of concept based models. Subsequently, we propose a potential general adversarial training-based defense mechanism to increase robustness of these systems to the proposed malicious attacks. Extensive experiments on one synthetic and two real-world datasets demonstrate the effectiveness of the proposed attacks and the defense approach.
Money laundering has become one of the most relevant criminal activities in modern societies, as it causes massive financial losses for governments, banks and other institutions. Detecting such activities is among the top priorities when it comes to financial analysis, but current approaches are often costly and labor intensive partly due to the sheer amount of data to be analyzed. Hence, there is a growing need for automatic anti-money laundering systems to assist experts. In this work, we propose DELATOR, a novel framework for detecting money laundering activities based on graph neural networks that learn from large-scale temporal graphs. DELATOR provides an effective and efficient method for learning from heavily imbalanced graph data, by adapting concepts from the GraphSMOTE framework and incorporating elements of multi-task learning to obtain rich node embeddings for node classification. DELATOR outperforms all considered baselines, including an off-the-shelf solution from Amazon AWS by 23% with respect to AUC-ROC. We also conducted real experiments that led to the discovery of 7 new suspicious cases among the 50 analyzed ones, which have been reported to the authorities.
Graph Neural Networks (GNNs) have made rapid developments in the recent years. Due to their great ability in modeling graph-structured data, GNNs are vastly used in various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite their great potential in benefiting humans in the real world, recent study shows that GNNs can leak private information, are vulnerable to adversarial attacks, can inherit and magnify societal bias from training data and lack interpretability, which have risk of causing unintentional harm to the users and society. For example, existing works demonstrate that attackers can fool the GNNs to give the outcome they desire with unnoticeable perturbation on training graph. GNNs trained on social networks may embed the discrimination in their decision process, strengthening the undesirable societal bias. Consequently, trustworthy GNNs in various aspects are emerging to prevent the harm from GNN models and increase the users' trust in GNNs. In this paper, we give a comprehensive survey of GNNs in the computational aspects of privacy, robustness, fairness, and explainability. For each aspect, we give the taxonomy of the related methods and formulate the general frameworks for the multiple categories of trustworthy GNNs. We also discuss the future research directions of each aspect and connections between these aspects to help achieve trustworthiness.
We study general properties such as the solution representation of a moving boundary value problem of the Black-Scholes equation, its min-max estimation, lower and upper gradient estimates, and strict monotonicity with respect to the spatial variables of the solution. These results are used in the study of a structural model of pricing puttable bond with credit risk. We first prove the solution representation of a special fixed boundary value problem of the Black-Scholes equation, the min-max estimate, the lower and upper gradient estimates, and the strict monotonicity with respect to the spatial variables of the solution. Then, these results are applied to give the solution representation of a moving boundary value problem of the Black-Scholes equation with moving boundary in the form of an exponential function, the min-max estimate, the lower and upper gradient estimates, and the strict monotonicity results on the spatial variables of the solution. Finally, we illustrate how these results can be used in the derivation of analytical pricing formulae and financial analysis of price functions of puttable bonds with credit risk (corporate bonds with one early redemption date). Our results can be used for the derivation and analysis of the analytical pricing formulae of the one-factor structural model of a more general puttable bonds with credit risk (corporate bond with several early redemption dates).
Alignment of financial market incentives and carbon emissions disincentives is key to limiting global warming. Regulators and standards bodies have made a start by requiring some carbon-related disclosures and proposing others. Here we go further and propose a Carbon Equivalence Principle: all financial products shall contain a description of the equivalent carbon flows from greenhouse gases that the products enable, as well as their existing description in terms of cash flows. This description of the carbon flows enabled by the project shall be compatible with existing bank systems that track cashflows so that carbon flows have equal standing to cash flows. We demonstrate that this transparency alone can align incentives by applying it to project finance examples for power generation and by following through the financial analysis. The financial requirements to offset costs of carbon flows enabled in the future radically change project costs, and risk that assets become stranded, thus further increasing costs. This observation holds whichever partner in the project bears the enabled-carbon costs. Mitigating these risks requires project re-structuring to include negative emissions technologies. We also consider that sequestered carbon needs to remain sequestered permanently, e.g., for at least one hundred years. We introduce mixed financial-physical solutions to minimise this permanence cost, and price to them. This complements previous insurance-based proposals with lesser scope. For financial viability we introduce project designs that are financially net-zero, and as a consequence are carbon negative. Thus we see that adoption of the Carbon Equivalence Principle for financial products aligns incentives, requires product redesign, and is simply good financial management driving sustainability.
Machine learning (ML) methods are used in most technical areas such as image recognition, product recommendation, financial analysis, medical diagnosis, and predictive maintenance. An important aspect of implementing ML methods involves controlling the learning process for the ML method so as to maximize the performance of the method under consideration. Hyperparameter tuning is the process of selecting a suitable set of ML method parameters that control its learning process. In this work, we demonstrate the use of discrete simulation optimization methods such as ranking and selection (R&S) and random search for identifying a hyperparameter set that maximizes the performance of a ML method. Specifically, we use the KN R&S method and the stochastic ruler random search method and one of its variations for this purpose. We also construct the theoretical basis for applying the KN method, which determines the optimal solution with a statistical guarantee via solution space enumeration. In comparison, the stochastic ruler method asymptotically converges to global optima and incurs smaller computational overheads. We demonstrate the application of these methods to a wide variety of machine learning models, including deep neural network models used for time series prediction and image classification. We benchmark our application of these methods with state-of-the-art hyperparameter optimization libraries such as $hyperopt$ and $mango$. The KN method consistently outperforms $hyperopt$'s random search (RS) and Tree of Parzen Estimators (TPE) methods. The stochastic ruler method outperforms the $hyperopt$ RS method and offers statistically comparable performance with respect to $hyperopt$'s TPE method and the $mango$ algorithm.
Financial time-series forecasting is one of the most challenging domains in the field of time-series analysis. This is mostly due to the highly non-stationary and noisy nature of financial time-series data. With progressive efforts of the community to design specialized neural networks incorporating prior domain knowledge, many financial analysis and forecasting problems have been successfully tackled. The temporal attention mechanism is a neural layer design that recently gained popularity due to its ability to focus on important temporal events. In this paper, we propose a neural layer based on the ideas of temporal attention and multi-head attention to extend the capability of the underlying neural network in focusing simultaneously on multiple temporal instances. The effectiveness of our approach is validated using large-scale limit-order book market data to forecast the direction of mid-price movements. Our experiments show that the use of multi-head temporal attention modules leads to enhanced prediction performances compared to baseline models.
Causality detection draws plenty of attention in the field of Natural Language Processing and linguistics research. It has essential applications in information retrieval, event prediction, question answering, financial analysis, and market research. In this study, we explore several methods to identify and extract cause-effect pairs in financial documents using transformers. For this purpose, we propose an approach that combines POS tagging with the BIO scheme, which can be integrated with modern transformer models to address this challenge of identifying causality in a given text. Our best methodology achieves an F1-Score of 0.9551, and an Exact Match Score of 0.8777 on the blind test in the FinCausal-2021 Shared Task at the FinCausal 2021 Workshop.
This paper proposes a design and simulation based investigative analysis of a vertically mounted bifacial solar photovoltaic model in Marine Drive Road, Cox's Bazar. Cox's bazar is a famous tourist destination which seems to be a flexible site for implementing such energy harvesting system without affecting the nearby eco-system and solves the existing land shortage problem. Moreover, the infrastructure will provide insulation to noise related problem faced by nearby residents, arising from traffic noises. A model road of 200 meters is reconnoitered for energy harvesting by solar power using three prominent software namely PVSOL, PVsyst, and SAM where a promising mean annual yield of 70492.9 kWh is obtained, and the bifacial gain is calculated to be 12.26%. In addition, a deviation analysis is performed among each of the software and it is found that PVSOL and PVsyst have shown less deviation. Furthermore, a comprehensive financial analysis shows total installation cost to be 84759.74$.
Deep neural networks have demonstrated remarkable performance in many data-driven and prediction-oriented applications, and sometimes even perform better than humans. However, their most significant drawback is the lack of interpretability, which makes them less attractive in many real-world applications. When relating to the moral problem or the environmental factors that are uncertain such as crime judgment, financial analysis, and medical diagnosis, it is essential to mine the evidence for the model's prediction (interpret model knowledge) to convince humans. Thus, investigating how to interpret model knowledge is of paramount importance for both academic research and real applications.
Time series has wide applications in the real world and is known to be difficult to forecast. Since its statistical properties change over time, its distribution also changes temporally, which will cause severe distribution shift problem to existing methods. However, it remains unexplored to model the time series in the distribution perspective. In this paper, we term this as Temporal Covariate Shift (TCS). This paper proposes Adaptive RNNs (AdaRNN) to tackle the TCS problem by building an adaptive model that generalizes well on the unseen test data. AdaRNN is sequentially composed of two novel algorithms. First, we propose Temporal Distribution Characterization to better characterize the distribution information in the TS. Second, we propose Temporal Distribution Matching to reduce the distribution mismatch in TS to learn the adaptive TS model. AdaRNN is a general framework with flexible distribution distances integrated. Experiments on human activity recognition, air quality prediction, and financial analysis show that AdaRNN outperforms the latest methods by a classification accuracy of 2.6% and significantly reduces the RMSE by 9.0%. We also show that the temporal distribution matching algorithm can be extended in Transformer structure to boost its performance.