Portfolio optimization is a fundamental challenge in quantitative finance, requiring robust computational tools that integrate statistical rigor with practical implementation. We present skfolio, an open-source Python library for portfolio construction and risk management that seamlessly integrates with the scikit-learn ecosystem. skfolio provides a unified framework for diverse allocation strategies, from classical mean-variance optimization to modern clustering-based methods, state-of-the-art financial estimators with native interfaces, and advanced cross-validation techniques tailored for financial time series. By adhering to scikit-learn's fit-predict-transform paradigm, the library enables researchers and practitioners to leverage machine learning workflows for portfolio optimization, promoting reproducibility and transparency in quantitative finance.
Financial markets are noisy yet contain a latent graph-theoretic structure that can be exploited for superior risk-adjusted returns. We propose a quantum stochastic walk (QSW) optimizer that embeds assets in a weighted graph: nodes represent securities while edges encode the return-covariance kernel. Portfolio weights are derived from the walk's stationary distribution. Three empirical studies support the approach. (i) For the top 100 S\&P 500 constituents over 2016-2024, six scenario portfolios calibrated on 1- and 2-year windows lift the out-of-sample Sharpe ratio by up to 27\% while cutting annual turnover from 480\% (mean-variance) to 2-90%. (ii) A $5^{4}=625$-point grid search identifies a robust sweet spot, $\alpha,\lambda\lesssim0.5$ and $\omega\in[0.2,0.4]$, that delivers Sharpe $\approx0.97$ at $\le 5\%$ turnover and Herfindahl-Hirschman index $\sim0.01$. (iii) Repeating the full grid on 50 random 100-stock subsets of the S\&P 500 adds 31\,350 back-tests: the best-per-draw QSW beats re-optimised mean-variance on Sharpe in 54\% of cases and always wins on trading efficiency, with median turnover 36\% versus 351\%. Overall, QSW raises the annualized Sharpe ratio by 15\% and cuts turnover by 90\% relative to classical optimisation, all while respecting the UCITS 5/10/40 rule. These results show that hybrid quantum-classical dynamics can uncover non-linear dependencies overlooked by quadratic models and offer a practical, low-cost weighting engine for themed ETFs and other systematic mandates.
The Diversification Quotient (DQ), introduced by Han et al. (2025), is a recently proposed measure of portfolio diversification that quantifies the reduction in a portfolio's risk-level parameter attributable to diversification. Grounded in a rigorous theoretical framework, DQ effectively captures heavy tails, common shocks, and enhances efficiency in portfolio optimization. This paper further explores the convergence properties and asymptotic normality of empirical DQ estimators based on Value at Risk (VaR) and Expected Shortfall (ES), with explicit calculation of the asymptotic variance. In contrast to the diversification ratio (DR) proposed by Tasche (2007), which may exhibit diverging asymptotic variance due to its lack of location invariance, the DQ estimators demonstrate greater robustness under various distributional settings. We further evaluate their performance under elliptical distributions and conduct a simulation study to examine their finite-sample behavior. The results offer a solid statistical foundation for the application of DQ in financial risk management and decision-making.
We investigate multi-period mean-risk portfolio optimization for long-horizon Defined Contribution plans, focusing on buffered Probability of Exceedance (bPoE), a more intuitive, dollar-based alternative to Conditional Value-at-Risk (CVaR). We formulate both pre-commitment and time-consistent Mean-bPoE and Mean-CVaR portfolio optimization problems under realistic investment constraints (e.g., no leverage, no short selling) and jump-diffusion dynamics. These formulations are naturally framed as bilevel optimization problems, with an outer search over the shortfall threshold and an inner optimization over rebalancing decisions. We establish an equivalence between the pre-commitment formulations through a one-to-one correspondence of their scalarization optimal sets, while showing that no such equivalence holds in the time-consistent setting. We develop provably convergent numerical schemes for the value functions associated with both pre-commitment and time-consistent formulations of these mean-risk control problems. Using nearly a century of market data, we find that time-consistent Mean-bPoE strategies closely resemble their pre-commitment counterparts. In particular, they maintain alignment with investors' preferences for a minimum acceptable terminal wealth level-unlike time-consistent Mean-CVaR, which often leads to counterintuitive control behavior. We further show that bPoE, as a strictly tail-oriented measure, prioritizes guarding against catastrophic shortfalls while allowing meaningful upside exposure, making it especially appealing for long-horizon wealth security. These findings highlight bPoE's practical advantages for Defined Contribution investment planning.
This paper develops and estimates a multivariate affine GARCH(1,1) model with Normal Inverse Gaussian innovations that captures time-varying volatility, heavy tails, and dynamic correlation across asset returns. We generalize the Heston-Nandi framework to a multivariate setting and apply it to 30 Dow Jones Industrial Average stocks. The model jointly supports three core financial applications: dynamic portfolio optimization, wealth path simulation, and option pricing. Closed-form solutions are derived for a Constant Relative Risk Aversion (CRRA) investor's intertemporal asset allocation, and we implement a forward-looking risk-adjusted performance comparison against Merton-style constant strategies. Using the model's conditional volatilities, we also construct implied volatility surfaces for European options, capturing skew and smile features. Empirically, we document substantial wealth-equivalent utility losses from ignoring time-varying correlation and tail risk. These findings underscore the value of a unified econometric framework for analyzing joint asset dynamics and for managing portfolio and derivative exposures under non-Gaussian risks.
Reinforcement learning (RL) has shown significant promise for sequential portfolio optimization tasks, such as stock trading, where the objective is to maximize cumulative returns while minimizing risks using historical data. However, traditional RL approaches often produce policies that merely memorize the optimal yet impractical buying and selling behaviors within the fixed dataset. These offline policies are less generalizable as they fail to account for the non-stationary nature of the market. Our approach, MetaTrader, frames portfolio optimization as a new type of partial-offline RL problem and makes two technical contributions. First, MetaTrader employs a bilevel learning framework that explicitly trains the RL agent to improve both in-domain profits on the original dataset and out-of-domain performance across diverse transformations of the raw financial data. Second, our approach incorporates a new temporal difference (TD) method that approximates worst-case TD estimates from a batch of transformed TD targets, addressing the value overestimation issue that is particularly challenging in scenarios with limited offline data. Our empirical results on two public stock datasets show that MetaTrader outperforms existing methods, including both RL-based approaches and traditional stock prediction models.
Portfolio optimization involves selecting asset weights to minimize a risk-reward objective, such as the portfolio variance in the classical minimum-variance framework. Sparse portfolio selection extends this by imposing a cardinality constraint: only $k$ assets from a universe of $p$ may be included. The standard approach models this problem as a mixed-integer quadratic program and relies on commercial solvers to find the optimal solution. However, the computational costs of such methods increase exponentially with $k$ and $p$, making them too slow for problems of even moderate size. We propose a fast and scalable gradient-based approach that transforms the combinatorial sparse selection problem into a constrained continuous optimization task via Boolean relaxation, while preserving equivalence with the original problem on the set of binary points. Our algorithm employs a tunable parameter that transmutes the auxiliary objective from a convex to a concave function. This allows a stable convex starting point, followed by a controlled path toward a sparse binary solution as the tuning parameter increases and the objective moves toward concavity. In practice, our method matches commercial solvers in asset selection for most instances and, in rare instances, the solution differs by a few assets whilst showing a negligible error in portfolio variance.
In the era of big data, data mining has become indispensable for uncovering hidden patterns and insights from vast and complex datasets. The integration of multimodal data sources further enhances its potential. Multimodal Federated Learning (MFL) is a distributed approach that enhances the efficiency and quality of multimodal learning, ensuring collaborative work and privacy protection. However, missing modalities pose a significant challenge in MFL, often due to data quality issues or privacy policies across the clients. In this work, we present MMiC, a framework for Mitigating Modality incompleteness in MFL within the Clusters. MMiC replaces partial parameters within client models inside clusters to mitigate the impact of missing modalities. Furthermore, it leverages the Banzhaf Power Index to optimize client selection under these conditions. Finally, MMiC employs an innovative approach to dynamically control global aggregation by utilizing Markovitz Portfolio Optimization. Extensive experiments demonstrate that MMiC consistently outperforms existing federated learning architectures in both global and personalized performance on multimodal datasets with missing modalities, confirming the effectiveness of our proposed solution.
Portfolio optimization requires dynamic allocation of funds by balancing the risk and return tradeoff under dynamic market conditions. With the recent advancements in AI, Deep Reinforcement Learning (DRL) has gained prominence in providing adaptive and scalable strategies for portfolio optimization. However, the success of these strategies depends not only on their ability to adapt to market dynamics but also on the careful pre-selection of assets that influence overall portfolio performance. Incorporating the investor's preference in pre-selecting assets for a portfolio is essential in refining their investment strategies. This study proposes a volatility-guided DRL-based portfolio optimization framework that dynamically constructs portfolios based on investors' risk profiles. The Generalized Autoregressive Conditional Heteroscedasticity (GARCH) model is utilized for volatility forecasting of stocks and categorizes them based on their volatility as aggressive, moderate, and conservative. The DRL agent is then employed to learn an optimal investment policy by interacting with the historical market data. The efficacy of the proposed methodology is established using stocks from the Dow $30$ index. The proposed investor-specific DRL-based portfolios outperformed the baseline strategies by generating consistent risk-adjusted returns.
Optimization problems with norm-bounding constraints arise in a variety of applications, including portfolio optimization, machine learning, and feature selection. A common approach to these problems involves relaxing the norm constraint via Lagrangian relaxation, transforming it into a regularization term in the objective function. A particularly challenging class includes the zero-norm function, which promotes sparsity in statistical parameter estimation. Most existing exact methods for solving these problems introduce binary variables and artificial bounds to reformulate them as higher-dimensional mixed-integer programs, solvable by standard solvers. Other exact approaches exploit specific structural properties of the objective, making them difficult to generalize across different problem types. Alternative methods employ nonconvex penalties with favorable statistical characteristics, but these are typically addressed using heuristic or local optimization techniques due to their structural complexity. In this paper, we propose a novel graph-based method to globally solve optimization problems involving generalized norm-bounding constraints. Our approach encompasses standard $\ell_p$-norms for $p \in [0, \infty)$ and nonconvex penalties such as SCAD and MCP. We leverage decision diagrams to construct strong convex relaxations directly in the original variable space, eliminating the need for auxiliary variables or artificial bounds. Integrated into a spatial branch-and-cut framework, our method guarantees convergence to the global optimum. We demonstrate its effectiveness through preliminary computational experiments on benchmark sparse linear regression problems involving complex nonconvex penalties, which are not tractable using existing global optimization techniques.
Recent advances in deep learning have spurred the development of end-to-end frameworks for portfolio optimization that utilize implicit layers. However, many such implementations are highly sensitive to neural network initialization, undermining performance consistency. This research introduces a robust end-to-end framework tailored for risk budgeting portfolios that effectively reduces sensitivity to initialization. Importantly, this enhanced stability does not compromise portfolio performance, as our framework consistently outperforms the risk parity benchmark.
Portfolio optimization faces challenges due to the sensitivity in traditional mean-variance models. The Black-Litterman model mitigates this by integrating investor views, but defining these views remains difficult. This study explores the integration of large language models (LLMs) generated views into portfolio optimization using the Black-Litterman framework. Our method leverages LLMs to estimate expected stock returns from historical prices and company metadata, incorporating uncertainty through the variance in predictions. We conduct a backtest of the LLM-optimized portfolios from June 2024 to February 2025, rebalancing biweekly using the previous two weeks of price data. As baselines, we compare against the S&P 500, an equal-weighted portfolio, and a traditional mean-variance optimized portfolio constructed using the same set of stocks. Empirical results suggest that different LLMs exhibit varying levels of predictive optimism and confidence stability, which impact portfolio performance. The source code and data are available at https://github.com/youngandbin/LLM-MVO-BLM.
We revisit the Bayesian Black-Litterman (BL) portfolio model and remove its reliance on subjective investor views. Classical BL requires an investor "view": a forecast vector $q$ and its uncertainty matrix $\Omega$ that describe how much a chosen portfolio should outperform the market. Our key idea is to treat $(q,\Omega)$ as latent variables and learn them from market data within a single Bayesian network. Consequently, the resulting posterior estimation admits closed-form expression, enabling fast inference and stable portfolio weights. Building on these, we propose two mechanisms to capture how features interact with returns: shared-latent parametrization and feature-influenced views; both recover classical BL and Markowitz portfolios as special cases. Empirically, on 30-year Dow-Jones and 20-year sector-ETF data, we improve Sharpe ratios by 50% and cut turnover by 55% relative to Markowitz and the index baselines. This work turns BL into a fully data-driven, view-free, and coherent Bayesian framework for portfolio optimization.
Typical deep reinforcement learning (DRL) agents for dynamic portfolio optimization learn the factors influencing portfolio return and risk by analyzing the output values of the reward function while adjusting portfolio weights within the training environment. However, it faces a major limitation where it is difficult for investors to intervene in the training based on different levels of risk aversion towards each portfolio asset. This difficulty arises from another limitation: existing DRL agents may not develop a thorough understanding of the factors responsible for the portfolio return and risk by only learning from the output of the reward function. As a result, the strategy for determining the target portfolio weights is entirely dependent on the DRL agents themselves. To address these limitations, we propose a reward factor matrix for elucidating the return and risk of each asset in the portfolio. Additionally, we propose a novel learning system named Factor-MCLS using a multi-critic framework that facilitates learning of the reward factor matrix. In this way, our DRL-based learning system can effectively learn the factors influencing portfolio return and risk. Moreover, based on the critic networks within the multi-critic framework, we develop a risk constraint term in the training objective function of the policy function. This risk constraint term allows investors to intervene in the training of the DRL agent according to their individual levels of risk aversion towards the portfolio assets.
Algorithmic trading relies on extracting meaningful signals from diverse financial data sources, including candlestick charts, order statistics on put and canceled orders, traded volume data, limit order books, and news flow. While deep learning has demonstrated remarkable success in processing unstructured data and has significantly advanced natural language processing, its application to structured financial data remains an ongoing challenge. This study investigates the integration of deep learning models with financial data modalities, aiming to enhance predictive performance in trading strategies and portfolio optimization. We present a novel approach to incorporating limit order book analysis into algorithmic trading by developing embedding techniques and treating sequential limit order book snapshots as distinct input channels in an image-based representation. Our methodology for processing limit order book data achieves state-of-the-art performance in high-frequency trading algorithms, underscoring the effectiveness of deep learning in financial applications.
The mean-variance portfolio model, based on the risk-return trade-off for optimal asset allocation, remains foundational in portfolio optimization. However, its reliance on restrictive assumptions about asset return distributions limits its applicability to real-world data. Parametric copula structures provide a novel way to overcome these limitations by accounting for asymmetry, heavy tails, and time-varying dependencies. Existing methods have been shown to rely on fixed or static dependence structures, thus overlooking the dynamic nature of the financial market. In this study, a semiparametric model is proposed that combines non-parametrically estimated copulas with parametrically estimated marginals to allow all parameters to dynamically evolve over time. A novel framework was developed that integrates time-varying dependence modeling with flexible empirical beta copula structures. Marginal distributions were modeled using the Skewed Generalized T family. This effectively captures asymmetry and heavy tails and makes the model suitable for predictive inferences in real world scenarios. Furthermore, the model was applied to rolling windows of financial returns from the USA, India and Hong Kong economies to understand the influence of dynamic market conditions. The approach addresses the limitations of models that rely on parametric assumptions. By accounting for asymmetry, heavy tails, and cross-correlated asset prices, the proposed method offers a robust solution for optimizing diverse portfolios in an interconnected financial market. Through adaptive modeling, it allows for better management of risk and return across varying economic conditions, leading to more efficient asset allocation and improved portfolio performance.
Solving large-scale, continuous-time portfolio optimization problems involving numerous assets and state-dependent dynamics has long been challenged by the curse of dimensionality. Traditional dynamic programming and PDE-based methods, while rigorous, typically become computationally intractable beyond a few state variables ($\sim$3-6 limit in prior studies). To overcome this critical barrier, we introduce the \emph{Pontryagin-Guided Direct Policy Optimization} (PG-DPO) framework. PG-DPO leverages Pontryagin's Maximum Principle (PMP) and backpropagation-through-time (BPTT) to directly inform neural network-based policy learning. A key contribution is our highly efficient \emph{Projected PG-DPO (P-PGDPO)} variant. This approach uniquely utilizes BPTT to obtain rapidly stabilizing estimates of the Pontryagin costates and their crucial derivatives with respect to the state variables. These estimates are then analytically projected onto the manifold of optimal controls dictated by PMP's first-order conditions, significantly reducing training overhead and enhancing accuracy. This enables a breakthrough in scalability: numerical experiments demonstrate that P-PGDPO successfully tackles problems with dimensions previously considered far out of reach (up to 50 assets and 10 state variables). Critically, the framework accurately captures complex intertemporal hedging demands, a feat often elusive for other methods in high-dimensional settings. P-PGDPO delivers near-optimal policies, offering a practical and powerful alternative for a broad class of high-dimensional continuous-time control problems.
Financial scenario simulation is essential for risk management and portfolio optimization, yet it remains challenging especially in high-dimensional and small data settings common in finance. We propose a diffusion factor model that integrates latent factor structure into generative diffusion processes, bridging econometrics with modern generative AI to address the challenges of the curse of dimensionality and data scarcity in financial simulation. By exploiting the low-dimensional factor structure inherent in asset returns, we decompose the score function--a key component in diffusion models--using time-varying orthogonal projections, and this decomposition is incorporated into the design of neural network architectures. We derive rigorous statistical guarantees, establishing nonasymptotic error bounds for both score estimation at O(d^{5/2} n^{-2/(k+5)}) and generated distribution at O(d^{5/4} n^{-1/2(k+5)}), primarily driven by the intrinsic factor dimension k rather than the number of assets d, surpassing the dimension-dependent limits in the classical nonparametric statistics literature and making the framework viable for markets with thousands of assets. Numerical studies confirm superior performance in latent subspace recovery under small data regimes. Empirical analysis demonstrates the economic significance of our framework in constructing mean-variance optimal portfolios and factor portfolios. This work presents the first theoretical integration of factor structure with diffusion models, offering a principled approach for high-dimensional financial simulation with limited data. Our code is available at https://github.com/xymmmm00/diffusion_factor_model.
Fundamental and necessary principles for achieving efficient portfolio optimization based on asset and diversification dynamics are presented. The Commonality Principle is a necessary and sufficient condition for identifying optimal drivers of a portfolio in terms of its diversification dynamics. The proof relies on the Reichenbach Common Cause Principle, along with the fact that the sensitivities of portfolio constituents with respect to the common causal drivers are themselves causal. A conformal map preserves idiosyncratic diversification from the unconditional setting while optimizing systematic diversification on an embedded space of these sensitivities. Causal methodologies for combinatorial driver selection are presented, such as the use of Bayesian networks and correlation-based algorithms from Reichenbach's principle. Limitations of linear models in capturing causality are discussed, and included for completeness alongside more advanced models such as neural networks. Portfolio optimization methods are presented that map risk from the sensitivity space to other risk measures of interest. Finally, the work introduces a novel risk management framework based on Common Causal Manifolds, including both theoretical development and experimental validation. The sensitivity space is predicted along the common causal manifold, which is modeled as a causal time system. Sensitivities are forecasted using SDEs calibrated to data previously extracted from neural networks to move along the manifold via its tangent bundles. An optimization method is then proposed that accumulates information across future predicted tangent bundles on the common causal time system manifold. It aggregates sensitivity-based distance metrics along the trajectory to build a comprehensive sensitivity distance matrix. This matrix enables trajectory-wide optimal diversification, taking into account future dynamics.
OmniEcon Nexus is an open-source, high-performance simulation engine for global microeconomic and macroeconomic analysis. Built with advanced deep learning, agent-based modeling, and optimization techniques, it enables detailed forecasting, risk analysis, policy generation, and portfolio optimization. This system supports up to 5 million agents and is designed as a comprehensive tool for governments, researchers, and developers to explore economic dynamics. Core Features Economic Forecasting: Predicts short-term and mid-term economic trends using deep learning models. Agent-Based Simulation: Models up to 5M agents (citizens, businesses, governments) with behavioral psychology. Portfolio Optimization: Optimizes asset allocation using the Sharpe ratio and real-time market data. Policy Generation: Automatically generates and evaluates macroeconomic policies with Q-learning. Risk Analysis: Assesses market volatility and systemic risk using network analysis. Market Psychology: Estimates PMI and agent psychological states (Fear, Greed, Complacency, Hope). Technical Overview Deep Learning Components MicroEconomicPredictor: Architecture: GRU, LSTM, Transformer Encoder, and a custom QuantumResonanceLayer. Configuration: Default hidden_dim=8192, num_layers=24, input_dim=72. Purpose: Forecasts short-term (short_pred) and mid-term (mid_pred) economic growth. Implementation: See MicroEconomicPredictor.forward() for details. QuantumResonanceLayer: Mechanism: Combines linear transformation with sinusoidal phase shifts and layer normalization. Purpose: Enhances prediction accuracy with quantum-inspired dynamics. Agent-Based Modeling HyperAgent: Roles: Citizens, businesses, governments. Attributes: Wealth, innovation, trade flow, resilience, psychological state. Behavior: Updated via interact(), influenced by market data, global context, and policies. Scale: Supports 5M agents with multiprocessing (Pool). Optimization and Policy Portfolio Optimization: Method: Uses scipy.optimize.minimize with SLSQP to maximize Sharpe ratio. Inputs: Short-term/mid-term predictions, volatility, crowd sentiment. Constraints: Total weights = 1, stocks + gold ≤ 80%. See: optimize_portfolio(). Policy Generation: Algorithm: Q-learning with state hashing (generate_policy()). Inputs: PMI, fear/greed indices, market momentum, volatility. Outputs: Policies like tax reduction, interest rate hikes, subsidies. Evaluation: Assesses impact via evaluate_policy_impact() using resilience, cash flow, consumption metrics. Network Analysis Systemic Risk Network: Structure: Directed graph (networkx.DiGraph) tracking trade dependencies. Metric: Systemic Risk Score (SRS) via calculate_systemic_risk_score() with betweenness centrality. Reflexive Network: Storage: Policy history in reflection_network. Retrieval: ANN-based (annoy) policy suggestions in suggest_reflexive_policy(). Real-Time Data Integration Sources: Yahoo Finance (yfinance): Market momentum, volatility, commodity prices. Twitter (tweepy): Crowd sentiment via hashtag analysis. World Bank (requests): Historical GDP, trade, inflation. Fallback: Simulated data if API keys are unavailable.