Augmenting large language models for financial sentiment analysis: a heuristic sparse mixture-of-experts framework
Abstract
Financial sentiment analysis (FSA) is essential for extracting actionable market insights from unstructured textual data to support investment decisions and risk management. Although large language models (LLMs) demonstrate remarkable capabilities in general natural language processing, they nevertheless struggle to comprehensively capture the domain-specific semantics, numerical relationships, event impacts, and implicit sentiment expressions critical for robust financial sentiment interpretation within a unified framework. To address these limitations, this study proposes HSMoE-FSA, a novel Heuristic Sparse Mixture-of-Experts framework designed to augment LLMs for financial sentiment analysis. Specifically, the framework integrates four specialized LLM-based experts: FinSem for interpreting financial terminology, FinNum for modeling quantitative sensitivities, FinEvent for evaluating market-impact events, and FinSent for detecting latent sentiment cues. Leveraging an entropy-based routing mechanism, the framework dynamically activates optimal expert combinations through Top-K sparse gating and weighted aggregation, thereby enabling synergistic multi-dimensional feature fusion. Furthermore, a phased training strategy decouples expert module learning from router calibration to ensure stability. Extensive evaluations on benchmark datasets demonstrate that HSMoE-FSA achieves state-of-the-art performance, with weighted F1 scores ranging from 89.67% to 92.29%, representing absolute improvements of 0.72% to 1.74% over current leading methods. Moreover, qualitative analysis further confirms the framework's superior capability in resolving nuanced financial semantics and implicit sentiment through expert collaboration.