Hybrid Deep Learning Model: LLM-Enhanced Linear Self-Attention Transformer-CNN for Stock Price Prediction

Abstract

The unpredictability and inherent complexities of financial markets pose persistent challenges in accurately forecasting stock prices. Recent innovations in deep learning methodologies, specifically the integration of technical analysis with linguistic processing through large language models (LLMs), have demonstrated significant potential in financial forecasting applications. Despite substantial advances in deep learning for stock price prediction, current approaches face critical limitations in effectively integrating multi-modal data while lacking sophisticated technical analysis capabilities, particularly when simultaneously modeling both temporal dependencies in financial time series and spatial features from stock chart visualizations across diverse market regimes and volatility conditions.

This study proposes an innovative hybrid deep learning framework aimed at addressing these multi-modal integration challenges, technical analysis limitations, and cross-market adaptability issues through a structured three-phase methodology. In the initial phase, we leverage an LLM to conduct thorough technical analysis of historical financial data, transforming raw market information into rich textual representations. These representations encapsulate intricate financial patterns using well-established technical indicators such as Bollinger Bands (BBs). By interpreting these financial metrics, the LLM generates structured textual summaries that enhance the downstream predictive modeling process.

The second phase involves dual processing pipelines. One pipeline utilizes an optimized attention-based Transformer to process the LLM-generated embeddings, capturing long-range market dependencies. Concurrently, the second pipeline deploys a Convolutional Neural Network (CNN) to analyze visual stock chart patterns, extracting spatially relevant features that complement the temporal insights derived from the Transformer model.

In the concluding phase, the outputs from both pipelines—textual, sequential, and visual data—are aggregated to generate holistic stock price predictions. The architecture employs Transformers for extracting extended temporal relationships within market sequential information, whereas Convolutional Networks detect regional spatial configurations from graphical stock representations. Large language models enhance our dataset by generating sophisticated technical metrics, subsequently embedded into numerical vectors through FinBERT encoding techniques. The fusion of these multimodal representations enables a more robust and accurate forecasting model, capitalizing on the complementary strengths of each component to enhance predictive performance.