AQAI QuantAI research lab for systematic strategies

Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysisOur backtest

This LSTM's edge rests on a 26-parameter sector intercept

Döbelt's embedding correlates 0.97 with trailing sector returns, while its 0.69 net Sharpe fails to separate from a random forest

2026-09-08 · 8 min read · U.S. equities: large-cap U.S. stocks selected from available listings, using yearly top-N market capitalization as an S&P 500-like universe proxy

Reviewing: Cross-Sectional Heterogeneity in LSTM Networks for Financial Time Series · Julius Döbelt · Read it on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

Daily Dollar-Neutral Sector-Embedded LSTM Cross-Sectional Large-Cap Long-Short

Backtest period 2020-01-01 to 2025-10-08 · hypothetical, net of modelled costs

Why these figures are not the paper's (3)

Run on a different market than the paper

The paper trades S&P 500 constituents with point-in-time membership. Exact historical S&P 500 membership is not available, so the implementable universe would use the yearly top 500 U.S. stocks by aiquant_screening_table.capitalization or a similar large-cap U.S. equity screen. The mechanism—daily cross-sectional return forecasting with sector heterogeneity, short-term reversal, and industry momentum—should still apply to a liquid large-cap U.S. equity universe, but reported paper performance would not transfer directly.

The paper's own figures describe its universe and do not carry over to ours.

This is not a replication of the paper (3)

  • Exact survivorship-bias-free S&P 500 point-in-time membership is not available; use yearly top-N market-cap U.S. equities as a proxy universe.
  • TRBC sector classifications may not be available exactly; use available sector/industry classifications from instruments_full or industries as the categorical embedding input.
  • Macro-financial covariates are buildable only if they map to available macro_indicators/FRED tickers or readable index series; unavailable proprietary macro variables would need to be omitted or replaced by available macro series.

The figures below measure what we could run, not the paper's own method, so they are not evidence for or against its claim.

Our own audit found this run does not follow the paper faithfully (8)

  • Four-year rolling study period split into exactly three years of training and one subsequent year of strictly out-of-sample testing (invalidates: Exact comparability to the paper's 27 complete non-overlapping one-year OOS periods and reported full-sample statistical results)
  • For each stock in the S&P 500 at the end of the training period, retain all stocks available in the training history and keep them in the trading set as long as price data is available, even if delisted during the trading period (invalidates: All paper results tied to survivorship-bias-free Refinitiv S&P 500 constituent reconstruction)
  • Compute simple one-day returns from total return price indices: R^s_t = (P^s_t - P^s_{t-1}) / P^s_{t-1} (invalidates: Exact return, accuracy, Sharpe, drawdown, VaR/CVaR, kurtosis, yearly performance, and transaction-cost-adjusted results from the paper)
  • Market benchmark in paper is Kenneth French value-weighted market return constructed as market excess return plus risk-free rate (invalidates: Paper's market benchmark return, Sharpe, and transaction-cost-adjusted market comparisons)

4 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total -9.6%Oct 2025
Sharpe
-0.03
Total Return
-9.6%
Max Drawdown
-140.3%
CAGR
-1.7%
Volatility
50.3%
Beta vs SPY
0.23
Trades
30,705

What the paper reports for its own strategy

  • Sector LSTM (main 3-layer, k=10, 1995-2024): mean daily long-short return 0.100% before costs, Newey-West t = 7.81, annualized Sharpe 1.41; after 2bp-per-half-turn costs 0.053% daily and annualized Sharpe 0.69
  • Sector LSTM appendix figures (after 2bp per half turn): mean daily return 0.053%, t-statistic 4.106, annualized return 13.3%, excess annualized return 11.2%, annualized volatility 16.41%, share of positive days 52.2%, max drawdown -55.6%, 1% VaR -2.808%, 1% CVaR -4.092%
  • Base LSTM (3-layer, after 2bp per half turn): mean daily return 0.033%, annualized Sharpe 0.39; appendix: annualized return 8.4%, t-statistic 2.615, max drawdown -83.0%
  • Covariate LSTM (3-layer, after 2bp per half turn): mean daily return 0.020%, annualized Sharpe 0.16 (before costs 0.071% daily, t = 4.89, SR 0.83)
  • Sector LSTM mean of the 27 period-specific annualized Sharpe ratios: 1.39 (sd 1.59) before costs, vs 1.16 (sd 1.43) for the base LSTM
  • Sector LSTM traded-portfolio directional accuracy 0.525 (k=10) and full cross-section accuracy 0.506

A trailing industry-momentum tilt accounts for this LSTM's claimed edge. It comes from a two-dimensional lookup table covering eleven sectors, attached to the final layer and carrying 26 free parameters. Döbelt adds it to the Fischer and Krauss architecture. On his results, after-cost annualized Sharpe rises from 0.39 to 0.69. The paper leaves open whether those 26 parameters need a recurrent network around them.

The last-layer addition

The model predicts whether a stock's next-day return will finish above or below the cross-sectional median of the S&P 500. Its input is a 60-day sequence of standardized daily returns. Both the standardization mean and standard deviation come only from the training window. A many-to-one LSTM with hidden size 25 reduces that sequence to its final hidden state, which the base model converts into two logits.

The sector version makes one change. For each stock, its sector under TRBC (The Refinitiv Business Classification) selects an entry from a learnable 11-by-2 matrix. That two-dimensional vector is joined to the hidden state before the linear layer. The resulting logit combines a temporal term, a sector loading and a bias.

The sector loading acts as a cross-sectional intercept. When the embedding favors Energy, an Energy stock can qualify as an outperformer with a weaker temporal signal. Döbelt explicitly says the base and sector models use identical temporal machinery.

Trading follows a fixed rule.

Each day, the portfolio buys the 10 stocks with the highest class-1 probability and shorts the 10 with the highest class-0 probability. Positions last one day and the process repeats. Costs are 2 basis points per half turn, applied only when a position opens or closes. A name held for three consecutive days therefore pays twice rather than six times.

The data begins with Refinitiv monthly constituent lists from January 1995 to December 2024, converted into a membership matrix. Daily total return indices incorporate dividends and corporate actions. Delisted names remain while prices are available. Döbelt divides the sample into 27 rolling four-year blocks, using three years for training and validation followed by one year strictly out of sample. Each block contains roughly 465,000 observations, including about 125,000 in the test set. Pooling the blocks produces 6,793 daily long-short return observations. He uses 60 lags instead of the 240 in Fischer and Krauss, specifically to prevent any training sequence from overlapping a test date.

The paper identifies two sources of return. Over roughly the five days before selection, the temporal component buys recent losers and shorts recent winners, matching Lehmann and Jegadeesh short-horizon reversal. Meanwhile, the embedding favors sectors with stronger performance over the trailing three years, matching Moskowitz and Grinblatt industry momentum.

How far does 0.69 go?

Across 1995-2024, the three-layer sector LSTM earns 0.100% per day before costs, with a Newey-West t of 7.81 and a Sharpe of 1.41. After 2bp per half turn, daily return falls to 0.053%. The corresponding figures are 13.3% annualized return, 16.41% annualized volatility, a 0.69 Sharpe, a -55.6% max drawdown and 52.2% positive days. Transaction costs consume 0.047% per day, nearly half the gross edge. I did not find a turnover figure in the paper against which to stress that assumption.

The significance table sharply limits the result. Döbelt uses the Ledoit-Wolf one-sided studentized block bootstrap with block length 19 and 1,000 iterations. The sector LSTM's 0.69 exceeds the base LSTM's 0.39 at p = 0.045 and the covariate LSTM's 0.27 at p = 0.009. Its comparison with the random forest's 0.53 gives p = 0.204. Against a value-weighted market buy-and-hold at 0.523, p = 0.177.

Only the author's own two LSTM variants lose significantly on Sharpe, and Döbelt prints the p-values directly. Accuracy looks better under the Diebold-Mariano test. The sector LSTM beats the random forest at p = 0.0095, while the comparison with the base LSTM reaches only p = 0.0915. His explanation is straightforward: the embedding enters as a static intercept rather than a time-varying input.

Across sixteen specifications, traded-portfolio hit rates range from 0.514 to 0.525. For the full cross-section, they run from 0.502 to 0.506. The appendix also gives market buy-and-hold a slightly shallower max drawdown, -54.6% versus -55.6% for the sector LSTM. Döbelt's risk claim covers the three active strategies, and within that group the sector portfolio leads. The base LSTM suffered a -83.0% drawdown and the random forest lost -76.2%. On 9 March 2020, the random forest's worst day was -16.72%, compared with -8.25% for the sector model.

The specification table adds another warning. Within the sector architecture, after-cost Sharpe is 0.34 with one layer, 0.45 with two, 0.69 with three and 0.55 with five. Döbelt reports all sixteen specifications and says validation loss determined the three-layer choice. Results within a single architecture span 0.35. The gap over the random forest the model is supposed to beat is 0.16.

Döbelt acknowledges the limitation. His conclusion describes the incremental signal as "modest, conditional on market regime, and has weakened in the last decade". In the same sentence, he calls learnable embeddings "a parsimonious and interpretable way to improve LSTM-based return prediction". The case for parsimony is credible when the addition has 26 parameters. Trading requires a higher bar: those 26 parameters add 0.16 of Sharpe over a random forest, with p = 0.204.

Trailing sector returns in disguise

Döbelt derives a scalar sector contribution from the output weights, then compares it with sector performance. In 2007, its correlation with the training-window median sector return is 0.97. The correlation with realized mean daily sector returns during the test year is 0.74. Energy receives the largest positive loading, and most of the long portfolio consists of Energy stocks.

In effect, the embedding makes a one-year-ahead wager on three-year trailing sector performance. During 2007, it placed Energy 87.6 percentage points more heavily in the long leg than in the short leg.

Across all 27 forecasting periods, average correlation with realized returns is 7.6%. Alignment also forecasts whether the sector model adds value. Pearson r is 0.38 against the yearly accuracy difference, with permutation p = 0.025, and 0.40 against the Sharpe difference, with p = 0.019. The Holm-adjusted result is 0.038. The sector model gains when trailing rankings continue and loses when they reverse.

The reversal appears clearly in 2003. Sector contribution correlates 0.86 with trailing sector returns, then -0.82 with realized mean returns and -0.91 with realized total returns. Technology carried a -0.08 loading and was shorted by -68.0 percentage points, yet returned 53.73%, the best sector result. Utilities received a favorable loading and was long by 63.8 points, then returned 15.09%.

The paper places the negative-correlation episodes in 1999-2003, 2010-2013 and 2021-2024, each following a crash, and interprets them through Daniel and Moskowitz. I would want to see an industry-momentum portfolio with a reversal overlay and a crash filter. The paper does not run that benchmark. Its three comparisons are the base LSTM, a Random Forest and a value-weighted market buy-and-hold.

The broader regime evidence is even less favorable. In the early study periods, annualized Sharpes exceed 4 for every model. They are about 2 by 2008. After 2008, average daily returns remain consistently below 0.1%, and several years have negative Sharpe ratios. From 2018 through 2024, the random forest is marginally positive or negative. Döbelt states the deterioration plainly: with the exception of 2020, there is no year afterwards where all the models perform exceptionally well. The pooled 0.69 is a 1990s result carrying two decades of drag.

Our run used a substitute universe

We adapted the architecture to assets we can trade. Exact survivorship-bias-free point-in-time S&P 500 membership is not available to us, so we substituted a yearly top-500 U.S. stocks by capitalization screen. We also lacked TRBC sector codes and keyed the embedding to the sector labels available to us. Several macro series used by the paper were unavailable, so we left them out and ran only the sector model. We therefore make no covariate comparison at all. No number below tests the paper's claim.

Our run spans 2020-01-02 to 2025-10-08. It uses the three-layer, H=25, D=2 architecture with 60-day standardized return inputs, label smoothing 0.1, gradient clip 2.0, weight decay 1e-4 and early stopping patience 20. Each day it goes long 10 and short 10. Cumulative return was -9.57%, with a Sharpe of -0.03. The win rate came to 44.72%, versus the paper's 52.2% positive-day share.

Döbelt's 0.69 is net of costs and pooled across 1995-2024 on his constituent set. Our -0.03 covers 2020 to 2025 on a capitalization screen. The universes differ, as do the periods and price series. Our inputs use close-price returns, while his use total-return indices.

Risk reveals more of the difference. Our portfolio operated at around 200% gross and realized 50.33% annualized volatility. Döbelt's sector LSTM realized 16.41%. We charged a per-share commission of $0.004 and modelled zero slippage. At our leverage, that commission bites roughly three times as hard as his 2bp per half turn. Beta to SPY was 0.23, more market exposure than this portfolio construction should carry. Maximum drawdown reached -140.25%, compared with -55.6% for the paper's sector LSTM. Our leveraged run and his portfolio are plainly different animals.

The window likely explains much of the gap. Döbelt's year-by-year evidence shows the strategy weakening in the post-2009 period. It also shows negative alignment between the embedding and realized sector returns in 2021-2024, almost our entire test span, with 2020 as the one favorable year. The universe proxy, close-price inputs and leverage are other plausible contributors, though we cannot quantify any of them. Together, they could take 0.69 down to roughly zero. They do not fully explain the negative drift. Our automated single pass on a substitute universe is evidence about our implementation first, rather than a verdict on the authors' work.

A post-2008 horse race against a managed industry-momentum and short-term-reversal portfolio with a crash filter would settle the issue more directly. Döbelt instead compares the model with the base LSTM, the Random Forest and market buy-and-hold. If the embedding beats that managed portfolio, it earns its place. For now, the paper's strongest result is the one that cuts against its headline: the network recovers two known factors and stops paying when they do.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

For each test year Y in 2020..2025:
    Select top 500 STOCK symbols by aiquant_screening_table capitalization for year Y, is_adr=false
    Build daily close-to-close simple returns for available symbols
    Use the prior 3 years as training data and year Y as out-of-sample test data
    For each stock:
        Compute mean and stdev using training returns only
        Standardize both train and test returns with those fixed training statistics
    Create samples using 60 lagged standardized returns
    Label each sample as 1 if next-day stock return > next-day cross-sectional median, else 0
    Train sector_lstm:
        input = 60-day standardized return sequence
        static input = sector/industry identifier from instruments_full
        architecture = 3-layer many-to-one LSTM, H=25, sector embedding D=2 at final logits
        objective = cross-entropy with label smoothing alpha=0.1
        optimizer = Adam, weight_decay=0.0001, global_grad_clip=2.0
        early stop on validation_loss with patience=20 and restore best weights

For each test trading day t:
    For every eligible symbol with required close data:
        score_up = model probability of class 1 for t+1 relative return
        score_down = model probability of class 0 for t+1 relative return
    Long the 10 highest score_up names
    Short the 10 highest score_down names
    Weight longs +1/10 and shorts -1/10, subject to 10% max position size and max leverage 4.0
    Execute at the close using the platform's market-on-close convention
    Hold for one trading day, then rebalance to the next day's selected book
    Apply transaction costs on entries, exits, and position changes only