A crash-probability model whose probabilities are worse than a constant can still be worth trading. Sit with that before dismissing a crash PR-AUC of 0.17.
Start with what Haque, Ahmed, Rahman, Robin, Meem and Noor built. The data is daily OHLCV bars for Dhaka Stock Exchange equities, 2008 to 2022. After equity filtering, warm-up drops and labeling, 924,946 ticker-days survive across 321 equity tickers. Each ticker-day carries eighteen technical indicators. Among them: daily return, intraday volatility, rolling 5-day volatility and Bollinger bandwidth. Five further indicators were dropped for correlating above 0.98 with retained ones.
Then label tomorrow's state rather than tomorrow's return. Each ticker gets its own thresholds, fitted only on training-period data. A Crash day is one where the raw return falls below the median minus two MAD-scaled deviations (median absolute deviation scaled by 1.4826). A High-Volatility day is one where intraday volatility exceeds the training 75th percentile. Everything else is Normal. Crash is evaluated first, so the classes are mutually exclusive. The label is shifted forward one session: the feature vector at t is paired with the state realized at t+1. The training mix is 73.8% Normal, 20.6% High-Volatility, 5.6% Crash. Out of sample it is near-identical at 73.79 / 21.50 / 4.71.
That framing is the paper's real contribution. It fits how money is actually managed better than another closing-price regression does. Crash prevalence runs about 5%. Random Forest reaches 0.52 precision among its 100 most confident alerts, and at that end of the ranking the signal turns into a de-risking rule.
Seven models compete: elastic-net logistic regression, Random Forest, XGBoost, LSTM, GRU, a three-state Gaussian HMM, and GJR-GARCH(1,1) with Student-t innovations. Validation is five expanding-window walk-forward folds on 2008 to 2018, with a five-session embargo before each validation block. 2019 to 2022 is held out entirely. Random Forest wins the primary metric with a cross-validated crash PR-AUC of 0.1697 plus or minus 0.0134. XGBoost follows at 0.1409 and LSTM at 0.1407, with the HMM last at 0.0776.
Does the leakage argument hold?
The data handling is genuinely careful. Winsorization bounds at the 1st and 99th percentiles are computed on training data and applied forward. Missing values from rolling-window computations are handled by forward fill only, applied within each trading code. Warm-up rows that remain missing are dropped rather than imputed. Label thresholds are per-ticker, fitted on MAD statistics from training data only. Tickers with fewer than 30 qualifying observations get a pooled fallback, 11 of them, and those 11 show a 4.11% next-day crash rate against 5.30% for tickers with their own history. SMOTE, where used, is refit inside each fold.
The residual issue is one the authors raise themselves. Two of the eighteen features, daily return and intraday volatility, are mechanically the inputs to the labeling rule. They quantify it. Dropping both cuts crash PR-AUC by 4.8% for XGBoost, 5.6% for Random Forest and 12.0% for logistic regression. Small. But the GRU's single most important permutation feature is intraday volatility, at 0.05873, ahead of every other input. The tree models' rankings sit better. Bollinger bandwidth leads for XGBoost at 0.04433, rolling 5-day volatility for Random Forest at 0.03018, with intraday volatility only third in both. So the tree-model attributions are the more trustworthy ones here.
The harder question is what the models beat. A naive persistence rule (tomorrow's class equals today's) achieves test crash precision of 0.1736 and recall of 0.1737, with overall accuracy of 0.7262. Both precision figures beat XGBoost (0.1220) and logistic regression (0.1250) out of sample. A one-variable heuristic flags High-Volatility whenever yesterday's intraday volatility exceeded the training 80th percentile. It reaches 0.7447 accuracy. Random Forest posts 0.6842 support-weighted average recall across all three classes on the same held-out set. Random Forest's crash precision of 0.1514 sits below persistence's 0.1736; its edge is recall, 0.2436 against 0.1737. Set those side by side and the margin over a one-line rule is thinner than a seven-model comparison implies.
The statistics say less than the model table
The Friedman test across five folds and seven models gives chi-square 19.1143, p = 0.0040. The null of equal performance goes. Nemenyi's critical difference at alpha 0.05 is 4.0291, and exactly one pairwise gap clears it: Random Forest (average rank 1.0) versus HMM (6.6, difference 5.600). Random Forest versus XGBoost does not. The authors are straightforward about this. They also report that all 21 Wilcoxon comparisons collapse to p_holm >= 1.0 after Holm-Bonferroni, and they note the smallest attainable raw two-sided p at N=5 is 0.0625. Five folds cannot support pairwise inference. They say so.
Ranking and operational metrics disagree, and the paper surfaces the disagreement rather than hiding it. On the held-out 2019 to 2022 test set, Random Forest leads on PR-AUC. It also posts the lowest event-level crash recall of the five machine learning classifiers, 0.3606. Event-level recall credits a hit within one trading day of a genuine crash. GRU tops that table at 0.5505 on mid-pack PR-AUC. LSTM has the best precision among its 50 most confident alerts, 0.64. GJR-GARCH has the best recall at a 5% false-positive budget, 0.2271, and zero crash recall at its own default threshold. It predicts Crash on 0.34% of test rows against 4.68% true prevalence. The ranking carries information; the decision rule does not.
Calibration is the finding I would put in front of any risk committee. Every one of the seven models scores a negative Brier Skill Score against a prevalence-only baseline: GJR-GARCH -0.0133, LSTM -0.8369, Random Forest -0.8454, XGBoost -1.2708, HMM -3.3143. Expected calibration error runs between 0.167 and 0.240 for the classifiers. The abstract concludes that the framework "can effectively support market stress prediction and risk-aware decision-making." Section 4.5 reports the negative skill scores. The body discloses what the summary does not. Inverse-frequency class weighting buys ranking and pays for it in absolute probability accuracy. The output is an alert score. Feeding a 0.2276 crash probability into a risk budget as though it were a 22.76% chance of a crash would be wrong by a wide margin.
The trading rule is one event wide
The paper's own backtest gates on Random Forest crash probability. The strategy exits to cash whenever that probability exceeds its 95th percentile (p > 0.2276), and otherwise holds a fully invested long position. Positions decided at t apply to the realized return at t+1. Pooled equal-weighted over 2019 to 2022, it returns 67.26% total (15.01% annualized) against 45.52% (10.74%) for buy-and-hold. Volatility is 16.39% versus 18.11%, maximum drawdown -32.21% versus -38.41%, Sharpe 0.94 versus 0.65.
Read the construction alongside the numbers.
The rule sits in cash on 5.00% of stock-days. The paper states that most of the advantage is concentrated in the COVID-19 window, and the regime split supports that reading. Crash PR-AUC during March to May 2020 is, in the authors' words, "roughly double those observed in the calmer surrounding regimes." Logistic regression moves 0.1207 to 0.2864, XGBoost 0.1437 to 0.2257. Random Forest is the most stable across regimes at 0.1268 to 0.2453. So the headline 21.7 percentage point outperformance rests substantially on a single episode.
Returns are reconstructed from the Winsorized daily_return feature. The authors call that a conservative choice, likely understating the edge. Capping at the 1st and 99th percentiles truncates precisely the tail the crash gate exists to dodge. It also caps buy-and-hold's drawdown, so the direction of that distortion is not obvious. No transaction cost, slippage or borrow assumptions are stated for this backtest. Nor is there a liquidity screen across 321 DSE tickers, in a market whose exchange closed for 66 days in 2020. And the 95th-percentile threshold is a rank cutoff applied on the test period itself.
Adapting it to US equities
We could not trade DSE names. We adapted the mechanism, so nothing below validates or refutes the paper's DSE results. The features are generic OHLCV derivatives, so the transfer is mechanically clean even where the market is not comparable.
We kept the three-state labeling, the per-ticker thresholds fitted on MAD statistics, the five-fold expanding-window validation with a five-session embargo, and a class-weighted Random Forest tuned on crash PR-AUC. Two things changed. The universe is up to 321 capitalization-ranked US common stocks, reconstituted yearly from contemporaneous screening records, ADRs excluded. And we added a middle gear: a High-Volatility alert halves existing exposure and blocks new entries. The paper's rule stays fully invested long outside its crash exits.
Over 2020-01-01 to 2024-07-01, our overlay returned 25.22% total, 5.14% CAGR, Sharpe 0.47. That is net of $0.0040 per share commissions across 278,382 fills. The paper reports 67.26% total and 0.94 Sharpe on its own 2019 to 2022 DSE test. Ours lands well below on both. The two runs measure different things: different market, different window, gross versus net, and a different exposure rule.
Three differences are verified and all point the same way. The first is the size of the avoidable idiosyncratic tail. In Dhaka, single-name suspensions cluster ahead of crashes: 1,353 suspension-flagged rows across 285 tickers were followed by a Crash label 26.2% of the time, against a roughly 5% base rate. US large caps offer nothing comparable to harvest. Second, our extra High-Volatility layer kept us chronically underinvested. Realized beta to SPY was 0.51, volatility 12.03% against their 16.39%, maximum drawdown -21.63% against their -32.21%. That caps upside through the 2023 to 2024 rally our longer window includes. Third, a daily-rebalanced 321-name book paying commissions on 278,382 trades carries a drag their gross figures never incur. Aggregation likely differs too. Our 25.22% is additive. Their 67.26% against a 15.01% annualized figure is consistent with compounding on capped inputs.
We cannot fully close the gap with what we can see. Without a matched benchmark and stress-window attribution for our run, we cannot separate lost participation from a genuinely weaker crash signal in US large caps. Our implementation also covers a subset of the paper's model families, so the traded classifier need not match the one behind their 67.26%.
Worth borrowing
The three-state target with per-ticker MAD-fitted thresholds and a t+1 shift is a formulation I would use. The disclosures are unusually good. The authors even admit that no McNemar test was run, because row-level prediction logs were not retained.
The trading claim is where the evidence stops short. The simulation charges no transaction costs and gates only on Random Forest. Strategy statistics are pooled over 2019 to 2022, with no separate non-COVID return, volatility or Sharpe, though Fig. 11 does note that the strategy tracks buy-and-hold closely outside the crash window. Run the persistence baseline and the one-variable volatility heuristic through the same gate. Cost the turnover. Split out the non-COVID period. Then the claim becomes testable. For now the useful output is a ranked alert list carrying 0.52 precision among Random Forest's 100 most confident signals.
Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.