AQAI QuantAI research lab for systematic strategies

Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysisOur backtest

Six percent better than persistence, and only from day four

Buchegger and Gonon recover the level, skew and curvature of SPX surfaces, then understate their size by a quarter

2026-09-08 · 10 min read · US ETF options and US equity ETFs, primarily SPY options and SPY shares for hedging

Reviewing: Arbitrage-Aware Multi-Step Forecasting of Implied Volatility Surfaces: Modelling Surface Trajectories Using Latent Diffusion · Dominik Manuel Buchegger and Lukas Gonon · Read it on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

Arbitrage-Aware SPY Surface Diffusion Relative-Value Strategy

Backtest period 2020-01-01 to 2024-07-01 · hypothetical, net of modelled costs

Why these figures are not the paper's (3)

Run on a different market than the paper

The paper uses SPX index-option surfaces, whereas the implementable version would use SPY ETF-option surfaces and SPY as the hedge underlying. The surface-dynamics mechanism—forecasting changes in the level, skew, curvature, and return-volatility relationship of a liquid broad-equity options surface—should survive this substitution, but any reported forecasting results on SPX do not transfer directly to SPY.

The paper's own figures describe its universe and do not carry over to ours.

This is not a replication of the paper (3)

  • The exact OptionMetrics/IvyDB quote-cleaning and operator-deep-smoothing pipeline cannot be reproduced because the platform does not provide the paper's raw SPX quote data, bid-ask information, or its full historical 2000-2025 sample.
  • The implementation must construct surfaces from available end-of-day option contract prices, implied volatilities, and Greeks, rather than replicate the paper's pretrained graph-neural-operator smoothing procedure exactly.
  • The available options history is only suitable for the platform's modern coverage window, rather than the paper's full pre-2020 training sample. A diffusion model can be trained, but it will be a shorter-sample SPY-surface implementation rather than a replication of the reported SPX model.

The figures below measure what we could run, not the paper's own method, so they are not evidence for or against its claim.

Our own audit found this run does not follow the paper faithfully (6)

  • Underlying instrument: The implementation uses SPY options and SPY returns instead of the paper's SPX options and SPX returns. (invalidates: All reported SPX forecast errors, CRPS and energy-score comparisons, PCA statistics, return-surface correlations, arbitrage shares, repair magnitudes, interval coverages, sample counts, and split counts.)
  • Surface smoother: The paper's pretrained SPX operator smoother is replaced by a smoother trained or fine-tuned on pre-test SPY data. (invalidates: The paper's autoencoder reconstruction error, static-arbitrage diagnostics, factor alignment, distributional scores, and RMSE and MAE gains.)
  • Predictive-dispersion calibration: A rolling horizon-and-factor-specific residual-dispersion correction is applied before trading. (invalidates: The paper's raw 50% and 95% interval coverages and direct comparisons with its uncalibrated CRPS and multivariate energy scores.)
  • The specification contains incompatible surface prefilters: the operational minimum is K/F=exp(−1.5)=0.2231301601, while constants specify 0.55; applying 0.55 excludes 62 of the paper’s 170 supported grid coordinates—including 10 of 21 coordinates at τ=1—and leaves only 108.

2 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total -1.0%Jul 2024
Sharpe
-0.00
Total Return
-1.0%
Max Drawdown
-329.1%
CAGR
-0.2%
Volatility
175.2%
Beta vs SPY
0.16
Trades
158

What the paper reports for its own strategy

  • Surface point-forecast RMSE aggregated over 30 horizons: 0.01262 vs persistence 0.01343, +6.09% gain; MAE gain +3.45%; bias +0.00022 (test set 2023-Aug 2025, no transaction costs; forecasting exercise, no trading strategy)
  • h=1: RMSE 0.00619 vs persistence 0.00535 (-15.72%); MAE -34.49%
  • h=5: RMSE 0.01017 vs 0.01043 (+2.52%); MAE -0.75%
  • h=10: RMSE 0.01223 vs 0.01305 (+6.28%); MAE +4.46%
  • h=21: RMSE 0.01367 vs 0.01459 (+6.29%); MAE +4.30%
  • h=30: RMSE 0.01460 vs 0.01567 (+6.83%); MAE +4.95%

The model earns a place as a scenario engine with a calibration problem. As a volatility forecast you can trade, its one-day result fails the test.

Buchegger and Gonon construct a two-stage generator for implied volatility surfaces. Stage one reduces every daily surface to eight numbers. A residual CNN encodes the log time-scaled surface from a fixed 170-point grid spanning log-forward-moneyness and maturity. The decoder uses a coordinate network. Feed it any (m, tau), and it returns an implied vol.

Two soft no-arbitrage penalties enter the training loss. Butterfly violations are measured through Durrleman's condition (lambda 1e-3), while calendar monotonicity carries lambda 1e-4. Both penalties are calculated on a dense 0.02-step grid. Admissibility therefore sits inside the decoder instead of depending on a parametric family or post-hoc repair.

Stage two produces the forecast. A Transformer denoiser receives 21 trading days of latent displacements, anchored at the window's opening point, together with daily returns and 27 contemporaneous state variables. From those inputs it jointly generates 30 days of future latent displacements and 30 daily log returns. Generation takes one non-autoregressive pass with 500 diffusion steps and a v-parameterised target. Each forecast origin gets a thousand paths.

The headline depends on an additional adjustment. Validation results showed that the predictive mean overstated movement, so the authors freeze the diffusion model and estimate a horizon-specific scaling gate. This multiplier pulls the mean toward persistence, with an average fitted value around 0.80. The original sampled residuals are restored, leaving dispersion unchanged. For the final model, the gate uses the same training plus validation data previously seen by the diffusion model.

The data comprises OptionMetrics IvyDB SPX end-of-day quotes obtained through WRDS, covering January 2000 to August 2025. A pretrained graph neural operator smooths each day, producing 6,429 daily surfaces. The chronological split contains 5,511 training observations through 2021, 251 validation observations in 2022 and 667 test observations from 2023 onward. The authors refit the final model on 5,712 combined trajectories.

Across all 30 horizons, the model mean records RMSE of 0.01262 against 0.01343 for functional persistence, a 6.09% gain. The MAE gain is 3.45%, with bias of +0.00022. This remains a forecasting exercise. There is no trading P&L and no allowance for transaction costs. The return channel matches the zero-return baseline, and the authors claim no return predictability.

Why the persistence column matters more than the 6%

Buchegger and Gonon's related-work section argues that few forecasting studies test persistence thoroughly, and fewer defeat it convincingly. Their own evaluation makes the comparison directly and publishes the losing column. At h=1, persistence wins decisively. Model RMSE is 0.00619 against 0.00535, a 15.72% deterioration, while MAE is 34.49% worse.

RMSE gains become positive at roughly h=4, with MAE following at h=6. The improvement then settles into a narrow range instead of increasing: +6.28% at h=10, +6.29% at h=21 and +6.83% at h=30. Corresponding MAE gains are 4.46%, 4.30% and 4.95%. Absolute errors still rise. At h=10, model RMSE reaches 0.01223 against persistence at 0.01305. The figures are 0.01367 against 0.01459 at h=21, then 0.01460 against 0.01567 at h=30. Error nearly triples from h=1 even as the percentage advantage stays near 6%.

Yesterday's surface wins at one day. From two weeks out, the model leads by about 6%. This reversal is the paper's most useful result. The authors argue that recent dynamics contain medium-horizon information, while noise dominates the next-day innovation. Their results partly bear that out. Autoencoder-only reconstruction RMSE is 0.0039, roughly 63% of the full h=1 error of 0.00619. Representation error therefore dominates at one day. The authors also find that increasing autoencoder expressiveness can make the diffusion problem harder.

The location of the 6% gain is harder to celebrate. RMSE improves by 11.69% at short maturities, 6.46% at medium maturities and 4.99% at long maturities. Both wings improve by about 6% to 7%, compared with 2.45% in the central region. The paper links this pattern to movement: the largest gains appear where the surface moves most, while the stable centre offers little movement to exploit.

Liquidity gives the same pattern another reading. The paper reports local half-spreads from about 3.5 IV bps at the money to 45 bps in the deep wings. An 11.69% RMSE improvement at short maturities has value, as does 6% to 7% in both wings over weekly-to-monthly horizons. Those regions also make a mid-market improvement hardest to monetise.

Directions right, amplitude a quarter short

Distributional forecasting is the paper's stated purpose, and its largest weakness appears there. Realised log surface increments are close to three-dimensional. Their first three principal components explain 83.95%, 5.41% and 2.41%, reaching 91.76% cumulatively. When generated surfaces are projected onto the same basis, the loadings align at 0.996, 0.962 and 0.831. The model reproduces the shapes of level, skew and curvature.

Table 2 supports that result through correlations between realised-PCA scores and surface descriptors, with generated values averaged across sample paths. At h=21, PC1's correlation with average-IV changes is +0.977 in realised data and +0.963 in generated data.

Amplitude falls short.

At h=21, PC1 generated standard deviation is 0.669 against a realised 0.923. By h=30, the comparison is 0.769 against 1.021. At a month, the dominant factor is about a quarter too small. Pointwise coverage tells the same story. Nominal 50% intervals cover 0.400 at h=1 and 0.340 at h=30. Nominal 95% coverage declines from 0.856 to 0.811. As the paper puts it, uncertainty expands over time, though insufficiently.

The authors acknowledge both underdispersion and poor one-day performance in the conclusion. Their defence is that the framework should be judged as a probabilistic trajectory model rather than as a one-step point forecaster. The concession appears in the introduction and conclusion. The abstract presents only the favourable side, saying the framework generates realistic probabilistic multi-step scenarios while outperforming the persistence benchmark in point forecasting. It omits the h=1 loss and underdispersion.

That defence deserves pressure because calibration is the purpose of a trajectory model. The 0.340 figure averages coverage over grid points and test origins at h=30. Intervals this narrow will understate stress moves in the factor responsible for 83.95% of increment variance. Undercoverage changes location as well. It is worst in the short-maturity right wing at h=1, then shifts toward the central region at medium and long maturities farther out. Realistic-looking paths serve a different purpose from correctly sized paths. A risk system needs the latter.

Two further results make the weakness clearer. Against a bootstrap persistence benchmark, most of the short-horizon probabilistic gain vanishes. At h=1, average IV level improves by only +4.4%. The model loses on 3M skew, 3M curvature and the short-long spread by -20.9%, -26.8% and -61.0%. Against deterministic persistence, those same descriptors show +10.4%, +5.7% and -26.6%.

Performance improves with horizon. At h=10, the model beats bootstrap persistence in 8 of 9 descriptors and 7 of 9 grid buckets. By h=30, those counts rise to 8 and 8, leaving 3M curvature at -1.5% as the remaining loss.

The h=1 leverage effect is also badly missed. Table 3 reports a generated correlation between returns and changes in average IV of -0.464, against -0.791 realised. Agreement becomes closer from h=5 onward. At h=5, realised and generated values are -0.326 and -0.290. They move to -0.276 and -0.219 at h=10, then -0.208 and -0.186 at h=21.

Soft enforcement of admissibility

Among individual sampled surfaces, 88.1% are arbitrage-free. The share rises to 98.4% for the ensemble mean and 97.7% for the median. Calendar violations occur in 0.00% of checks, while butterfly violations appear in 0.13% for samples.

At each surface's worst grid point, the required repair averages 15% of the median half bid-ask spread for individual samples and 5% for ensemble surfaces. The typical correction is 1.4 IV bps, compared with local half-spreads from 3.5 to 45 bps. Sample worst-case repair reaches 605% of the half-spread, while the ensemble mean peaks at 25%. Disabling both penalties cuts the arbitrage-free share to about 24% and increases required corrections two- to fivefold.

The authors make a careful claim from these results. The mechanism most likely enforces admissibility rather than eliminating exploitable arbitrage, since the training surfaces contain only small violations. For an option-book simulator, using ensemble-level surfaces reduces the violating share to 1.6% (98.4% arbitrage-free), compared with 11.9% for individual DDPM samples. A path-dependent risk calculation samples individual paths and therefore inherits the larger figure.

The evaluation covers one underlying and one brief out-of-sample period: 667 test observations, 637 origins and overlapping 30-day paths. Perhaps twenty to thirty non-overlapping trajectories are effectively independent. We did not find standard errors or significance tests for any gain column.

No alternative model is run. DYSANOS, VolGAN, ConvLSTM and the functional-PCA neural SDE work receive discussion only. The conclusion identifies DYSANOS and a SANOS-based diffusion model as the first comparisons to run next. This research family often omits benchmarks altogether. We encountered the same problem while reviewing a survey of thirty diffusion-in-finance papers with no common scorecard (/articles/thirty-diffusion-papers-four-scorecards-nothing-to-compare-them-on). In that setting, a paper that publishes its -15.72% RMSE loss at h=1 and releases code deserves more credit than one that hides both.

What we ran on SPY

We could not reproduce the paper's setup. Our source has no raw SPX quotes, no bid-ask detail and no history before the modern window. The paper's daily operator-based smoothing step was therefore unavailable to us. Our usable options history spans 2020-01-01 to 2024-07-01, a modern-coverage window far removed from the paper's 5,511 training observations through 2021.

We also cannot hold SPX. We instead built surfaces from available end-of-day SPY option prices, implied vols and Greeks, then hedged with SPY shares. The mechanism should carry across that substitution because forecasting changes in level, skew, curvature and the return-volatility relationship poses the same problem for a liquid broad-equity surface in SPY as in SPX. Their SPX forecast errors, factor loadings and arbitrage shares do not transfer to SPY.

Our implementation also replaced the latent diffusion model with an AR(1) forecast of conditional ATM implied vol. This is an adaptation of the trade idea. It tests nothing about the paper's claim.

We traded delta-hedged SPY straddles, roughly 25-delta risk reversals and butterflies using contracts with 35 to 180 DTE. Entry required a one-sided forecast probability of at least 0.90, and expected edge had to exceed 1.25 times estimated round-trip cost. We sized positions inversely to package vega, allowed three concurrent structures at most and hedged daily at the close. Costs were $0.65 per contract, plus a 0.5% half-spread debit on each option leg and 1bp of SPY hedge turnover.

The held-out trading period ran from January 2023 to July 2024. Our figures were -1.01% total return, Sharpe approximately 0.00, 175.21% annualised volatility, maximum drawdown of -329.11% and beta to SPY of 0.16. The win rate was 64.7%, while profit factor reached 0.71. Average wins were $450.10 against average losses of $1,161.99.

The paper reports a gross +6.09% improvement in surface RMSE, 0.01262 against persistence at 0.01343, with no trading result at all. We report -1.01%. These are different objects, measured on different universes with different signal models. The gap supplies no evidence about their work.

Most of the discrepancy can be traced. An AR(1) fitted to the ATM level has no skew or curvature dynamics. Using it to trade risk reversals and butterflies amounts to a coin flip that pays two to four spreads per round trip. The paper finds its edge at short maturities and in the wings, exactly where our surrogate lacks information.

Our conditioning window is dominated by the COVID spike and the 2022 bear market. It is much narrower and more extreme than their 5,511 observations. Any fitted magnitude for departure from persistence is likely to be miscalibrated for 2023-24.

A book hedged to delta at every close should not produce 175% annualised volatility and 0.16 beta to SPY under a 4.0 leverage cap. Those figures suggest a sizing or hedge-accounting error rather than a signal result. A -329% drawdown is also impossible under that cap. The combination of a 64.7% win rate and 0.71 profit factor reinforces the diagnosis. We cannot fully explain our own P&L. These numbers should be read as a diagnostic of our implementation.

One result would change my view from scenario engine to trading tool: nominal 50% grid-average coverage rising from 0.340 to near 0.50 at h=30, while retaining the 6.29% RMSE gain at h=21 across more than one underlying. Until then, its honest use is stress generation for short-dated skew and wing risk, with PC1 dispersion about a quarter short at h=21 and h=30.

Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

At each SPY market close:
  1. Load only same-date-or-earlier SPY closes, option marks, Greeks, and contract metadata.
  2. Retain liquid options with 35-180 DTE, K/F in [0.75, 1.25],
     close >= $0.05, open interest >= 500, and volume >= 100.
  3. Update the chronological volatility forecast.
     - Research specification: arbitrage-aware 8-factor surface autoencoder plus
       a calibrated 30-day latent diffusion ensemble.
     - Supplied backtest implementation: AR(1) conditional ATM-IV surrogate.
  4. Form eligible same-expiry packages from real listed contracts:
     - Level: ATM call-plus-put straddle.
     - Skew: approximately 25-delta put/call risk reversal.
     - Curvature: wing/body/wing butterfly.
     Reject a package if a required leg, close, Greek, or acceptable delta match is absent.
  5. Evaluate eligible 5-30 trading-day horizons. Require a one-sided probability
     of at least 0.90 and expected edge of at least 1.25 times estimated round-trip cost.
  6. Reject entries that fail the dense-grid calendar or butterfly residual screen.
  7. Size accepted packages by inverse absolute package vega, using integer contracts,
     with at most three concurrent structures and all premium, capital, and leverage caps.
  8. Execute option legs at recorded EOD closes and establish the SPY delta hedge at
     the same close. Recalculate the hedge daily from contemporaneous option deltas.
  9. Exit when the selected forecast horizon is reached, the opposite-direction
     probability reaches 0.60, the arbitrage screen fails, or DTE reaches 10.
     If a real mark is unavailable, defer a non-expiry exit; settle expiry from payoff.