The interval supplies the denominator
Conformal Kelly's claimed edge vanished once the tuned window ended. The 2.1 points of annual growth over a textbook standard deviation came entirely from development. Calibration carried into the sealed data. Growth failed to follow.
The test uses Eight liquid US ETFs and daily bars, with one model for each asset. An expanding-window ridge regression sends four unstandardised features into a forecast of the forward H-day summed return. Those inputs are momentum over 21, 63 and 252 days, plus EWMA(20) volatility. Lambda is 10. The model refreshes every 21 trading days, with the first prediction after 750 days. Five horizons, 12, 16, 21, 27 and 34 days, are rescaled to a 21-day reference, then blended using weights proportional to 1/q_h^2. Forecasting strength is deliberately limited. Even a drift-only version, using a pure intercept and zero features, produces 23.26% annualised net log growth during development, equal to 82% of the final result.
The conformal machinery enters through the denominator. Its nonconformity score is simply the absolute residual from the 21-day forecast. On each date, the scale takes the 75% empirical quantile of the latest 500 scores whose outcomes have landed. Geometric shrinkage then pulls that quantile toward an expanding all-history anchor with weight 0.3. The half-width is divided by 1.2816 and labelled sigma. Fractional Kelly sets f = 0.15 times mu over sigma squared, winsorised at plus or minus 0.75 for each asset. Gross exposure is renormalised to 2.0 whenever it crosses that level. Execution is lagged one day, and turnover costs 5 bps per unit. The live book averages equally the target books formed during the last five days.
The case for using interval width as the Kelly scale comes from influence functions. A quantile has a bounded one, mean absolute deviation has a linear one, and standard deviation has a quadratic one in the residual. Ryan argues that nonlinear position sizing passes estimation variance in sigma directly into compounded wealth. In his abstract, width stability can matter more than local sharpness when an interval determines a position instead of describing a forecast. Config B adds one overlay: a book-level multiplier based on the trailing 21-day pooled rate of lower-interval breaks. It is clipped between 0.25 and 1, and beta equal to 1 leaves no free sensitivity parameter.
The frozen Kaggle snapshot covers 2006-05 through 2024-09-20. Training stops at 2015-12-31. Development spans 2016-2021 and 1,511 days; everything from 2022 onward was sealed before a single evaluation. Config A compounds at 28.45% in development, with Sharpe 1.336 and a 27.68% maximum drawdown. Config B records 25.84%, Sharpe 1.386 and drawdown 20.26%. Realised coverage reaches 0.7483 against 0.7500 nominal.
Results change sharply on the sealed period from 2022-01-01 to 2024-09-20. Coverage is 0.7450 across 5,192 cells. Config A growth drops to 8.47%, while Config B reaches 7.01%; their Sharpes are 0.453 and 0.422. Config A clears only the harness-capped 0.75x SPY bar at +6.30% and trails ungeared equal weight at +9.50%. Both configurations finish last among the eleven lockbox entries on Sharpe and Calmar. The registration's secondary variant trades the final 21 rows of the sample and earns +11.47% at Sharpe 0.565, ahead of the three unlevered bars. Registration rules prevent its promotion to primary. Ryan describes the 3.0-point difference as mechanical because the two books diverge solely over those last 21 days.
Coverage survived; the portfolio did not.
The cap runs the book
For Config A, pre-cap gross leverage averages 4.28 and breaches the 2.0 ceiling on 97.7% of development days. Ryan highlights the implication himself. Kappa and the z constant become inert, inverse-volatility tilting must lose, correlation is empirically irrelevant, and constant de-levering changes nothing. A constant-leverage control matched to realised gross of 1.83 leaves maximum drawdown at 0.2768, exactly where Config A stood. Its Sharpe is 1.273, compared with 1.386 for Config B.
The experiment therefore amounts to a capped inverse-scale-squared cross-sectional tilt held at maximum gross on nearly every day. Four equity ETFs provide +0.237 of Config A's +0.3120 total arithmetic contribution. Average equity exposure is +1.23 within a +1.57 net book. The paper says it plainly: "This strategy is substantially a long-equity book."
The per-asset clip carries more weight than its place in the design suggests. At matched gross, a sweep produces 0.2322 at c=0.26, 0.2809 at 0.75, 0.2819 at 0.90 and 0.2477 at 2.0. The harness sets the value exogenously at 0.75. Ryan attributes the outcome to luck rather than design. Water-filling, the apparently more principled alternative that preserves exact Kelly ratios among unpinned assets, cuts growth by 4.0 points and raises drawdown from 0.281 to 0.371. Cross-sectional performance depends heavily on the clip, and the paper treats that dependence as load-bearing.
Did the slow estimator really win?
This is the tradable part of the paper. It also has the thinnest out-of-sample backing. With leverage matched exactly, at post-cap gross 1.957 to 1.959, the conformal quantile returns 0.2774 against 0.2566 for a rolling residual standard deviation. The spread is 2.1 points a year. Mean absolute deviation falls between them at 0.2713. A sigma fixed on pre-2016 residuals and left untouched reaches 0.2669, still ahead of the rolling standard deviation at 0.2476.
Every locally adaptive version Ryan tested performs worse here. Volatility-scaled scores lose 3.3 points at h=21. Adaptive Conformal Inference loses 1.6, recency weighting loses 1.4, and Mondrian within asset class loses 2.4. Ryan lists EnbPI, the ensemble-bootstrap construction for dependent series, as related and untested.
Ryan acknowledges the objection I would press. The abstract presents the design rule as a development-window result. Section 5.1 says the grids were searched and the winning entries were selected values, rather than pre-specified hypotheses. In his own wording, confirmatory weight belongs only to the lockbox.
His answer is replication within development. Independent ablation blocks produce the same estimator ordering twice: leg-2 gives 0.2774, 0.2713 and 0.2566, while leg-3 gives 0.2821, 0.2693 and 0.2476. A gross-matched replay reduces the leg-3 conformal-versus-standard-deviation spread from 3.5 points to roughly 2.2, without changing the ranking. Yet both blocks evaluate the same 1,511 days from the same eight ETFs. Their return paths overlap almost completely, and the search had already entered that region. Two paired contrasts within one window make a coding accident less plausible. They offer no evidence from another regime.
Estimation variance in sigma is Ryan's proposed mechanism. He measures it directly in one setting only, using the holding-period residual. At H=21, the standard deviation of the daily log-change in q is 0.00343, against 0.00391 at H=5. For the other methods, he explicitly describes the mechanism as conjectured rather than measured. Roughly 200 configurations were evaluated on the same 1,511-day window.
The design rule remains where it began. Pre-registration froze two configurations and nine comparison bars for the sealed test. Its second commitment barred sensitivity reruns after unsealing, and the paper contains no lockbox rerun of the sigma-source comparison. The 2.1 points belongs to development. Once sealed observations arrived, the economic result from that window fell to about 30% of its former value.
One disclosed detail makes the constraint sharper. The frozen configurations use z = 1.2816, inherited from alpha = 0.20, although alpha = 0.25 calls for 1.1503. The difference is equivalent to a 24% change in kappa. Commitment 2 of the registration blocks a post-unsealing sensitivity rerun. Ryan documents the mismatch and leaves it in place because a correction would alter the frozen commits.
Our run used different data
Our figures need their own boundary. The published experiment starts in 2006 and relies on a frozen Kaggle price snapshot. We used our own US ETF daily price history, which generally begins around 2010, so exact numerical replication was never possible. Trading the disclosed rule does not require the agent search or lockbox protocol. We implemented the fixed configuration as published, including the Config B downside dial.
Our backtest runs from 2020-01-01 to 2025-10-08 on the same eight ETFs and contains 11,446 trades. Annual compounding is 6.68%, with Sharpe 0.41, annualised volatility 19.19%, maximum drawdown 40.4% and Calmar 0.17. We added $0.004 a share to the paper's 5 bps turnover charge, subject to a $1 minimum and a 1% cap. Comparisons require care because the windows differ. Ryan reports 25.84% growth and Sharpe 1.386 for Config B over the tuned 2016-2021 development period. His sealed 2022-2024 lockbox produces 7.01% and Sharpe 0.422. Our 6.68% and 0.41 are four times below the development result and essentially level with the lockbox result.
Timing accounts for much of that proximity. Our sample shares two years with his development period. The rest consists of his sealed window plus 2025. The outcome is a rough echo of his out-of-sample record.
Two further differences in our run push in the same direction: the additional per-share commission on a portfolio turning over about 15 times a year, and warmup. The rule waits 750 days before making its first prediction. Each label then needs 21 more days to land, followed by 500 landed scores. Fully forming the scale therefore requires about 1,271 trading days of history. Our traded period begins on 2020-01-01. Early-sample noise in sigma is precisely the failure mode identified by the paper's ablations.
Our maximum drawdown of 40.4% exceeds the 31.7% Ryan reports for Config B in the lockbox. Our Calmar is 0.17, versus his 0.303 there and 1.376 during development. Volatility in our run is lower at 19.19%, against his 22.8%, which rules out gearing as the explanation for the deeper drawdown. Among the two candidates, warmup and scale formation appear likelier, alongside the extra per-share cost. We also cannot exclude a position-cap discrepancy. Our run may have limited each position to 12.5% of the book, while the paper applies 75% winsorisation. Ryan's clip sweep begins at c=0.26, where it earns 0.2322 against 0.2809 at 0.75. A cap that restrictive could remove several points a year.
The claim left standing
Calibration transferred without an economic payoff. Lockbox coverage of 0.7450 against 0.7500 is the most precisely measured sealed result. Ryan estimates an effective sample of only order 65 to 260 after accounting for overlap and cross-asset correlation, implying a standard error around 0.03 to 0.05. The competing portfolios did better. Equal weight at 2x returned +16.79%, inverse-vol risk parity at 2x returned +15.76%, and ungeared equal weight delivered +9.50%. Each beat Config A at +8.47%. Applying a 4% charge to gross above 1.0 lowers Config A's lockbox growth to about +4.7% annually, beneath every unlevered bar.
The dial reduced out-of-sample drawdown from 36.6% to 31.7%. Sharpe also declined, from 0.453 to 0.422, so the pre-declared "Sharpe no worse" condition failed. Its development placebo had already found that 31 of 40 circular shifts beat realised growth. The Ulcer index was 0.0698, compared with a placebo mean of 0.0622.
Ryan calls the configured strategy uninvestable and locates the contribution in the calibration mechanism. I agree, then go further. The paper's interesting claim is that Kelly sizing should use a slow, unweighted, per-asset quantile in its denominator. All supporting evidence comes from paired comparisons inside the window optimised by a 200-configuration search. I would change my mind after seeing the same 2.1-point contrast on a second universe where the gross cap does not bind on 98% of days. Ryan identifies exactly that experiment as his highest-value follow-up and says his immutable harness prevented him from running it. Naming the right experiment does not supply its result. Until somebody runs it, the design rule rests on one window.