A Rate Dashboard Without Published Forecast Errors

A twelve-month rate forecast has little value unless its usual error is known. This paper gives no such figure.

What reaches the ALM desk

Bauer, Holländer, Wolff, Ostermair, Aiad and Hasebrook target asset and liability management (ALM) inside a bank. Duration and curve positioning on the balance sheet are the decisions at the end of the process. A monthly Bayesian VAR supplies forecasts for 2, 5, 10 and 30 year euro swap rates. Text-derived topic and sentiment signals appear alongside them on one screen. The commercial case rests on better curve calls, better hedges and better stress inputs.

The authors built a Python/Dash dashboard around an R BVAR. It has four modules and was tested as a proof of concept within one major European bank. There is no external replication, and the data file is not public.

The first module applies SBERT embeddings, UMAP for dimensionality reduction and HDBScan to cluster text. A large language model, accessed through an OpenAI-API-compatible provider, writes topic labels and document summaries. The paper identifies analyst reports from financial institutions, ECB and Federal Reserve publications, and financial press articles as its sources. A few-shot FinBERT-style transformer powers the second module. It assigns three separate document-level scores covering inflation, economic activity and employment across ECB accounts and decisions, plus FOMC minutes and statements. The econometric core forms the third module. The fourth combines Bloomberg forward curves, internal bank estimates and consensus government bond forecasts grouped by expected curve shape.

The monthly dataset covers euro area and US macro-financial series from December 1986. Estimation begins in January 1999, when the swap data starts. Appendix 1 names roughly 55 indicators drawn from Refinitiv/LSEG, Bloomberg, ECB, EuroStat, the US EIA and the Dallas Fed. The sentiment series spans 09/2014 to 09/2024, and the paper works through an example containing 40 documents. The final full-period BVAR run carries the label 01/1999 to 11/2024.

The published evidence is sparse. Readers get Table 1's forecast comparison as of 02.12.2024 and four probability-weighted scenarios in Tables 2a and 2b. They get no accuracy statistics of any kind.

Why we did not backtest it

We did not backtest this. The target is a European bank's ALM interest rate exposure, while the data available to us contains no euro-area bank balance-sheet exposures, no internal ALM or regulatory risk measures, and no swap or Euribor/€STR curves. Available news-text tables and embeddings could only approximate the central bank and news text, rather than reproduce the prototype's document corpus.

An adaptation would have used a rate forecast to trade US Treasury duration ETFs (TLT, IEF, SHY). Those are different instruments on a different curve and in a different currency. No result from us would test the paper's claims, so we are not producing one.

Inside the BVAR

Seven predictors remain after screening more than forty candidates: euro area inflation, industrial production, the ECB refinancing rate, import-weighted commodity prices, the EuroStoxx 50, US inflation and the US manufacturing PMI. The model uses Five lags. It runs 50,000 MCMC draws after a 25,000 burn-in. Its Minnesota prior has lambda mode 0.2 and standard deviation 0.4, supplemented by sum-of-coefficients and unit-root priors under automatic hierarchical selection.

In the final full-sample script, the ECB refi rate appears fourth among eleven variables. It precedes the PMI, commodity and equity blocks. Code comments offer three alternative orderings without selecting one. Users feed shocks into the scenario engine, which returns conditional forecasts with lower, mid and upper levels for each tenor.

Table 1 contains the paper's headline figures, a forecast comparison as of 02.12.2024 at the twelve-month horizon. The BVAR mid rate is 1.94% at 2Y, 2.00% at 5Y, 2.13% at 10Y and 1.92% at 30Y. On the same date, the Bloomberg forward curve shows 1.86, 1.97, 2.09, 1.87. Table 1 labels another row ISP, described in the text as an internal bank estimate. That row reads 1.99, 2.07 at 3Y, 2.23 at 5Y, then 2.81 at 10Y and 2.59 at 30Y. Spot on 29.11.2024 was 2.11, 2.07, 2.16, 1.96.

Almost the forward curve

The comparison deserves another look. At 10Y, the BVAR gives 2.13 against 2.09 from the forward curve, a difference of four basis points. At 30Y, its 1.92 stands five basis points from 1.87. The internal forecast is 68 basis points above the model at 10Y and 67 above it at 30Y. After the bank's development effort, the model lands nearly on market-implied pricing. Its existing internal forecast is the outlier.

The paper leaves that result untouched.

Section 3.3.4 discusses disagreement among forecast sources in general terms. It never addresses the BVAR mid rate lying within four basis points of the forward curve at 10Y and five at 30Y. With gaps that small, the forwards could have been read from a screen.

The scenario module makes the stronger case for the model. Scenario 2 fixes inflation at minus 0.25%, industrial production at minus 2.75%, the ECB refi at minus 0.13% and the EuroStoxx at minus 5.05%. Under those assumptions, the 2Y swap mid falls from 2.11 to 1.71. The 10Y minus 2Y slope moves from plus 0.04 to plus 0.23. For stress work, a 40 basis point front-end move tied to an industrial production shock of minus 2.75% and an equity shock of minus 5.05% is usable.

The missing error table

A rolling backtest sits in the appendix. It generates twelve-month-ahead forecasts across the last 240 monthly observations, roughly twenty years, for 2, 5, 10 and 30 year swaps. Benchmarks consist of a VAR(1), auto.arima at 2Y and 5Y, and a fixed ARIMA(1,1,1) at 10Y and 30Y. The code calculates RMSE for every tenor. Its Diebold-Mariano call remains commented out: # dm.test(error_bvar, arima_error, alternative = "less").

We did not find a single resulting RMSE or MAE in the paper's text or tables. Table 1 reports point forecasts from one date, while Tables 2a and 2b report scenario levels. Neither the model's twelve-month error distribution nor its performance against an ARIMA is shown.

The limitations section speaks more plainly. The prototype "is still a prototype and not yet a production-ready system". Before production, the paper says, "model calibration and robustness testing must first be further expanded, especially the validation of signals derived from sentiment analysis against historical interest rate movements". Section 4.3 says the PoC "is not a deterministic forecasting model but is based on probabilistic estimation methods", and warns against reading the forecasts as exact predictions. Section 4.1 takes a firmer tone, claiming that "the BVAR model ensures statistical accuracy by integrating key macroeconomic indicators". No accuracy figure supports that sentence.

The abstract goes further. According to the paper, the system "enables a more precise and flexible prediction of interest rate developments". It also claims that multi-perspective AI-driven forecasting "provides substantial added value for banks, by increasing transparency, strengthening evidence-based decision-making, and improving risk steering". Transparency and process support those three grounds, and the dashboard plausibly supplies both. Precision can be measured. Yet no RMSE, MAE or Diebold-Mariano statistic appears anywhere in the paper. This review disputes that clause.

The backtest code raises another concern. In the BVAR script, the rolling loop calls predict(mod, ...), although mod was fitted to the entire 1999 to 2024 sample before the loop starts. The VAR benchmark script repeats the problem. Inside the loop it calls VAR(dat, p = 1, ...) on the full data frame instead of train_data. As written, the rolling forecasts estimate parameters using observations that follow the forecast date. Any errors produced by those scripts would be optimistic. The paper publishes none of them, so no reported result depends on the issue. Anyone extending the work should repair the loop first.

Seven predictors, four probabilities, one sample

The Granger screen uses the full 1999 to 2024 sample with max_lag = 1. The BVAR is subsequently estimated on the same observations. Feature selection and estimation therefore share every observation.

The regime subsamples leave far less room. The increasing-yield window in the Granger screen, 01/2021 to 06/2024, contains 42 monthly observations. Its corresponding BVAR script starts at 01/2021 without an end date, giving roughly 42 to 47 months. It fits seven variables with five lags, amounting to 35 coefficients per equation plus a constant. The prior carries nearly all of the model.

The authors acknowledge the broader difficulty. Shrinkage priors, they write, "mitigate overfitting risk but can restrict model flexibility when confronted with structural changes". They also state that the scenario engine "relies on prior distributions that introduce subjectivity into forecast results".

Scenario probabilities are 50%, 15%, 20% and 15%, based on cluster frequencies from Bloomberg analyst forecasts. They work as a stress-test weighting convention. The probability-weighted aggregate curve should not be read as a distribution.

Text beside the forecast

The sentiment output is genuinely interesting as a monitoring series. It covers 09/2014 to 09/2024 on a scale running roughly from minus 8 to plus 10. In 04/2020, economic-activity sentiment reaches minus 8 in ECB accounts and minus 7 in FOMC minutes. Inflation sentiment drops to minus 7 in 2022. The paper further reports that, within the prototype, longer central bank reports produced stronger signals for predicting rate decisions than immediate statements did.

Inflation sentiment follows a deliberately non-monotonic design. Excessively high and excessively low inflation both receive negative scores. This fits a central bank reaction function and prevents the series from serving as a conventional polarity score.

All seven variables in the final BVAR are macro and market series. Sentiment is absent. The text analysis consequently appears beside the forecast on the dashboard and supplies context. Whether it changes the forecast, over which horizon and by how much remains open. The Audrino and Offner work cited by the paper, concerning the effect of macroeconomic news sentiment on interest rates, suggests that it might. This is the missing test I most want to see.

Where it belongs

As a workspace, the design is worth copying. One screen puts the market-implied curve, internal estimate, analyst consensus and conditional BVAR into direct comparison. The 68 basis point difference at 10Y between the model and internal forecast should start a conversation instead of remaining buried in separate spreadsheets. For ALM, the scenario builder also offers a defensible translation from a macro story to a curve shape, complete with directional and slope labels.

As a forecasting model, it belongs in the challenger slot until the error table appears. One figure would change my assessment: the BVAR's twelve-month RMSE against the paper's own benchmarks, auto.arima at 2Y and 5Y and ARIMA(1,1,1) at 10Y and 30Y, across the 240-observation test window already defined in the appendix. The RMSE code exists. The training window must be truncated before the loop runs.