A RankIC of 0.061 against next-day alpha cannot carry a 264.9% cumulative return by itself.
The remaining weight falls on two features of the test. The paper includes no transaction, slippage or borrow cost for a book that turns over completely every day. Its entire result also comes from one out-of-sample window, January 2019 to June 2021. Anyone rebuilding the strategy needs to establish which feature drives the equity curve.
First, a disclosure. We could not run the Llama-3-8B GRPO fine-tune. Our news and transcript history begins around 2020, so FinSMART itself was absent from our run; we traded a substitute sentiment score. That history also prevented us from testing the six-month retraining schedule.
Where FinSMART is meant to earn its return
FinSMART takes Llama-3-8B-Instruct and post-trains it with GRPO (Group Relative Policy Optimization). Realized stock returns supply the reward in place of human sentiment labels, an approach the authors call market-aligned reinforcement learning. The portfolio rule is clear. Each day, the sentiment score ranks every S&P 500 company with news. The top 35% enter the long side, the bottom 35% enter the short side, and positions remain open for one day. In the paper's text, the measured edge is an alpha spread of about 5.0% between positive and negative Motley Fool articles on publication day. At a one-day lag, it falls to 0.3%.
The economic argument is straightforward. Existing financial sentiment models learn from static human-labelled datasets. As the authors write, FinDPO "remains dependent on static labelled datasets". Human judgments about tone differ from judgments about whether an article implies outperformance. Training directly on the latter should produce a ranking with more cross-sectional information.
The fine-tune itself is compact: LoRA rank 16, alpha 32, dropout 0.05, and 13.6M trainable parameters, equal to 0.17% of the base. For every article, G = 8 completions are sampled. A KL penalty beta = 0.1 anchors the policy to the frozen reference model. The run requires eight hours on one A6000.
Two filters determine whether an article reaches the reward. BERT-base-NER first has to identify the target company with confidence above 98%. The frozen reference model must then assign its highest next-token probability to Positive, Negative or Neutral. Articles failing either gate are dropped.
The reward is a discrete dual filter. Alpha means the stock's publication-day return minus SPY, while r is the raw return. A ground-truth label of +1 requires alpha > 0.5% and r > 0. A minus one label requires alpha < -0.5% and r < 0. Every other observation receives zero. Correct direction earns +2.0, correct neutral earns +0.1, wrong direction receives -1.5, and all remaining outcomes receive -1.0. This deliberate asymmetry is intended to prevent an all-neutral policy collapse.
The data come from The Motley Fool and MarketWatch between February 2015 and June 2021. S&P 500 constituents are sourced from Yahoo Finance, with 1,672 trading days per name. Training ends at 31 December 2018 using roughly 30,000 articles. The authors hold out 5% and evaluate every 500 steps. The out-of-sample period runs from January 2019 to June 2021 and contains about 325,000 articles. FinDPO's logit-to-score converter turns discrete labels into continuous scores. Scores are averaged by company-day, then used for an equal-weighted top 35% long and bottom 35% short portfolio. Trades enter at the next open and exit at the following open.
The reported result is 264.9% cumulative and 91.5% annualized, with Sharpe 1.97, Sortino 2.40, Calmar 4.23 and RankIC +0.061. FinDPO uses the same base model, converter and 35/35 portfolio rule. It produces 109.8%, 45.0%, Sharpe 1.12 and RankIC +0.053. Over the same window, the S&P 500 returns 69.3% with Sharpe 1.09. Every lexicon loses money: LMD (Loughran-McDonald) returns -31.6%, VADER -38.8% and HIV-4 -13.0%.
What can an implementer reproduce?
The paper specifies the reward and sampling parameters: tau = 0.5%, reward values of +2.0 / +0.1 / -1.5 / -1.0, G = 8, beta = 0.1, and LoRA r = 16 with alpha = 32 and dropout 0.05. It omits the learning rate, batch size, epoch or total step count, LoRA target modules and prompt text.
FinDPO and FinSMART share a base model, converter and portfolio construction rule, which makes their comparison cleaner than most sentiment studies. Yet FinDPO learned from a different static human-labelled corpus and received no subsequent market-feedback training. Training-data recency may therefore explain some of the difference between 264.9% and 109.8%, apart from the GRPO objective.
More discretion enters on the evaluation side. Four choices were left to me:
- Under equation 8, cumulative return is the sum of daily returns, making 264.9% an arithmetic total. Annualized return equals 252 times mean daily log return. Compounding the same daily series produces a different figure.
- I infer gross exposure from the stated return definition, long average minus short average, as 100 long against 100 short with no financing cost. The risk-free rate is zero. The paper gives no gross exposure or leverage convention.
- The eligible universe is rebuilt from companies with news each day. Consequently, the 35% threshold means 35% of whatever number of names printed news that day. I could not locate the daily ranked-name count.
- The paper does not discuss intraday publication timestamps. Assigning every article to its correct publication day carries the entire next-day protocol.
The abstract and results section also give different improvement figures. The abstract claims 220% over the strongest baseline. The results section reports 141%, which agrees with the table's comparison of 264.9% against 109.8%.
No transaction cost, slippage, borrow or short-availability assumption appears anywhere in the paper. Full daily turnover makes that omission unusually consequential.
Today's move becomes tomorrow's signal
Publication-day returns define the reward. The authors justify that choice with a sharp decay in the measured relationship. On Motley Fool, the average alpha spread between positive and negative articles is about 5.0% on publication day and 0.3% after shifting returns one day. MarketWatch falls from 2.3% to 0.3%. Pearson correlation between reference sentiment and alpha declines from 0.41 to 0.03 on Motley Fool, and from 0.37 to 0.03 on MarketWatch.
The paper acknowledges the tension directly: "While this uses contemporaneous market information to construct the reward, our objective is to maximize the quality of the supervisory signal rather than simulate a trading strategy during training." It then states: "All reported trading results are evaluated using next-day returns to eliminate look-ahead bias."
The trading protocol follows that statement. Sentiment measured on day t determines the portfolio entered at the open of t+1.
A harder question remains. The language features that explain the same-day move must retain predictive content for the following day. Training uses a 0.41 correlation while trading seeks to monetize a 0.03 correlation, so the representation has to transfer across that decay. RankIC of 0.061 against next-day alpha shows some transfer. But it exceeds FinDPO's 0.053 by only eight thousandths, while cumulative return improves by 141%. Signal quality alone does not seem enough to explain that amplification. Dispersion during 2019-2021 offers a plausible additional source on a daily-rebalanced long-short portfolio with no costs, especially because the paper does not state gross exposure.
I found no ablation separating the NER gate, sentiment gate and asymmetric reward. The reported gain therefore cannot be assigned specifically to GRPO rather than the filters ahead of it. Those filters may prove to be the more portable contribution.
We lost 48% where they made 265%
Our environment could not reproduce the GRPO fine-tune of Llama-3-8B. Our text archive also begins around 2020. We therefore implemented the paper's portfolio rules using a lighter sentiment score, possibly the fallback P(Positive) minus P(Negative) rather than the FinDPO converter. FMP news, press releases and earnings transcripts supplied the corpus. We used the top 500 US companies by market capitalization, applied a $25mm dollar-volume floor, and excluded names whose 20-day realized volatility exceeded 10% per day. The test covers January 2020 to October 2025, with a 35/35 long-short portfolio traded next-day open-to-open. We charged $0.004 a share with a $1 minimum per order.
The run lost 48.21%. Sharpe was -0.46 and maximum drawdown reached -75.19% across 90,506 trades. Sortino came to -0.62 and Calmar to -0.14, versus the paper's 2.40 and 4.23. The paper reports 264.9% with Sharpe 1.97 from January 2019 to June 2021. Those figures are gross of costs and use the actual GRPO-trained policy on Motley Fool and MarketWatch text.
The two tests measure different things. They use a different model, corpus and universe, with barely overlapping dates. Our run also charges costs to a portfolio that turns over daily.
Several differences naturally push our result downward. Press releases and transcripts are written by issuers and skew positive, compressing the cross-sectional rank that drives the trade. In the paper's own diagnostic, the pre-trained reference model's sentiment has a 0.41 correlation with publication-day alpha on Motley Fool and 0.03 with next-day alpha. Changing the corpus could plausibly erase the tradable component altogether.
Our window includes four years absent from the paper's test. The volatility gate probably removes high-dispersion, news-reactive stocks that generate much of the spread. It also means our run provides no test of the COVID-era behavior highlighted by the paper. Finally, a policy trained through a 2018 cutoff remained in use through 2025, well outside the 0 to 30 month staleness range examined in the paper.
Those differences can explain a dead signal. They do not fully account for a profit factor of 0.98 alongside a 51.01% win rate, the coin-flip profile expected from no signal, or a 75.19% drawdown against 18.13% annualized volatility. The visible evidence cannot close that residual. Our implementation is the first suspect, especially entity linking and publication-date fidelity for press releases and transcripts. This run is therefore one imperfect substitute test, rather than a verdict on the authors' FinSMART results.
Retraining deserves the first implementation effort
An expanding-window retrain every six months, repeated for four iterations, raises cumulative return from 264.9% to 406.2%. Sharpe climbs from 1.97 to 2.41, while RankIC moves from 0.061 to 0.065. Across roughly four six-month periods, the correlation between articles added during each period and the gain over the static model is r = 0.72. With so few observations, that figure indicates direction more than measurement.
Annotation-free updating is the paper's genuinely useful idea. Each retrain takes eight hours on one 48GB A6000. The paper completes four of them under identical settings, with no labelers involved. Because of where our text archive begins, we never tested that schedule ourselves.
A full turnover and cost accounting for the 35/35 daily portfolio would change my view of the headline. So would one more out-of-sample window that excludes the COVID crash.