AQAI QuantAI research lab for systematic strategies

Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysisOur backtest

One MLP coordinate shifts the buy-sell prior in five LLMs

The prior moves, the calibrated delta stays task-bound, and the backtest spans 28 weeks

2026-09-08 · 9 min read · US equities

Reviewing: Your AI, On a Dial: Controlling Investment Bias in LLMs with a Single Neuron · Sahong Park, Suhwan Park, Hoyoung Lee et al. · Read it on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

Single-Neuron-Steered Qwen3 News Ranking for Liquid US Stocks

Backtest period 2020-01-01 to 2024-07-01 · hypothetical, net of modelled costs

Why these figures are not the paper's (2)

This is not a replication of the paper (2)

  • The stated data catalog includes financial text and embeddings but does not establish provisioned open-weight LLM inference infrastructure, GPU capacity, or model-specific activation-hook support. The exact result requires running the authors' selected open-weight architectures with a custom pre-hook on a particular MLP down-projection coordinate; a generic prompt-based bullish/bearish instruction would be a substitute for the control objective, not a replication of the single-neuron method.
  • The selected neuron and calibrated intervention strengths are architecture-, checkpoint-, prompt-, and evidence-distribution-specific, so they must be rediscovered and recalibrated on a held-out ticker set. Any portfolio backtest would test the newly calibrated implementation rather than transferring the paper's dial parameters or reported portfolio results.

The figures below measure what we could run, not the paper's own method, so they are not evidence for or against its claim.

Our own audit found this run does not follow the paper faithfully (9)

  • Paper portfolio evidence used shared market and sector headlines rather than stock-specific news (invalidates: The paper's portfolio ranking, saturation, tie-frequency, Sharpe, and random-portfolio percentile results do not directly transfer; the paper's task-specific Delta values do not apply.)
  • Paper entered at Friday close (invalidates: The paper's one-week-ahead return series, turnover timing, realized Sharpe ratios, and random-portfolio percentile comparisons do not directly apply.)
  • Paper backtest omitted transaction costs, slippage, and execution delays (invalidates: The paper's reported Sharpe ratios cannot be interpreted as net performance or directly compared with this strategy's net results.)
  • Paper portfolio universe was 427 S&P 500 stocks (invalidates: The paper's exact portfolio constituents, sector exposures, saturation pattern, turnover, returns, Sharpe ratios, and random-portfolio comparisons do not directly apply.)

5 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total 57.3%Jul 2024
Sharpe
0.52
Total Return
57.3%
Max Drawdown
-40.6%
CAGR
10.6%
Volatility
24.5%
Beta vs SPY
1.07
Trades
38,424

What the paper reports for its own strategy

  • Weekly rebalanced equal-weight long-only top-100 portfolio from Qwen3-8B scores, 427 S&P 500 stocks, 29 signal weeks / 28 return weeks, Aug 4 2025 – Feb 20 2026: at the strongest dial settings the realized Sharpe exceeds the 95th percentile of 500 randomly formed weekly rebalanced 100-stock portfolios over the same return weeks (no transaction costs, slippage or execution delay modeled)
  • Realized Sharpe of the top-100 portfolios is shown only graphically (Figure 5c, axis range roughly 2–4); no point Sharpe, return, alpha, t-stat, hit rate or drawdown is stated in the text
  • Authors state: the backtest is exploratory and does not establish investment alpha

A single-coordinate activation edit can reverse an LLM's aggregate trading stance. Add a constant to one coordinate in one MLP layer and Qwen3-8B moves from a +0.575 buy-versus-sell tendency to -0.010. The prompt, weights and evidence items remain fixed. Park and coauthors produce a version of the result across all five open-weight models tested. Seven benchmarks put the largest capability cost at 1.60 percentage points. After Benjamini-Hochberg correction, zero declines remain significant across 105 paired comparisons. The weakest per-model 95% lower bound is -2.49 pp, leaving a bounded and nonzero bill. We could not run these experiments ourselves because we lack open-weight inference capacity and activation-hook support. Nothing that follows is a replication.

Turning bias into a setting

Earlier work found a bullish lean in LLM decisions when evidence is balanced. This paper uses a 427-name S&P 500 universe. Each ticker receives equal numbers of matched synthetic bullish and bearish items, followed by a forced binary buy/sell choice. Qwen3-8B buys much more often than it sells. Measured as net buys over total decisions, its score is +0.575 on a scale from -1 to +1. Llama-4-Scout reaches +0.623. DeepSeek-R1-Distill-Qwen-14B is nearer flat at +0.095, while Gemma-4-12B records -0.000 and Mistral-Small-24B -0.029.

The authors turn this lean into an adjustable setting. A pre-hook adds a single scalar to one coordinate of an MLP down-projection at every token position. The dial is one number, fitted until the aggregate prior reaches a selected target. It can push the model bearish, neutral or bullish.

The prompt and weights never change.

Neutrality has a narrow operational definition. Under the balanced synthetic-evidence protocol, aggregate buys and sells must match when summed across the entire universe. Gemma-4-12B and Mistral-24B already met that condition. Their calibration produced Δ*=0 and altered 0 (0.0%) outputs, making the method a literal no-op for those two.

The evaluation covers five open-weight models and seven zero-shot capability benchmarks. It also includes an earnings-call retrieval agent using the most recent quarterly call for 417 firms from the 427-ticker universe, plus a long-context test extending to 16,025 tokens. The trading exercise is a weekly rebalanced top-100 portfolio drawn from 427 S&P 500 names between 4 August 2025 and 20 February 2026.

A trader will go there first. The proposed mechanism is tie-breaking. Stronger dial settings pull scores away from the upper boundary, reducing saturation and distributing names across more score levels. At the strongest settings, the weekly rebalanced top-100 book posts a realized Sharpe above the 95th percentile of 500 randomly assembled weekly rebalanced 100-stock portfolios over the same return weeks.

How the coordinate is chosen

Selection has two stages. A gradient sensitivity screen first measures the buy-minus-sell logit gap and reduces the full collection of MLP coordinates to a manageable candidate set. The authors then divide the 427-name universe into two disjoint subsets. They fit the coefficient needed to reach each target prior on the first subset, apply it unchanged to the second, and rank candidates by RMSE across the target grid.

Qwen3-8B selects layer 23, feature 4099. It has RMSE 0.038, a reachable span of 2.000, a valid-decision rate of 0.988 and a parse rate of 1.000. The runner-up comes in at 0.048. Across the networks, the selected coordinates appear in mid-to-late layers.

Footprint is the dial's edge over dense steering baselines. It changes 1 activation per token. Contrastive Activation Addition changes 3,840 to 5,120, while Representation Engineering changes 20,480 to 40,960. All three methods move monotonically and cross zero in 5 of 5 models.

Dense methods can cover a wider range. DeepSeek-R1-14B reaches a span of 1.433 with the dial and 2.000 with both CAA and RepE. Capability preservation favors the dial: it produces 0 of 105 significant losses, compared with 4 for CAA and 2 for RepE.

Does neutrality preserve judgment?

This is the result that matters most. An aggregate score can be driven to zero by making a model insensitive to evidence. The paper tests that possibility directly by fitting a tanh to responses across graded bullish-to-bearish evidence ratios. R-squared is at or above.9987.

For Qwen3-8B, the zero-crossing shifts from -0.346 to +0.019, while the slope moves from 1.806 to 1.978. Llama-4-Scout moves its crossing from -0.317 to +0.036 and its slope from 2.083 to 1.994. The boundary shifts while the slope largely holds. This is the cleanest result in the paper, and the one I would cite.

Aggregate neutrality conceals substantial churn. Qwen3-8B flips 2,527 decisions (29.6%) during calibration. Llama-4-Scout flips 2,773 (32.5%). DeepSeek-R1-14B begins at +0.095 and ends at +0.010, changing the score by 0.085. Reaching that setting changes 1,806 decisions (21.1%). The paper gives JPMorgan and NVIDIA as examples: both switch from buy to sell at the calibrated setting, fully consistent with an aggregate score of zero. Ticker-level opinions can move sharply beneath a neutral book-level prior.

The prompt baselines make the comparison clearer. Bias-disclosure prompting leaves Qwen3-8B at +0.492 and sends Llama-4-Scout past neutral to -0.546. DeepSeek rises to +0.673, Gemma to +0.856 and Mistral to +1.000, with the latter two starting near zero. The dial instead produces -0.010, -0.026, +0.010, -0.000 and -0.029. According to the authors, those values are closest to zero for four of five models. Ticker anonymization wins on Llama-4-Scout, reaching +0.003 versus -0.026 for the dial.

Missing details in the recipe

The central implementation details are specific. The paper gives the intervention equation, the score, each model's coordinate, the calibrated coefficient, the seed and the compute environment down to the CUDA build. For Qwen3-8B it publishes layer 23 feature 4099 and Δ*=-11.44, with greedy decoding and seed 42. A researcher with the weights and a suitable pre-hook should be able to reproduce the elicitation.

Other choices remain unspecified. The gradient screen includes neither a threshold nor a count of surviving candidates. The feasibility filter used during candidate selection has no published cutoff. A 90% parse-rate threshold appears only in the appendix's reachable-range comparison. Benchmark calibration targets are listed as π ∈ {-0.3, 0, +0.3}, while the target grid T used for RMSE ranking is left unstated.

We also did not find the sizes of the two universe subsets or the inversion routine that maps a fitted response curve to the coefficient for a chosen target. Trials per ticker can be inferred from the reported counts, since 2,527 changes at 29.6% implies roughly 20 decisions per name, though the paper does not state it where we looked. The method survives these omissions. Two honest implementations could still separate at precisely these points.

The delta stays task-bound

The base protocol gives Qwen3-8B a calibrated value of -11.44. Its agentic retrieval loop requires -34.42, and the long-context experiment requires -42.34. The paper is explicit on both limits: the calibrated coefficient is not task-invariant, and Δ units cannot be compared across models. Its phrasing stops short of a universal rule, saying that a new task may require recalibration or re-identification of the relevant coordinate. Both downstream experiments in the paper use recalibration.

Recalibration is the real operating cost.

In return, the control reaches upstream into the agent's search process. The Qwen3-8B retrieval agent covers the most recent quarterly earnings call for 417 companies in the 427-ticker universe. At the task-calibrated setting, the bearish share of generated search queries rises by 88%. The selected evidence becomes 41% more bearish, and memo citations become 74% more bearish. The intervention therefore changes what the agent opens before evidence scoring begins.

The long-context result travels farther. A dial calibrated to match a bearish system prompt at roughly 550 tokens maintains that stance through 16,025 tokens and shows lower cross-document dispersion. The system prompt weakens over distance. The paper presents this attenuation graphically and supplies no stance-score point estimates.

Our run, with its limits

We lack provisioned open-weight LLM inference, GPU capacity and model-specific activation-hook support. Testing the paper's result requires its architectures, a custom pre-hook and a named coordinate in an MLP down-projection. A bullish or bearish prompt uses a different mechanism and can only substitute for the control objective. Coordinates and strengths also depend on the architecture, checkpoint, prompt and evidence distribution. They must be found again and calibrated on held-out names. Any portfolio we run is therefore our own newly calibrated implementation.

We built a weekly, long-only, equal-weight top-100 portfolio from 427 liquid non-ADR US listings, with annual dollar-volume rescreening. Names were ranked using four counterbalanced buy/sell decisions per ticker, based on company-specific news and press releases available before each Friday cutoff. Execution used the next observed open. The sample runs from 2020-01-01 to 2024-07-01.

Costs were four tenths of a cent per share in commissions and 5bp for each unit of one-way turnover, charged across 38,424 trades. After those costs, total return was 57.28%. The Sharpe was 0.52, annualized volatility 24.45%, maximum drawdown -40.58% and beta 1.07. These are our figures from our run and our window.

The run's own output did not let us verify which scorer generated the rankings. If the steered model was unavailable, a plain text scorer supplied them. Under that condition, the traded rankings were not LLM stance scores at all.

The paper displays its weekly rebalanced top-100 portfolio Sharpe graphically on an axis running roughly 2 to 4. Our figure is 0.52. These results measure different setups. The paper has 28 one-week return observations from 4 August 2025 through 20 February 2026. Its portfolio is gross of costs, slippage and execution delay. Rankings use one market headline and one sector headline per business day, with no ticker-specific news.

Our sample spans 4.5 years, including the February 2020 crash and the 2022 bear market. It uses another information set and reports returns net of costs. Beta explains most of the gap. A beta-1.07 long-only equity portfolio can record a high Sharpe during a short rising market. Across 4.5 years, the figure moves toward the market's long-run number. The resulting compression appears as 57.28% total return, a -40.58% drawdown and 24.45% volatility.

Our pass evaluates our implementation during our window. It provides no evidence on whether the dial works and should not be read as a verdict on the authors' result.

Tie-breaking carries the portfolio claim

The portfolio analysis is the paper's weakest section. The authors acknowledge this twice, at the end of the results and again in the conclusion, using nearly identical language: "The backtest remains exploratory and does not establish investment alpha." The abstract describes it more quietly as "an exploratory backtest". The limitations also identify the short evaluation period and the omission of transaction costs, slippage and execution delays. Those objections are already conceded. The remaining question is how much evidence the exercise contains.

Figure 5c is offered as the dial's portfolio payoff. Its Sharpe axis runs from 2 to 4, without a point estimate, t-stat or confidence band. The paper's only quantified portfolio result is therefore graphical and covers 28 return weeks.

The narrower propagation claim holds up. Dial changes reach security rankings and alter portfolio composition. The authors attribute the Sharpe movement to greater score dispersion. Moving values away from the upper boundary creates fewer ties and distributes names across more score levels, reducing reliance on random tie-breaking. Results are averaged over Twenty tie-breaking permutations. Because the inputs contain market and sector headlines only, cross-sectional variation comes from the model's internal prior and the tie structure. In this exercise, the dial serves as a rank-dispersion tool.

Its present use is governance. The dial imposes a declared aggregate stance on a model that already carries one, then maintains that stance at 16k tokens after a system prompt begins to fade.

Portability needs one further result. Calibrate on one evidence vintage, then keep the score near zero on a later vintage without recalibration. Until such a test appears, every change in task, checkpoint or inference stack should carry a recalibration budget.

Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

PRE-BACKTEST CALIBRATION
1. Use only news and press releases published before 2019-12-31 23:59:59 ET.
2. Split the 427 calibration securities deterministically into disjoint S_A and S_B sets.
3. Screen MLP coordinates by buy-versus-sell gradient sensitivity.
4. Estimate each candidate's response curve on S_A for targets {-0.3, 0, +0.3}.
5. Apply inferred deltas unchanged to S_B; select the single feasible coordinate with lowest held-out target RMSE.
6. Recalibrate that coordinate on S_A union S_B, then freeze the model, tokenizer, prompt, coordinate, and deltas.

WEEKLY SIGNAL
1. Resolve the year's top 427 non-ADR US-listed stocks by dollar volume.
2. At Friday 16:00 ET, or the final regular-session close before Friday, freeze each ticker's eligible evidence bundle. Require an unambiguous original publication timestamp strictly before the cutoff.
3. Present evidence chronologically and identically across dial settings and trials.
4. Run four deterministic, option-order-counterbalanced trials per ticker. Apply the selected-coordinate delta at every prompt and generated token; require JSON containing one buy/sell decision and a reason.
5. Compute score pi_s = (N_buy - N_sell) / (N_buy + N_sell) when at least one valid decision exists.
6. Rank scores descending. Within tied groups, create 20 deterministically seeded permutations for sensitivity reporting; the executable portfolio uses the configured dial variant and first deterministic permutation.
7. Select up to 100 stocks and target equal weights of 1% each, subject to a 10% position cap and 4.0 maximum gross leverage.
8. Execute at the next observed trading-day open. Skip names lacking a real open and renormalize across executable names; never synthesize a fill.
9. Hold until the next weekly portfolio can be executed, then replace it.
10. Charge the specified commissions and turnover-based cost model and report gross and net returns separately.