Regime-Based Portfolio Allocation with Hidden Markov Models and Reinforcement Learning

TL;DR

• What it is: a preprint that detects market regimes with a 3-state Gaussian Hidden Markov Model (HMM) and then picks a portfolio of SPY (stocks), TLT (long Treasuries) and GLD (gold) for each regime.
• Two ways to act on a regime: simple rotation rules (hold the best asset per regime) and a small reinforcement-learning (RL) policy that learns which mix to hold.
• Headline result: in the out-of-sample test, the RL policy had the highest Sharpe ratio (0.83, tied with a monthly equal-weight portfolio) and a shallower drawdown than buying and holding SPY (−23.5% vs −35.7%).
• Why it matters for us: it's a clean, transparent template for regime-gated strategy switching. That's exactly what MCI does with bots.

The idea in plain language

Markets don't behave the same way all the time. Calm periods, nervous periods and crashes each have their own "personality": different volatility, different winners and different correlations. The paper's bet is simple:

1. Figure out which personality the market is in right now (the regime).
2. Hold whatever has historically worked best in that regime.

The regime is "hidden" because you can't observe it directly. You only see daily returns, and you infer the regime from how those returns behave. That's what a Hidden Markov Model is built for.

How it works

Step 1: Data

Daily closing prices for SPY, TLT, GLD and the VIX from Yahoo Finance, covering 2004 to 2025. Prices become daily log returns, and the VIX becomes a daily change (ΔVIX) to capture volatility shocks.¹

Step 2: A warm-up Markov chain

Before the HMM, the authors bin ΔVIX into 3 volatility buckets and count how often the market moves from one bucket to another. The buckets turn out to be sticky: once you're in one, you tend to stay. That stickiness is the empirical reason a regime model makes sense at all.

Step 3: The Hidden Markov Model

• They fit 2-state and 3-state Gaussian HMMs with the Expectation-Maximization (EM) algorithm.
• Filtering (what's the regime probability today, using data up to today) uses the Hamilton forward filter. Smoothing (hindsight probabilities) uses a backward smoother, and the Viterbi path gives the single most likely regime sequence.
• 3 states win on both AIC and BIC (BIC 17,385 vs 18,010 for 2 states). The authors say 4 or more states barely improve BIC and make the probabilities unstable.¹

Regime character and share of days, per the paper's Tables 2–3:¹

Two details are worth noticing. All three regimes are very persistent (daily "stay" probabilities of 0.94, 0.91 and 0.87). And the model essentially never jumps straight from calm to crisis: it passes through the transitional regime first.

Step 4a: Rule-based rotation

Two simple rules map each regime to holdings (paper's Table 4):¹

Trades are executed with a one-day lag. The regime is decided at day t, and the position earns day t+1's return. This avoids look-ahead bias.

Step 4b: The reinforcement-learning policy

• State: the HMM's predicted regime (just 3 possible values).
• Actions: 7 fixed portfolio mixes of TLT, GLD and SPY.
• Learning: tabular policy iteration, a classic dynamic-programming method. It scores each action with the Bellman equation, using the regime-conditional expected return and the HMM's transition matrix.

The policy it learned on the training data is tiny and easy to read (paper's Table 5):¹

In other words: take equity risk unless the market is in crisis, then hide in gold. TLT gets no allocation at all.

Results

The data are split chronologically: the first 70% is for training, and the last 30% is a strict out-of-sample test. The backtest is daily, with a 1-day lag and zero transaction costs.

Out-of-sample test window (paper's Table 7):¹

Full sample, rotation strategies only (paper's Table 6): Top-1 had a 0.52 Sharpe and a −28.3% max drawdown, and 60/40 had 0.58 and −29.3%. Buy-and-hold SPY had a 0.46 Sharpe but a −59.6% drawdown. Buy-and-hold SPY still had the highest cumulative return (476.3% vs 407.8% for Top-1).¹

Read it with care

• "Outperforms" means risk-adjusted. In the test window, the RL policy beat SPY on raw return. The two rotation rules did not, and over the full sample SPY's cumulative return beat both of them. The real win is smaller drawdowns and better Sharpe.
• The RL edge is thin. Its out-of-sample Sharpe (0.83) ties a plain monthly equal-weight portfolio and is only just ahead of 60/40 rotation (0.82).
• No trading costs, slippage or liquidity limits. The authors flag this themselves. Regime strategies trade around regime boundaries, where costs bite.
• Gaussian emissions. Real returns have fat tails and jumps, so extreme shocks may be under-represented. The authors list this as a limitation too.¹
• Some internal inconsistencies. The Top-1 rule holds TLT in crisis, but the RL policy holds GLD, and the paper's explanation for the switch is qualitative. Treat the policy table as illustrative, not as a law.
• Venue. It's an arXiv preprint by independent researchers, it hasn't been peer-reviewed, and no code is linked.

How this relates to MCI-style phasing

Same architecture, different lens. Both MCI and this paper use regime gating, which works in two layers: first classify the regime, then switch exposure. The paper switches between ETFs. MCI switches bots on and off in a bot portfolio. The paper is evidence that this separation keeps the decision layer small and interpretable. A 3-row policy table is something you can audit, and the same holds for an MCI condition → bot on/off matrix.

The transitional state is the paper's "indecision" zone. In the transition matrix, calm and crisis only connect through State 1. That echoes MCI's idea of inserting indecision conditions (2AR/2RR after a markdown, 4DR/4RR after a markup) between trending phases instead of flipping straight from up to down.

Where MCI goes further:

• Direction and structure, not just volatility. A volatility HMM can't tell accumulation from distribution. Both are range-bound "transitional" periods. MCI's split (2AR vs 2RR, 4RR vs 4DR) depends on what came before the range, and that's exactly the information that matters for which bots to run.
• Timeframe. The paper works on daily ETF data. MCI is tuned for the 4H chart, where regimes are shorter and costs matter more.
• More states, with meaning. The paper finds that more than 3 statistical states become unstable. MCI's 8 conditions are defined by market structure rather than fitted blindly, which is one way around that instability.

Ideas worth stealing for MCI research:

1. Report persistence. Estimate a transition matrix between MCI conditions (how often 2AR leads to 3MU vs back to 1MD, and average stage duration). It's a compact, shareable way to show that the phases are real and sticky.
2. Run an HMM as a baseline. Fit a 3-state HMM on 4H returns (for example with hmmlearn; read the look-ahead warning in its deep-dive first) and check how its states overlap with MCI conditions. If MCI adds information beyond volatility, it should separate outcomes inside the HMM's single "transitional" bucket.
3. Learn the gating table. In the SOE backtester, swap a hand-written "condition → bots on/off" table for a tabular policy learned on training data and tested out-of-sample with a 1-bar lag. It stays interpretable, since it's still one row per condition.
4. Include costs from day one. Bot start/stop at stage boundaries is where frictions concentrate.

Key terms

• Hidden Markov Model (HMM): a model where the market sits in one of several unobserved states, each with its own return distribution, and moves between states with fixed probabilities.
• Transition matrix: the table of probabilities of moving from each state to each other state in one step.
• Viterbi path: the single most likely sequence of hidden states, given all the data.
• Policy iteration: a dynamic-programming method that repeatedly evaluates and improves a decision rule until it stops changing.
• Look-ahead bias: accidentally using information that wouldn't have been available at decision time. The 1-day lag guards against it.

Strategy hooks

Buildable ideas a strategy agent can pick up. Each one is a hypothesis to test, not a claim from the paper.

hooks:
  - id: vol-regime-gate-on-bots
    family: regime switching
    idea: Fit a 3-state Gaussian HMM on returns and pause trend bots in the high-volatility state.
    markets: [crypto majors, index ETFs]
    timeframes: [4H, 1D]
    data_needed: [OHLC closes, optional VIX or realized volatility]
    complexity: intermediate
    gate: run bots only while the filtered (causal) probability of the high-vol state is below a threshold
    test: walk-forward fit, 1-bar execution lag, costs on, compare drawdown and Sharpe against always-on bots
    success: lower max drawdown at similar or better return after costs
  - id: transitional-state-as-range-filter
    family: mean reversion
    idea: Treat the HMM transitional state as a range regime and enable mean-reversion bots only there.
    markets: [crypto majors, FX majors]
    timeframes: [4H]
    data_needed: [OHLC]
    complexity: intermediate
    gate: transitional state is the most likely filtered state for N consecutive bars
    test: compare mean-reversion PnL inside vs outside the transitional state
    success: mean-reversion edge concentrated inside the transitional state
  - id: regime-policy-table
    family: ML/AI
    idea: Learn a small per-regime allocation table with tabular policy iteration instead of hand-written rules.
    markets: [multi-asset ETFs]
    timeframes: [1D]
    data_needed: [daily closes for each asset, HMM regime labels]
    complexity: advanced
    gate: one row per regime, one action per row
    test: train on the first 70% of history, test on the last 30% with a 1-day lag and costs
    success: beats a monthly equal-weight portfolio on Sharpe out-of-sample
  - id: crisis-safe-haven-rotation
    family: tactical allocation
    idea: Rotate from equities to gold or treasuries when the crisis regime becomes most likely.
    markets: [SPY, GLD, TLT]
    timeframes: [1D]
    data_needed: [daily closes, VIX]
    complexity: beginner
    gate: crisis state most likely (filtered), with a minimum holding period to limit churn
    test: backtest with costs; count switches per year
    success: drawdown cut versus buy-and-hold with a manageable number of switches

Try it in Trade Holistic

Add a volatility-regime filter to my long strategy on daily SPY: only allow new longs while ATR as a percent of price is below its 100-day average and price is above its 200-day moving average. If ATR percent rises above 1.5 times its average, close the long at the next bar and stay flat until it calms down. Use the previous closed daily bar for every check.

Build a long-only regime rotation test on daily SPY: full size in calm conditions (ATR percent below its 100-day average), half size in transitional conditions (between 1 and 1.5 times the average) and flat in crisis conditions (above 1.5 times). Use a 3 ATR trailing stop, include commission and slippage, and compare it with buy-and-hold on the same dates.

Related

• Source paper: arXiv:2605.27848
• Concepts: Hidden Markov Model, regime gating
• Trade Holistic: MCI, SOE backtester, bot portfolio
• Repos: hmmlearn (deep-dive available), statsmodels (Markov-switching regression; no deep-dive yet)

Repos in this page