---
type: Alpha Explainer
title: "Regime-Based Portfolio Allocation with Hidden Markov Models and Reinforcement Learning"
description: "A 3-state HMM sorts the market into calm, transitional and crisis regimes, then a small, readable RL policy decides what to hold in each one."
resource: "https://arxiv.org/abs/2605.27848"
tags: [regime detection, hidden markov model, reinforcement learning, asset allocation, regime gating]
status: stable
generated: { by: grok-bot/alpha-explainer-2.0, at: "2026-10-16T02:45:00Z" }
verified:
  - { by: process:alpha-explainer-numcheck, at: "2026-10-16T03:05:00Z" }
stale_after: "2027-10-16T00:00:00Z"
sources:
  - id: arxiv-2605-27848
    resource: "https://arxiv.org/abs/2605.27848"
    title: "Regime-Based Portfolio Allocation Using Hidden Markov Models and Reinforcement Learning"
    author: "human:ajay-kumar-verma"
    last_modified: "2026-05-27T00:00:00Z"
# ---- Edge Brain OS extensions (see /schema.md) ----
kb_id: th-ex-20261016-hmm-rl-regime-allocation
chart_url: "https://alpha.tradeholistic.io/hmm-rl-regime-allocation/chart.png"
newsletter_summary: "A new paper labels the market as calm, transitional or crisis with a Hidden Markov Model, then holds whatever has worked best in that regime. In its out-of-sample test, a learned allocation policy beat buy-and-hold SPY on Sharpe ratio (0.83 vs 0.65) with a smaller max drawdown (-23.5% vs -35.7%). We explain how it works and how the same two steps (classify the regime, then switch exposure) can gate trading bots."
newsletter_hook: "Knowing whether the market is calm, shaky or in crisis can shrink your drawdowns; this paper shows a simple way to switch exposure."
newsletter_chart: assets/hmm-rl-regime-allocation/oos-results.svg
try_it_prompts:
  - role: filter
    prompt: "Add a volatility-regime filter to my long strategy on daily SPY: only allow new longs while ATR as a percent of price is below its 100-day average and price is above its 200-day moving average. If ATR percent rises above 1.5 times its average, close the long at the next bar and stay flat until it calms down. Use the previous closed daily bar for every check."
  - role: risk
    prompt: "Build a long-only regime rotation test on daily SPY: full size in calm conditions (ATR percent below its 100-day average), half size in transitional conditions (between 1 and 1.5 times the average) and flat in crisis conditions (above 1.5 times). Use a 3 ATR trailing stop, include commission and slippage, and compare it with buy-and-hold on the same dates."
try_it_look_for: "Judge it against buy-and-hold on the same dates: the filter earns its place if max drawdown and the worst years (check 2008, 2020 and 2022) shrink clearly while total return gives up less than the drawdown it saves, and it doesn't flip in and out every few days."
try_it_pitfall: "Lookahead and overfitting: the paper's regime labels are fitted on the full history, so only trade on values from closed bars, and don't fine-tune the 1.5x threshold on the same years you judge it on; check that nearby values (1.3x to 1.8x) give similar results."
herenow_slug: ubuntu-saddle-3ga7
published_url: "https://alpha.tradeholistic.io/hmm-rl-regime-allocation/"
source_type: paper
topics: [market regimes, volatility regimes, portfolio allocation, reinforcement learning]
alpha:
  strategy_families: [regime switching, tactical allocation, ML/AI]
  core_idea: "Label the volatility regime with an HMM, then hold what has worked best in that regime."
  markets: [US equity ETFs, treasuries, gold]
  asset_classes: [equities, rates, commodities]
  timeframes: [1D]
  data_needed: [daily closes, VIX]
  complexity: intermediate
mci_relevance: "Same two-layer design as MCI: classify the regime first, then switch exposure. The paper's transitional state plays the role of MCI's indecision ranges."
relations:
  source: /sources/arxiv-2605.27848.md
  repos: [/repos/gh-hmmlearn-hmmlearn.md, /repos/gh-statsmodels-statsmodels.md]
  concepts: [/concepts/hidden-markov-model.md, /concepts/regime-gating.md]
  trade_holistic: [/trade-holistic/mci.md, /trade-holistic/soe-backtester.md, /trade-holistic/bot-portfolio.md]
paper:
  authors: ["Ajay Kumar Verma", "Nunik Srikandi Putri", "Neo Paul Lesupi"]
  published: "2026-05-27"
  venue: "arXiv preprint (q-fin), not peer-reviewed"
reading_time: "7 min"
---


# TL;DR

- **What it is:** a preprint that detects market regimes with a **3-state Gaussian Hidden Markov Model (HMM)** and then picks a portfolio of **SPY (stocks), TLT (long Treasuries) and GLD (gold)** for each regime.
- **Two ways to act on a regime:** simple *rotation rules* (hold the best asset per regime) and a small *reinforcement-learning (RL) policy* that learns which mix to hold.
- **Headline result:** in the out-of-sample test, the RL policy had the highest Sharpe ratio (0.83, tied with a monthly equal-weight portfolio) and a shallower drawdown than buying and holding SPY (−23.5% vs −35.7%).
- **Why it matters for us:** it's a clean, transparent template for *regime-gated* strategy switching. That's exactly what MCI does with bots.

# The idea in plain language

Markets don't behave the same way all the time. Calm periods, nervous periods and crashes each have their own "personality": different volatility, different winners and different correlations. The paper's bet is simple:

1. **Figure out which personality the market is in right now** (the regime).
2. **Hold whatever has historically worked best in that regime.**

The regime is "hidden" because you can't observe it directly. You only see daily returns, and you infer the regime from how those returns behave. That's what a [Hidden Markov Model](/concepts/hidden-markov-model.md) is built for.

![How the system works, end to end](assets/hmm-rl-regime-allocation/pipeline.svg)

# How it works

## Step 1: Data

Daily closing prices for **SPY, TLT, GLD and the VIX** from Yahoo Finance, covering **2004 to 2025**. Prices become daily log returns, and the VIX becomes a daily change (ΔVIX) to capture volatility shocks.[^arxiv-2605-27848]

## Step 2: A warm-up Markov chain

Before the HMM, the authors bin ΔVIX into 3 volatility buckets and count how often the market moves from one bucket to another. The buckets turn out to be **sticky**: once you're in one, you tend to stay. That stickiness is the empirical reason a regime model makes sense at all.

## Step 3: The Hidden Markov Model

- They fit **2-state and 3-state Gaussian HMMs** with the Expectation-Maximization (EM) algorithm.
- **Filtering** (what's the regime probability today, using data up to today) uses the Hamilton forward filter. **Smoothing** (hindsight probabilities) uses a backward smoother, and the **Viterbi** path gives the single most likely regime sequence.
- **3 states win** on both AIC and BIC (BIC 17,385 vs 18,010 for 2 states). The authors say 4 or more states barely improve BIC and make the probabilities unstable.[^arxiv-2605-27848]

![The three regimes and how the market moves between them](assets/hmm-rl-regime-allocation/regimes.svg)

Regime character and share of days, per the paper's Tables 2–3:[^arxiv-2605-27848]

| Regime | Share of days | Character (from the paper) |
|---|---|---|
| State 0, low volatility | 50.83% | Stable expansion. SPY has the highest mean return and lowest dispersion. |
| State 1, transitional | 42.98% | Muted returns, a bit more volatility, no asset clearly dominates. |
| State 2, high volatility | 6.18% | Crisis. SPY has a strongly negative mean, while TLT and GLD hold up. |

Two details are worth noticing. All three regimes are very persistent (daily "stay" probabilities of 0.94, 0.91 and 0.87). And the model essentially never jumps straight from calm to crisis: it passes **through the transitional regime** first.

## Step 4a: Rule-based rotation

Two simple rules map each regime to holdings (paper's Table 4):[^arxiv-2605-27848]

| Regime | Top-1 rule | 60/40 rule |
|---|---|---|
| State 0, low vol | 100% SPY | 60% SPY / 40% GLD |
| State 1, transitional | 100% GLD | 60% GLD / 40% TLT |
| State 2, high vol | 100% TLT | 60% TLT / 40% GLD |

Trades are executed with a **one-day lag**. The regime is decided at day *t*, and the position earns day *t+1*'s return. This avoids look-ahead bias.

## Step 4b: The reinforcement-learning policy

- **State:** the HMM's predicted regime (just 3 possible values).
- **Actions:** 7 fixed portfolio mixes of TLT, GLD and SPY.
- **Learning:** *tabular policy iteration*, a classic dynamic-programming method. It scores each action with the Bellman equation, using the regime-conditional expected return and the HMM's transition matrix.

The policy it learned on the training data is tiny and easy to read (paper's Table 5):[^arxiv-2605-27848]

| Regime | RL holds |
|---|---|
| State 0, low vol | 100% SPY |
| State 1, transitional | 100% SPY |
| State 2, high vol | 100% GLD |

In other words: **take equity risk unless the market is in crisis, then hide in gold.** TLT gets no allocation at all.

# Results

The data are split chronologically: the **first 70% is for training**, and the **last 30% is a strict out-of-sample test**. The backtest is daily, with a 1-day lag and **zero transaction costs**.

![Out-of-sample Sharpe ratio and max drawdown](assets/hmm-rl-regime-allocation/oos-results.svg)

**Out-of-sample test window (paper's Table 7):**[^arxiv-2605-27848]

| Strategy | Cumulative return | Annualized return | Volatility | Sharpe | Max drawdown |
|---|---|---|---|---|---|
| RL policy π* (net) | 131.4% | 14.3% | 17.3% | 0.83 | −23.5% |
| Rotation, Top-1 | 110.4% | 12.6% | 16.0% | 0.79 | −21.8% |
| Rotation, 60/40 | 86.9% | 10.5% | 12.8% | 0.82 | −29.3% |
| Equal-weight (monthly) | 72.5% | 9.1% | 11.0% | 0.83 | −24.0% |
| Buy & hold SPY | 118.4% | 13.2% | 20.5% | 0.65 | −35.7% |

**Full sample, rotation strategies only (paper's Table 6):** Top-1 had a 0.52 Sharpe and a −28.3% max drawdown, and 60/40 had 0.58 and −29.3%. Buy-and-hold SPY had a 0.46 Sharpe but a −59.6% drawdown. Buy-and-hold SPY still had the **highest cumulative return** (476.3% vs 407.8% for Top-1).[^arxiv-2605-27848]

# Read it with care

- **"Outperforms" means risk-adjusted.** In the test window, the RL policy beat SPY on raw return. The two rotation rules did not, and over the full sample SPY's cumulative return beat both of them. The real win is smaller drawdowns and better Sharpe.
- **The RL edge is thin.** Its out-of-sample Sharpe (0.83) **ties a plain monthly equal-weight portfolio** and is only just ahead of 60/40 rotation (0.82).
- **No trading costs, slippage or liquidity limits.** The authors flag this themselves. Regime strategies trade around regime boundaries, where costs bite.
- **Gaussian emissions.** Real returns have fat tails and jumps, so extreme shocks may be under-represented. The authors list this as a limitation too.[^arxiv-2605-27848]
- **Some internal inconsistencies.** The Top-1 rule holds TLT in crisis, but the RL policy holds GLD, and the paper's explanation for the switch is qualitative. Treat the policy table as illustrative, not as a law.
- **Venue.** It's an arXiv preprint by independent researchers, it hasn't been peer-reviewed, and no code is linked.

# How this relates to MCI-style phasing

![Rough mapping between the paper's volatility states and MCI conditions](assets/hmm-rl-regime-allocation/mci-mapping.svg)

**Same architecture, different lens.** Both [MCI](/trade-holistic/mci.md) and this paper use [regime gating](/concepts/regime-gating.md), which works in two layers: *first classify the regime, then switch exposure*. The paper switches between ETFs. MCI switches bots on and off in a [bot portfolio](/trade-holistic/bot-portfolio.md). The paper is evidence that this separation keeps the decision layer **small and interpretable**. A 3-row policy table is something you can audit, and the same holds for an MCI condition → bot on/off matrix.

**The transitional state is the paper's "indecision" zone.** In the transition matrix, calm and crisis only connect *through* State 1. That echoes MCI's idea of inserting indecision conditions (2AR/2RR after a markdown, 4DR/4RR after a markup) between trending phases instead of flipping straight from up to down.

**Where MCI goes further:**

- **Direction and structure, not just volatility.** A volatility HMM can't tell *accumulation* from *distribution*. Both are range-bound "transitional" periods. MCI's split (2AR vs 2RR, 4RR vs 4DR) depends on *what came before* the range, and that's exactly the information that matters for which bots to run.
- **Timeframe.** The paper works on daily ETF data. MCI is tuned for the 4H chart, where regimes are shorter and costs matter more.
- **More states, with meaning.** The paper finds that more than 3 *statistical* states become unstable. MCI's 8 conditions are defined by market structure rather than fitted blindly, which is one way around that instability.

**Ideas worth stealing for MCI research:**

1. **Report persistence.** Estimate a transition matrix between MCI conditions (how often 2AR leads to 3MU vs back to 1MD, and average stage duration). It's a compact, shareable way to show that the phases are real and sticky.
2. **Run an HMM as a baseline.** Fit a 3-state HMM on 4H returns (for example with [hmmlearn](/repos/gh-hmmlearn-hmmlearn.md); read the look-ahead warning in its deep-dive first) and check how its states overlap with MCI conditions. If MCI adds information beyond volatility, it should separate outcomes inside the HMM's single "transitional" bucket.
3. **Learn the gating table.** In the [SOE backtester](/trade-holistic/soe-backtester.md), swap a hand-written "condition → bots on/off" table for a tabular policy learned on training data and tested out-of-sample with a 1-bar lag. It stays interpretable, since it's still one row per condition.
4. **Include costs from day one.** Bot start/stop at stage boundaries is where frictions concentrate.

# Key terms

- **Hidden Markov Model (HMM):** a model where the market sits in one of several unobserved states, each with its own return distribution, and moves between states with fixed probabilities.
- **Transition matrix:** the table of probabilities of moving from each state to each other state in one step.
- **Viterbi path:** the single most likely sequence of hidden states, given all the data.
- **Policy iteration:** a dynamic-programming method that repeatedly evaluates and improves a decision rule until it stops changing.
- **Look-ahead bias:** accidentally using information that wouldn't have been available at decision time. The 1-day lag guards against it.

# Strategy hooks

Buildable ideas a strategy agent can pick up. Each one is a hypothesis to test, not a claim from the paper.

```yaml
hooks:
  - id: vol-regime-gate-on-bots
    family: regime switching
    idea: Fit a 3-state Gaussian HMM on returns and pause trend bots in the high-volatility state.
    markets: [crypto majors, index ETFs]
    timeframes: [4H, 1D]
    data_needed: [OHLC closes, optional VIX or realized volatility]
    complexity: intermediate
    gate: run bots only while the filtered (causal) probability of the high-vol state is below a threshold
    test: walk-forward fit, 1-bar execution lag, costs on, compare drawdown and Sharpe against always-on bots
    success: lower max drawdown at similar or better return after costs
  - id: transitional-state-as-range-filter
    family: mean reversion
    idea: Treat the HMM transitional state as a range regime and enable mean-reversion bots only there.
    markets: [crypto majors, FX majors]
    timeframes: [4H]
    data_needed: [OHLC]
    complexity: intermediate
    gate: transitional state is the most likely filtered state for N consecutive bars
    test: compare mean-reversion PnL inside vs outside the transitional state
    success: mean-reversion edge concentrated inside the transitional state
  - id: regime-policy-table
    family: ML/AI
    idea: Learn a small per-regime allocation table with tabular policy iteration instead of hand-written rules.
    markets: [multi-asset ETFs]
    timeframes: [1D]
    data_needed: [daily closes for each asset, HMM regime labels]
    complexity: advanced
    gate: one row per regime, one action per row
    test: train on the first 70% of history, test on the last 30% with a 1-day lag and costs
    success: beats a monthly equal-weight portfolio on Sharpe out-of-sample
  - id: crisis-safe-haven-rotation
    family: tactical allocation
    idea: Rotate from equities to gold or treasuries when the crisis regime becomes most likely.
    markets: [SPY, GLD, TLT]
    timeframes: [1D]
    data_needed: [daily closes, VIX]
    complexity: beginner
    gate: crisis state most likely (filtered), with a minimum holding period to limit churn
    test: backtest with costs; count switches per year
    success: drawdown cut versus buy-and-hold with a manageable number of switches
```

# Try it in Trade Holistic

Paste a prompt into the Trade Holistic strategy-builder chat, then run it through the backtester.

**Prompt 1 (filter):**

```text
Add a volatility-regime filter to my long strategy on daily SPY: only allow new longs while ATR as a percent of price is below its 100-day average and price is above its 200-day moving average. If ATR percent rises above 1.5 times its average, close the long at the next bar and stay flat until it calms down. Use the previous closed daily bar for every check.
```

**Prompt 2 (risk):**

```text
Build a long-only regime rotation test on daily SPY: full size in calm conditions (ATR percent below its 100-day average), half size in transitional conditions (between 1 and 1.5 times the average) and flat in crisis conditions (above 1.5 times). Use a 3 ATR trailing stop, include commission and slippage, and compare it with buy-and-hold on the same dates.
```

**What to look for in the backtest:** Judge it against buy-and-hold on the same dates: the filter earns its place if max drawdown and the worst years (check 2008, 2020 and 2022) shrink clearly while total return gives up less than the drawdown it saves, and it doesn't flip in and out every few days.

**Pitfall to avoid:** Lookahead and overfitting: the paper's regime labels are fitted on the full history, so only trade on values from closed bars, and don't fine-tune the 1.5x threshold on the same years you judge it on; check that nearby values (1.3x to 1.8x) give similar results.

# Related

- Source paper: [arXiv:2605.27848](/sources/arxiv-2605.27848.md)
- Concepts: [Hidden Markov Model](/concepts/hidden-markov-model.md), [regime gating](/concepts/regime-gating.md)
- Trade Holistic: [MCI](/trade-holistic/mci.md), [SOE backtester](/trade-holistic/soe-backtester.md), [bot portfolio](/trade-holistic/bot-portfolio.md)
- Repos: [hmmlearn](/repos/gh-hmmlearn-hmmlearn.md) (deep-dive available), [statsmodels](/repos/gh-statsmodels-statsmodels.md) (Markov-switching regression; no deep-dive yet)

[^arxiv-2605-27848]: Verma, Putri and Lesupi, "Regime-Based Portfolio Allocation Using Hidden Markov Models and Reinforcement Learning", arXiv:2605.27848.