Deep Learning for Portfolio Optimisation

Summary

Classical portfolio construction is two problems in sequence: estimate expected returns and a covariance matrix, then solve an optimisation. The errors of the first stage dominate the second, and the first stage is never told what the second stage will do with its output. An end-to-end allocation network collapses the two: a sequence model reads a window of market data and emits portfolio weights directly, and the loss function is the performance statistic itself — a Sharpe ratio computed net of transaction costs, with regularisers on concentration and turnover. Nothing in the pipeline ever predicts a return. This article sets out the formulation, explains why differentiating a ratio behaves differently from differentiating a forecast error, gives a complete PyTorch implementation including the causal feature construction that most reproductions get wrong, and sets against the merits a blunt accounting of the statistical problem: the effective sample size is the number of periods, not the number of data points.

1. Introduction

Mean–variance optimisation is not usually wrong because the optimiser is wrong. It is wrong because of what it is fed. Expected returns are estimated with standard errors of the same order as the estimates themselves, the optimiser is a maximiser and therefore an error-maximiser, and the resulting weights are famously unstable with respect to inputs that are themselves unstable. Shrinkage, resampling and constraints are all ways of damping a problem created upstream.

A second, subtler objection is that the two stages have different objectives. A forecasting model minimises squared error, weighting every asset and every period equally. The portfolio does not care about squared error; it cares about the realised risk-adjusted return of a particular weighted combination, net of what it costs to trade into it. A forecast can improve on its own metric and make the portfolio worse, and in practice this is common.

End-to-end — or decision-focused — learning removes the intermediate quantity. If a network maps market data to weights, and the weights determine a differentiable performance statistic, then gradient descent can be pointed at the statistic. Expected returns and covariances never appear. Whatever the network learns about them is implicit, task-specific, and never written down.

2. The model

Weights by construction. The network’s output layer is a softmax across assets:

$${𝐰}_t=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}(f_{\theta }({𝐗}_t)),w_{i,t}>0,{\sum }_{i=1}^N{w}_{i,t}=1$$

Long-only and fully invested hold exactly, at every step of training, without a projection or a penalty. The initialisation matters as a consequence: an untrained network with small output weights produces approximately equal weights, so training begins at 1/N and learns tilts away from it.

The realised return. With simple returns r_i,t+1 and proportional cost c per unit turnover,

$$R_{t+1}={\sum }_{i=1}^N{w}_{i,t}{r}_{i,t+1}-c{\sum }_{i=1}^N|w_{i,t}-w_{i,t-1}|$$

The cost term is what forces the network to care about the path of its own weights. Without it, nothing penalises a model that re-allocates completely every period.

The objective. The loss is the negative Sharpe ratio of that series over the mini-batch, plus two regularisers:

$$ℒ(\theta )=-(\frac{\bar{R}}{\mathrm{s}\mathrm{d}(R)}+{\lambda }_{\mathrm{e}\mathrm{n}\mathrm{t}}𝔼[H(𝐰)\left]\right)+{\lambda }_{\mathrm{t}\mathrm{o}}𝔼[\Vert \Delta 𝐰{\Vert }_1]$$

where H(w) = −Σ w_i log w_i is the entropy of the weight vector. Entropy is maximised at equal weight and falls towards zero as the book concentrates, so the term is a differentiable diversification preference expressed in the objective rather than as a constraint. The turnover term appears twice in effect — once inside the Sharpe ratio through the cost deduction, and once as an explicit penalty — because the first is scaled by the (small) cost and the second controls turnover directly.

Why a ratio behaves differently. The gradient of a Sharpe ratio with respect to a return observation is

$$\frac{\partial }{\partial R_t}\left(\frac{\bar{R}}{\mathrm{s}\mathrm{d}(R)}\right)=\frac{1}{T\mathrm{s}\mathrm{d}(R)}(1-\mathrm{S}\mathrm{R}·\frac{R_t-\bar{R}}{\mathrm{s}\mathrm{d}(R)})$$

Two consequences follow. First, the objective is scale-free in the returns, so the network cannot improve its loss by levering up — the leverage decision is genuinely separate, which is why volatility scaling is applied after training rather than learned. Second, the gradient on a period is reduced, and can reverse sign, when that period’s return is an outlier in the favourable direction: the objective dislikes variance wherever it comes from. This is the behaviour the objective was chosen for, but it means a large gain and a large loss are not treated as opposites.

Volatility scaling, applied afterwards. The book is scaled so that its ex-ante volatility sits near a target:

$$m_t=\mathrm{c}\mathrm{l}\mathrm{i}\mathrm{p}(\frac{{\widehat{\sigma }}_{\mathrm{t}\mathrm{g}\mathrm{t}}}{{\widehat{\sigma }}_{p,t}},0,m_{max}),{\widehat{\sigma }}_{p,t}=\sqrt{252{\sum }_i{w}_{i,t}^2{\widehat{\sigma }}_{i,t-1}^2},R_{t+1}^{\mathrm{v}\mathrm{s}}=m_t{R}_{t+1}$$

Every input to m_t must be known at t. The volatility estimate is an exponentially weighted average lagged by one period; using a contemporaneous volatility, or one computed over the whole sample, converts a leverage rule into a look-ahead, and the effect on reported risk-adjusted return is large (Moreira & Muir, 2017; Harvey et al., 2018).

3. How it works

Features. Each decision point gets a look-back window of three features per asset: a price normalised by the last price in the window, a raw return, and a return divided by a trailing volatility estimate. The normalisation is what lets assets with very different volatilities share one network — without it, the highest-volatility asset dominates every gradient. All three are computed from information available at the decision point only.

Architecture. An LSTM (Hochreiter & Schmidhuber, 1997) reads the window and produces a hidden state per time step; a self-attention block (Vaswani et al., 2017) lets the model weight time steps by relevance rather than by recency alone; a feed-forward block with residual connections adds capacity; a linear head followed by softmax produces the weights from the final hidden state. The architecture is deliberately ordinary — the idea being tested is the objective, not the network.

Training. Mini-batches must be contiguous in time and must not be shuffled, for two reasons: the turnover term differences consecutive rows, and the Sharpe ratio is a property of a sequence. This single requirement rules out most default training loops. Adam with cosine-decayed learning rate, gradient clipping, dropout and early stopping on validation net Sharpe are standard; the ensemble-over-seeds and exponential smoothing of weights that practitioners add are both variance-reduction devices for an objective whose optimum is shallow.

Evaluation. Weights at t are applied to returns from t to t+1, costs are charged on the change in weights, and only then is the volatility multiplier applied. Walk-forward retraining on an expanding window is the honest protocol; a single train–test split on a fixed window answers a different and easier question.

4. Python implementation

PyTorch and NumPy. The four blocks are features, network, loss and training.

The look-back tensor. The two causality traps live in this function. The window-normalised price divides by the last price in the window, not the first and not a future price; and the volatility used to standardise returns is lagged by one period. Both are easy to get wrong and neither produces an error message.

import numpy as np
import torch
import torch.nn as nn

def build_tensors(R, window=50, vol_span=50):
    """Three scale-free features per asset: normalised price, raw return, vol-scaled return.

    R : (T, N) simple returns.  Returns X of shape (T - window, window, 3N) and the
    next-period return matrix Y of shape (T - window, N). Every feature at row t
    uses information up to and including t, so the map from X[t] to Y[t] is causal.
    """
    T, N = R.shape
    price = np.cumprod(1.0 + R, axis=0)
    ew = np.zeros_like(R)
    a = 2.0 / (vol_span + 1.0)
    var = R[:vol_span].var(axis=0)
    for t in range(T):                                    # EWMA variance, causal
        var = (1 - a) * var + a * R[t] ** 2
        ew[t] = np.sqrt(var)
    ew = np.vstack([ew[:1], ew[:-1]])                     # lag by one period
    X, Y = [], []
    for t in range(window, T - 1):
        p = price[t - window:t] / price[t - 1]            # window-normalised price
        r = R[t - window:t]
        z = R[t - window:t] / np.maximum(ew[t - window:t], 1e-8)
        X.append(np.concatenate([p, r, z], axis=1))
        Y.append(R[t + 1])                                # next-period return
    return (torch.tensor(np.array(X), dtype=torch.float32),
            torch.tensor(np.array(Y), dtype=torch.float32),
            torch.tensor(ew[window:T - 1], dtype=torch.float32))

The network. Roughly fifty thousand parameters for a small cross-section — worth holding in mind against the sample size discussed in Section 6.

class AllocationNet(nn.Module):
    """LSTM over the look-back window, self-attention across time, softmax over assets."""

    def __init__(self, n_features, n_assets, hidden=64, n_heads=4, dropout=0.30):
        super().__init__()
        self.lstm = nn.LSTM(n_features, hidden, batch_first=True)
        self.attn = nn.MultiheadAttention(hidden, n_heads, dropout=dropout,
                                          batch_first=True)
        self.ffn = nn.Sequential(nn.Linear(hidden, 2 * hidden), nn.GELU(),
                                 nn.Dropout(dropout), nn.Linear(2 * hidden, hidden))
        self.head = nn.Linear(hidden, n_assets)

    def forward(self, x):                                 # x: (B, window, 3N)
        h, _ = self.lstm(x)
        a, _ = self.attn(h, h, h, need_weights=False)     # attention over time steps
        h = h + a
        h = h + self.ffn(h)
        return torch.softmax(self.head(h[:, -1, :]), dim=-1)   # long-only, sums to one

The objective. The previous weight vector of the first row in a batch is detached, so gradients do not flow across the batch boundary while the turnover term still sees a sensible first difference.

def sharpe_loss(w, y, cost=1e-4, l_entropy=0.02, l_turnover=0.30, eps=1e-8):
    """Negative Sharpe ratio of the net-of-cost portfolio return, with two regularisers.

    The batch must be in time order: the turnover term differences consecutive rows.
    """
    w_prev = torch.cat([w[:1].detach(), w[:-1]], dim=0)
    turnover = (w - w_prev).abs().sum(-1)
    rp = (w * y).sum(-1) - cost * turnover
    sharpe = rp.mean() / (rp.std(unbiased=False) + eps)
    entropy = -(w * (w + eps).log()).sum(-1).mean()       # pushes away from corners
    return -(sharpe + l_entropy * entropy) + l_turnover * turnover.mean()

Training and the leverage overlay. Note range(0, len(X) - batch + 1, batch) with no shuffling, and that vol_scaled_return takes the lagged volatility array as an argument rather than recomputing it — the signature makes the causality requirement explicit.

def train(model, X, Y, epochs=30, batch=128, lr=1e-3, clip=1.0):
    opt = torch.optim.Adam(model.parameters(), lr=lr, weight_decay=1e-5)
    sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, T_max=epochs, eta_min=1e-5)
    for epoch in range(epochs):
        model.train()
        total = 0.0
        for i in range(0, len(X) - batch + 1, batch):     # sequential, never shuffled
            opt.zero_grad()
            loss = sharpe_loss(model(X[i:i + batch]), Y[i:i + batch])
            loss.backward()
            nn.utils.clip_grad_norm_(model.parameters(), clip)
            opt.step()
            total += loss.item()
        sched.step()
        if epoch % 10 == 0 or epoch == epochs - 1:
            print("  epoch %3d  loss %+.4f" % (epoch, total / max(1, len(X) // batch)))
    return model


def vol_scaled_return(w, y, sigma, target=0.10, periods=252, cap=5.0, cost=1e-4):
    """Scale the whole book so that its ex-ante volatility sits near the target."""
    w = w.detach().numpy()
    gross = (w * y.numpy()).sum(-1)
    turn = np.abs(np.diff(w, axis=0, prepend=w[:1])).sum(-1)
    port_vol = np.sqrt(((w ** 2) * (sigma.numpy() ** 2)).sum(-1)) * np.sqrt(periods)
    mult = np.clip(target / np.maximum(port_vol, 1e-8), 0.0, cap)
    return mult * (gross - cost * turn)

A worked example. The demonstration uses a synthetic panel with one slow common factor and a spread of factor loadings, so there is a learnable cross-sectional tilt and nothing else.

torch.manual_seed(0)
rng = np.random.default_rng(0)
T, N = 1400, 6
# synthetic panel: a slow common factor, persistent vol, mild cross-sectional spread
f = np.zeros(T)
for t in range(1, T):
    f[t] = 0.97 * f[t - 1] + rng.standard_normal() * 0.004
load = np.linspace(1.2, 0.3, N)
R = f[:, None] * load + rng.standard_normal((T, N)) * 0.008

X, Y, S = build_tensors(R)
n_train = int(0.7 * len(X))
print("tensors", tuple(X.shape), tuple(Y.shape))
model = AllocationNet(X.shape[-1], N)
train(model, X[:n_train], Y[:n_train], epochs=30)

model.eval()
with torch.no_grad():
    w = model(X[n_train:])
r = vol_scaled_return(w, Y[n_train:], S[n_train:])
print("mean weight    ", np.round(w.numpy().mean(0), 3))
print("mean turnover  %.4f" % np.abs(np.diff(w.numpy(), axis=0)).sum(-1).mean())
print("holdout periods %d  realised vol (annualised) %.3f"
      % (len(r), r.std() * np.sqrt(252)))

Output:

tensors (1349, 50, 18) (1349, 6)
  epoch   0  loss +0.3019
  epoch  10  loss -0.0339
  epoch  20  loss -0.0567
  epoch  29  loss -0.0597
mean weight     [0.39  0.108 0.002 0.001 0.001 0.499]
mean turnover  0.0363
holdout periods 405  realised vol (annualised) 0.101

Three features of that run are worth noting. The loss falls from positive to about −0.06 and flattens, which is the shape to expect from a shallow objective. The learned weights are highly concentrated — two assets hold roughly ninety per cent of the book despite the entropy term — which is what a softmax does when the cross-sectional signal is clean, and a warning about what it will do when the signal is noise. And the realised holdout volatility of 10.1 % against a 10 % target shows the overlay working: that is a property of the scaling arithmetic, not evidence that the allocation learned anything.

5. Merits

• No error propagation from estimated moments
  The dominant source of instability in classical construction — expected returns estimated with enormous relative error, then maximised over — is absent, because expected returns are never estimated. This is the core argument for the approach and it is a strong one.
• The objective is the thing actually cared about
  Transaction costs, turnover and concentration sit inside the loss, so the model trades off signal strength against trading friction during training rather than having friction imposed afterwards as a constraint it was never aware of.
• Constraints that hold exactly, for free
  Long-only and fully invested are properties of the softmax, not penalties that are approximately satisfied. Training never visits an infeasible point, and there is no projection step to interact with the optimiser.
• A sensible starting point
  A near-uniform initialisation means the untrained model is approximately equal-weighted — a benchmark that is hard to beat (DeMiguel, Garlappi & Uppal, 2009). Training can only move away from it deliberately, which makes the learned tilt the object of study.
• Non-linear, cross-sectional and path-dependent all at once
  The network can condition on interactions between assets and on the sequence of recent conditions, including its own previous weights through the turnover term. Expressing any of that in a mean–variance framework requires modelling it explicitly first.
• Leverage is separated from allocation
  Because a Sharpe ratio is scale-free, the network learns composition and the overlay sets scale. The two can be re-specified independently, and the risk target can be changed without retraining.
• Hypotheses are testable by ablation
  Removing a feature, a regulariser or the attention block and retraining is a clean experiment. The framework makes “does this component contribute” an answerable question in a way that a hand-tuned optimiser pipeline does not.

6. Demerits

• The effective sample size is the number of periods
  This is the central problem and it is not solvable by better architecture. A panel of N assets over T days contains NT numbers but only T realisations of the quantity being optimised, because the portfolio return is one number per period. Twenty years of daily data is about five thousand observations for a model with tens of thousands of parameters, and those observations are serially dependent and contain few independent market regimes.
• A mini-batch Sharpe ratio is a biased and noisy objective
  The ratio of sample mean to sample standard deviation is biased in small samples, and its value depends on batch size, so the loss is not comparable across configurations and the gradient is noisy in a way that scales with how short the batch is. Enlarging batches reduces the noise and reduces the number of updates.
• Variance reduction is the easiest way to improve the loss
  Because the objective is a ratio, shrinking the denominator raises it. The path of least resistance for the optimiser is to hold a low-volatility, highly concentrated combination that happened to be calm in the training window — which is a statement about the window, not about risk.
• Entropy regularisation is a weak defence against concentration
  As the worked example shows, a softmax head with a clean signal concentrates heavily even with an entropy term. There is no position limit unless one is imposed, and the usual fixes (weight caps, projection) reintroduce the constrained-optimisation machinery the approach set out to avoid.
• Nothing in the model is stationary
  Training assumes the mapping from features to good weights is stable. Relationships between assets change, and the network has no mechanism for noticing. Walk-forward retraining limits the damage and introduces its own problem: each retrain is a new model, so reported results are an average over models, and the variance across seeds is often of the same order as the differences being discussed.
• Opaque in a way that matters for governance
  There is no expected-return vector or covariance matrix to inspect, so there is no way to attribute a position to a view. “Why is this the largest holding” has no answer beyond the weights of the network. For anything subject to model validation or a risk mandate, that is a substantive obstacle, not a presentational one.
• Hyperparameters interact and the search overfits
  Cost coefficient, two regularisation weights, learning-rate schedule, window length, hidden size and dropout all interact, and each configuration is evaluated on the same short validation period. The resulting selection bias is rarely accounted for; deflating reported Sharpe ratios for the number of configurations tried is the minimum honest adjustment (Bailey & López de Prado, 2014).
• Costs are modelled as a constant
  A single proportional rate ignores spread variation, market impact, which assets are cheap to trade and the fact that impact grows with size. A model trained against a flat one-basis-point assumption will happily choose a turnover profile that a realistic cost model would reject.
• The volatility overlay is only as causal as its inputs
  Every term in the multiplier must be lagged, and the multiplier’s cap, target and estimator are three more choices that materially affect results. This is the single most common place where an otherwise careful implementation acquires look-ahead.

7. References

1. Bailey, D. H., & López de Prado, M. (2014). The deflated Sharpe ratio: Correcting for selection bias, backtest overfitting, and non-normality. The Journal of Portfolio Management, 40(5), 94–107. https://doi.org/10.3905/jpm.2014.40.5.094
2. Choueifaty, Y., & Coignard, Y. (2008). Toward maximum diversification. The Journal of Portfolio Management, 35(1), 40–51. https://doi.org/10.3905/jpm.2008.35.1.40
3. DeMiguel, V., Garlappi, L., & Uppal, R. (2009). Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? The Review of Financial Studies, 22(5), 1915–1953. https://doi.org/10.1093/rfs/hhm075
4. Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, 270(2), 654–669. https://doi.org/10.1016/j.ejor.2017.11.054
5. Harvey, C. R., Hoyle, E., Korgaonkar, R., Rattray, S., Sargaison, M., & Van Hemert, O. (2018). The impact of volatility targeting. The Journal of Portfolio Management, 45(1), 14–33. https://doi.org/10.3905/jpm.2018.45.1.014
6. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
7. Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem. arXiv:1706.10059. https://arxiv.org/abs/1706.10059
8. Markowitz, H. (1952). Portfolio selection. The Journal of Finance, 7(1), 77–91. https://doi.org/10.1111/j.1540-6261.1952.tb01525.x
9. Moreira, A., & Muir, T. (2017). Volatility-managed portfolios. The Journal of Finance, 72(4), 1611–1644. https://doi.org/10.1111/jofi.12513
10. Sharpe, W. F. (1994). The Sharpe ratio. The Journal of Portfolio Management, 21(1), 49–58. https://doi.org/10.3905/jpm.1994.409501
11. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998–6008.
12. Zhang, Z., Zohren, S., & Roberts, S. (2020). Deep learning for portfolio optimization. The Journal of Financial Data Science, 2(4), 8–20. https://doi.org/10.3905/jfds.2020.1.042

Disclosure

This article is written for educational purposes. It describes a modelling technique and provides reference code run on synthetic data; it is not investment research, a recommendation, or an offer to buy or sell any security. No performance is claimed or implied, and nothing here should be read as a prediction of returns. Any implementation decision remains the responsibility of the reader.