Efficient frontier: Markowitz portfolio optimization

In depth8 min read · 14 sources

The efficient frontier connects all portfolios that offer the highest expected return for their level of risk. Harry Markowitz described the principle in 1952 and thereby founded modern portfolio theory: what counts is not the individual investment, but how it interacts with the others. In practice, however, the optimization is notoriously sensitive. If you use it, you should know its weaknesses.

The idea: return and risk together

Markowitz (1952) proposed judging portfolios not by expected return alone but by return and variance together. A portfolio is efficient if no other portfolio offers a higher expected return for the same risk, or less risk for the same return. In a chart of volatility (horizontal) and expected return (vertical), the set of these portfolios forms a curve that bulges to the left. Seven years later, with Markowitz (1959), he presented a book version with computational methods for many assets.

The expected return is simply the weighted average. The variance, by contrast, depends on all pairs, and because the correlation is usually below 1, the portfolio's volatility is lower than the average of the individual volatilities. This is the mathematical core of diversification.

An example with two assets

Equities with an expected return of 7% and 16% volatility, bonds with 2% and 5%, correlation 0.1. Mixing the two produces a curved line. The curvature comes from the correlation.

The pure bond portfolio is not efficient: with 6.4% in equities, risk actually falls and return rises. Nothing below the minimum-variance portfolio lies on the efficient frontier. For two assets, there is a closed-form formula for it:

Substituting the values: (0.0025 − 0.0008) / (0.0281 − 0.0016) = 0.0017 / 0.0265 = 6.4%. The formula also shows why the minimum-variance portfolio needs no return estimate: it only requires volatilities and correlations.

Tangency portfolio: the highest Sharpe ratio

If there is also a risk-free asset (here 1%), the portfolio with the highest Sharpe ratio is special: any mix of it and instant-access savings is better than any other portfolio with the same risk.

Tobin (1958) drew a far-reaching conclusion from this, the separation theorem: all investors with the same expectations should hold the same portfolio of risky assets. Their different appetites for risk show only in how much of it they hold and how much they keep in the safe asset, or whether they leverage it with borrowed money. The straight line from the risk-free rate through the tangency portfolio is called the capital market line. It was precisely from this idea that Sharpe (1964) developed the CAPM.

The problem: estimation error

The theory assumes that expected returns, volatilities and correlations are known. In reality, they are estimated, usually from the past, and the optimization reacts extremely sensitively to these estimates. Michaud (1989) therefore called mean-variance optimizers error maximizers: they overweight precisely those assets whose return happened to be overestimated and whose risk was underestimated.

Best and Grauer (1991) describe exactly this pattern: even a small change in the expected return of a single asset can eliminate half of the securities from the optimal portfolio, while its return and risk barely change. According to Chopra and Ziemba (1993), errors in expected returns matter many times more than errors in variances and covariances. And among 14 optimization models, DeMiguel et al. (2009) found none that reliably beats simple 1/N equal weighting out of sample. The gain from optimal diversification is lost in the estimation error.

Remedies: more robust optimization

• Minimum variance: needs only volatilities and correlations, no return estimate.
• Weight limits: for example, at most 40% per asset and no short sales. Jagannathan and Ma (2003) show that such limits act like shrinkage of the covariance matrix and can reduce risk, even if, strictly speaking, they are wrong.
• Shrinkage of the covariance matrix: pulling extreme estimates toward a simple target (Ledoit-Wolf, see below and covariance estimation).
• Risk-based methods: risk parity with equal risk contributions (Maillard et al. 2010) or the maximum diversification portfolio of Choueifaty and Coignard (2008), which maximizes the ratio of weighted individual volatility to portfolio volatility.
• Other risk measures: minimum CVaR following Rockafellar and Uryasev (2000) directly optimizes the average loss in the worst 5% of months instead of the variance.
• Black-Litterman: starts from market weights and adds your own views, each with a level of confidence.
• Rolling tests: run the optimization only with data up to each point in time and measure the following years (walk-forward).

Ledoit and Wolf (2004) base this on the estimation error of the ordinary sample covariance matrix, which distorts precisely the weights of an optimizer. Their proposal: shrink the matrix toward a simple target, for example equal variances without correlation, with an intensity that can be estimated from the data.

Advanced: the mathematics of the efficient frontier

Without weight limits, the efficient frontier can be derived in closed form. Merton (1972) showed that in a chart of volatility and return, it is a hyperbola. With the quantities A = 1ᵀΣ⁻¹μ, B = μᵀΣ⁻¹μ, C = 1ᵀΣ⁻¹1 and D = BC − A², the minimum variance for a target return μ_P is:

The formula for the tangency portfolio explains the instability: the inverse Σ⁻¹ amplifies small differences between similar assets. If two funds are highly correlated, the matrix is almost singular, and tiny differences in return produce large offsetting positions, as in the example above. With weight limits, there is no longer a closed-form solution; a numerical optimizer then calculates the frontier point by point.

A second point concerns the target quantity: the optimization maximizes the expected arithmetic return of a single period. If you invest over many years, you care about the geometric return, which is considerably lower when volatility is high. Portfolios at the right-hand end of the efficient frontier therefore look more attractive than they are in the long run.

Limitations of the theory

• Variance as risk: mean and variance only describe a distribution completely if it is normal. Fat tails and skewness are not taken into account.
• A single period: the model plans for one period. Contributions, withdrawals and reallocations over many years require extensions or a Monte Carlo simulation.
• Stable parameters: correlations rise in crises, and returns change with the market environment. What was efficient yesterday may not be tomorrow.
• Historical returns as a forecast: they are the most uncertain ingredient. Better sources are valuations such as the Shiller P/E (CAPE) or equilibrium returns.

How Portfolio Lab calculates

The Optimization page estimates expected returns as the mean of the monthly returns times 12, and the covariance matrix either as the sample matrix or with Ledoit-Wolf shrinkage toward a scaled identity matrix. By default, weights lie between 0 and 100%; minimum and maximum weights can be set for each ticker. Eight objective functions are available: maximum Sharpe, minimum variance, risk parity, maximum diversification, minimum CVaR (95%, as a linear program over the historical months), maximum Sortino, minimum variance for a target return and maximum return for a target volatility. The efficient frontier consists of 40 points between the minimum-variance portfolio and the highest achievable return. The rolling test re-optimizes every 1, 3, 6 or 12 months, each time with the last 24 to 180 months (default: 60 months, annually). Details in the optimization methodology.

This article explains a method; it is not investment or tax advice. Worked examples are simplified and rounded. All terms in the glossary.