Survivorship bias: when only the winners are left
Survivorship bias arises when only what has survived until today goes into the calculation. Liquidated funds, bankrupt companies and abandoned strategies are missing, and they were usually the worst. The result then looks better than it was. In this article we calculate how large the distortion gets, where it hides and how to avoid it in your own backtests.
What survivorship bias is
Imagine a list of all equity funds that exist today and calculate their returns over the last ten years. It sounds like a fair comparison, but it is not. Funds that did poorly during those ten years were often merged into other funds or closed altogether. They no longer appear on today's list. What remains are the funds that were good enough to survive. Their average is higher than the average of all funds that were available ten years ago.
That is survivorship bias: a sample contains only the survivors of a selection process, and the selection is related to the very outcome you want to measure. For investors this is dangerous because the distortion always points in the same direction. It makes returns look too high, risks too low and success rates too good.
The point in time of the selection matters. A list you put together today contains knowledge from the future of the starting year: you already know which funds and stocks made it. An investor who invested ten years ago did not know that.
A worked example: ten funds, three disappear
Ten funds start at the same time. Over five years they earn the following returns per year. The three weakest are closed during this period and are missing later from any database that only contains today's funds. The benchmark index returns 5% per year.
| Fund | Return p.a. | still around today? |
|---|---|---|
| A | 9% | yes |
| B | 8% | yes |
| C | 7% | yes |
| D | 6% | yes |
| E | 5% | yes |
| F | 4% | yes |
| G | 3% | yes |
| H | −2% | no |
| I | −4% | no |
| J | −6% | no |
Numbers from the worked example above, heights to scale.
The example is deliberately simple, but the direction is always the same: counting only survivors raises the average and the success rate. Risk also looks smaller because the funds with the largest losses are missing. The dispersion of the seven survivors is much smaller than that of all ten funds.
Where survivorship bias hides
- Fund rankings: Rankings over ten or twenty years only show funds that still exist today. Liquidated and merged funds are missing. Studies of active funds therefore need databases that also contain dead funds.
- Backtests with today's index members: If you backtest the 30 current members of a blue-chip index over 30 years, you are calculating with companies that became large and successful. The firms that dropped out of the index or disappeared in that time are missing. The index itself is not affected: at every point in time it contained the members of that time.
- Stock databases: Stocks that are delisted, for example after a bankruptcy, often lack a final return in the data. Without this last, usually large loss, strategies with many small and weak stocks look better.
- Hedge fund and strategy databases: Funds report voluntarily. Those that do poorly stop reporting; those that join often bring their good track record with them.
- ETFs and thematic funds: ETFs get closed too, especially small thematic ETFs after a hype. If you compare today's most successful thematic ETFs, you do not see the ones that were shut down.
- Success stories: Books about successful investors and companies tell the stories of those who made it. How many failed with the same strategy is rarely mentioned.
Related biases
Survivorship bias rarely comes alone. Several similar errors arise when data look different in hindsight than they did at the time:
| Bias | What happens | Effect |
|---|---|---|
| Survivorship bias | Only funds or stocks that exist today are in the data | Return too high, risk too low |
| Delisting bias | Final return before delisting is missing | Losses of small companies are missing |
| Backfill bias (instant history) | New funds bring their good track record into the database | Early years too good |
| Look-ahead bias | Information is used before it was known | Signals look more accurate |
| Selection bias | Selection by a criterion that is related to the outcome | Result skewed toward the criterion |
Look-ahead bias is the closest relative: a backtest buys on December 31 based on a balance sheet that was only published in March. Survivorship bias is a special form of it, because the knowledge of which companies would survive was not available at the time either. More on errors in backtests in the article on backtest overfitting.
What the research shows
Brown et al. (1992) showed that survivorship bias not only shifts the average but can also create apparent persistence: when bad funds disappear, it looks in hindsight as if good funds stayed good. Anyone who wants to measure persistence in performance therefore needs data with all funds, including the liquidated ones.
Elton et al. (1996) tracked all funds that existed at the end of 1976 and, when funds merged, continued the return calculation using the terms of the merger. This allowed a precise estimate of the bias. Their starting point: funds usually disappear because of weak results. Malkiel (1995) used data on all equity funds existing each year and found that survivorship bias is more important than earlier studies had estimated. Carhart (1997) used such a database for his study of persistence in fund returns.
For stocks, Shumway (1997) showed that in a large US stock price database no correct final return was available for most stocks delisted for negative reasons, and that these missing returns are large. For hedge funds, Fung and Hsieh (2000) distinguish natural biases, which follow from the composition of the fund universe, from spurious ones, which arise only from the way the data are collected.
Advanced: how large the bias gets
If you split all funds into survivors (share s) and liquidated funds (share 1 − s), the overall average is a weighted mean. The bias, the difference between the average of the survivors and that of all funds, follows directly:
Bias = r̄_S − r̄_all = (1 − s) × (r̄_S − r̄_L)- s
- share of survivors
- r̄_S
- average return of the survivors
- r̄_L
- average return of the liquidated funds
Three rules follow. First, the bias grows with the length of the period, because the longer it is, the more funds disappear. Second, it is largest where results vary widely: more for small stocks, thematic funds and hedge funds than for broad index funds. Third, it affects metrics differently. Average returns rise, volatility and maximum drawdown fall, and success rates such as the share of funds above the index rise especially sharply, because it is precisely the losers at the bottom that are missing.
Another consequence concerns tests of skill. If only funds with a good track record survive, the sample makes it look as if funds had persistently good or bad managers. Anyone who wants to separate luck from skill must therefore calculate with all funds that were available at the start.
How to protect yourself
- Ask about the data: Does the study or ranking also include liquidated funds and stocks? If it says survivorship-bias-free or something equivalent, that is a good sign.
- Compare with the index, not with the fund average: An index contains the members of the time at every point. A fund average built from today's funds does not.
- Select with the knowledge of the time: Decide which rule you would have used to select at the starting point, and only then calculate. Do not take today's winners and place them in the past.
- Prefer broad building blocks to single names: A backtest with a broad index ETF is hardly affected. One with ten stocks that are well known today almost always is.
- Be skeptical of long rankings: The longer the period and the smaller the fund group, the more funds are probably missing.
The same caution applies to stock picking and to stories about investing legends: the strategy that led to a famous success is not automatically a good strategy. It only survived in this one case.
How Portfolio Lab calculates
The Backtest calculates with exactly the securities you select. If you pick today's well-known winners, survivorship bias sits in your selection, not in the calculation. This also applies to the templates, which contain today's funds and index members. The securities database and the fund ranking contain funds that exist today; liquidated funds are missing there. A ranking over ten years therefore only compares survivors.
The long series under Asset classes are less affected: factor returns according to Fama and French are formed from all stocks listed at the respective time, and index data from the index members of the time. If you want to test a single security over a long period, you can therefore compare it with a matching asset class. The backtest methodology describes where prices come from and how gaps are handled.
Frequently asked questions
What is survivorship bias in simple terms?
A distortion that arises because only the survivors are counted. Funds, stocks or strategies that were abandoned are missing from the data. Because they usually did poorly, the rest looks better than the whole group really was.
How large is survivorship bias in mutual funds?
It depends on the period, the fund group and the share of liquidated funds. Studies find noticeable differences for active equity funds between data with and without dead funds. The longer the period and the more results vary, the larger the bias.
Does survivorship bias affect ETFs too?
Yes, when you compare rankings of today's ETFs, because ETFs get closed too, especially small thematic ETFs. Broad index ETFs themselves, however, track their index with the members of the time.
Is the MSCI World free of survivorship bias?
The index return itself, yes: at every point in time the index contained the members of that time, including those that later dropped out. Only a backtest that takes today's index members back into the past is biased.
What is the difference from look-ahead bias?
With look-ahead bias, a backtest uses information that was not yet known at the time, such as later financial statements. Survivorship bias is a special case: the knowledge of who survives was not known at the time either.
How do I avoid survivorship bias in my own backtest?
Select the securities by a rule you could already have applied at the starting point, use broad indexes or asset classes instead of today's single names, and be skeptical of long fund rankings.
Literature
- Brown, Stephen J.; Goetzmann, William; Ibbotson, Roger G.; Ross, Stephen A. (1992): Survivorship Bias in Performance Studies. Review of Financial Studies 5 (4), pp. 553–580. doi:10.1093/rfs/5.4.553
- Carhart, Mark M. (1997): On Persistence in Mutual Fund Performance. The Journal of Finance 52 (1), pp. 57–82. doi:10.1111/j.1540-6261.1997.tb03808.x
- Elton, Edwin J.; Gruber, Martin J.; Blake, Christopher R. (1996): Survivor Bias and Mutual Fund Performance. Review of Financial Studies 9 (4), pp. 1097–1120. doi:10.1093/rfs/9.4.1097
- Fung, William; Hsieh, David A. (2000): Performance Characteristics of Hedge Funds and Commodity Funds: Natural vs. Spurious Biases. The Journal of Financial and Quantitative Analysis 35 (3), pp. 291–307. doi:10.2307/2676205
- Malkiel, Burton G. (1995): Returns from Investing in Equity Mutual Funds 1971 to 1991. The Journal of Finance 50 (2), pp. 549–572. doi:10.1111/j.1540-6261.1995.tb04795.x
- Shumway, Tyler (1997): The Delisting Bias in CRSP Data. The Journal of Finance 52 (1), pp. 327–340. doi:10.1111/j.1540-6261.1997.tb03818.x
This article explains a method; it is not investment or tax advice. Worked examples are simplified and rounded. All terms in the glossary.