Smart beta investors might be underestimating tracking errors and maximum relative drawdown in their indexes when compared with the broader market, according to industry experts.
Annualised extreme tracking errors relative to a cap-weighted index can range from between 4% to 18% for different smart beta strategies, thinks Eric Shirbini, London-based global product specialist at Edhec Risk Institute's Scientific Beta index provider.
"These are massive risks that people are taking and they are not well documented when people replace their active managers with these smart beta strategies," said Shirbini, speaking at a recent Edhec Risk conference.
He calculates that one-year rolling tracking errors at the ninety-fifth percentile ranged from 3.9% for one quality-focused equity index to 18% for one maximum diversification index, in relation to a cap-weighted index of 2,000 developed-world equities over 10 years.
Investment consultants agree the potential size of tracking errors is not well documented in fund marketing materials, despite being a key risk indicator, saying investors often have to ask for the metrics themselves.
"The goal of the ETF [exchange-traded fund] may be to track a particular index… but that index might have a lot of tracking error to the broad market, and I think sometimes that may not be as clear as it could be," says James Price, London-based senior investment consultant at Willis Towers Watson.
Feifei Li, head of investment management at Research Affiliates, a smart beta index provider in California, says the firm strives to inform investors of the tracking-error risk for its low-volatility equity indexes, for example, which is one example of a high tracking-error strategy.
According to Scientific Beta, one minimum volatility index it looked at had a one-year rolling ninety-fifth percentile tracking error of 10.2% over 10 years, which corresponds to a maximum relative drawdown of 17.2%.
Li says that typically the tracking error would not be included in Research Affiliates' marketing material, but adds: "When prospective investors require that kind of information, we do provide that."
Investment consultant bfinance says it tends to calculate the tracking errors on behalf of clients. "If we ask questions about this and [providers] are not really going to answer, we would look at it ourselves," says Julien Barral, senior associate at bfinance in London.
If more smart beta index providers disclosed tracking error over the long term, investors would observe higher maximum relative drawdowns, Shirbini says.
Scientific Beta calculates that, compared with a long-term cap-weighted index of 500 US stocks, smart beta strategies have a maximum relative drawdown ranging from 30% for one maximum decorrelation strategy to 40.1% for an efficient minimum volatility strategy, over a 45-year period to 2015.
Shirbini says "very few" smart beta funds have an explicit tracking-error constraint in the design of their methodology. Barral at bfinance estimates that more than half of all smart beta strategies will have few constraints on how high the tracking error could get.
Research Affiliate's Li says there are no tracking error constraints in the design of Research Affiliates' strategies.
Chris Mellor, equities product manager at Source, an ETF provider in London, says the firm's smart beta products do include controls on equity sectors and underlying country and liquidity risk, relative to a cap-weighted index.
"High tracking error is not in and of itself a risk that I want to remove," says Willis Towers Watson's Price. "Using low volatility as an example, the relative risk is quite high. Tracking errors of 9-10% versus the broad market would not be surprising… But that might be part of delivering the strategy you want in your portfolio."
Equally if the market is down 20%, those low volatility strategies might be down 10%. Without high tracking error, such downside protection cannot be achieved, he thinks.
"This is not necessarily a bad thing as you get something for those risks. It's a question of whether the investor can tolerate that level of risk," says Shirbini.
Cap-weighted indexes are still the benchmark used by most investors, but fund managers question whether fixating on a stockmarket benchmark is worthwhile for smart beta investors.
"Measuring a tracking error versus an underlying benchmark is an odd way to think about it. You're already making the decision to follow a different strategy and you should be aware of the fact that a different strategy will have different performance," says Source's Mellor.
He adds that the benchmarks will be subjective for different investors. Others argue that tracking errors give a useful indicator of risk.
"Theoretically you shouldn't really compare them against a standard index as that's what [smart beta strategies] are trying to get away from… but at the same time it gives [investors] a base to compare it to," says Barral.