Fake data can help backtesters, up to a point

Quant investors often complain they have only a single version of history against which to test their ideas.

One way to get round the problem has been to make history up. Quants have done that for a long time already – using bootstrapping or Monte Carlo simulations to create alternative time series data for the backtests they run.

A new idea, though, is to employ machine learning techniques to invent wholly artificial data. Quants are experimenting with these models and say they can produce data indistinguishable in some cases from the real thing.

The potential of the new ‘fake’ data gives cause for optimism. With it, quants can test strategies against scenarios that might have happened as well as those that did. There’s a caveat, though. Fake data may fix some of the shortcomings of conventional backtesting, but it can’t fix all of them.

The models the quants are using to generate the new data effectively learn the process by which past data was generated.

That’s worked impressively outside investing, where the models have been used to create anything from deep fake videos to so-called Ganimals – synthesised animals, like an elephant crossed with a cat – conjured up using generative adversarial networks (Gans).

Amazon used synthetic data to train its Alexa bot to understand instructions in Hindi. Rather than train the voice recognition software with millions of real commands, the tech firm generated fake samples from data on just a subset of recordings.

But applications in financial markets face a key difference from applications in other such fields. Markets are fast-changing systems, subject at times to sudden, unexpected regime shifts.

Backtesting with multiple versions of history may be better than backtesting with one. But the generative models still are recreating a version of events learnt from the past. And even a richer view of history could be a poor guide to the future.

One quant draws a parallel with forecasting climate change, a process in which what’s gone before – by definition – will be largely redundant. And in the case of equity markets, “even the most intimate knowledge of history isn’t going to tell you where Apple’s stock price is going to be,” he says.

In another way, too, historical data could prove a bad teacher.

The mechanics of the market are hugely complex, including the actions and motivations of thousands of investors, companies and intermediaries and the complex dynamics of market microstructure. That’s before accounting for the influence of the global macro environment, news events, and so on.

It’s never guaranteed the data will lead a model to a full understanding of those mechanics. That’s to say, the training set may provide only a patchy representation of the truth. “You could end up generating fake data that’s just too simplistic for what’s at stake,” says another quant. “It could be counterproductive.”

These are limitations the data-generation models have not faced outside finance. They also are limitations that apply to any form of backtesting, be it conventional or using fake data. But as investors proceed with the new techniques, they will need to keep sight of the problems that fake data cannot solve. A picture of an elephant or a cat looks like a picture of an elephant or a cat for ever. A picture of a market is always changing.