Backtesting vs Real-Time Trading: Why There’s A Discrepancy and How To Fix It

In life, we all look to the past as an indicative measure of the future. The same can be said for investments, where many financial managers conduct backtests to use past events to gain experience and make better choices in the future. 

But then, why does every investment sheet contain the same words of warning: past performance is no guarantee of future results? 

The answer lies in the fact that the market is ever-changing and that no two scenarios can ever be exactly the same.

Even so, backtesting your investment strategy gives you a great indication of how it would work in certain scenarios, arming you with the knowledge you need for whenever a similar situation arises.

Sadly, however, many financial managers face discrepancies in their backtesting that goes well beyond the fact that the market is ever-evolving... Discrepancies that could easily be avoided with the right software solutions!

What Is Backtesting?

Backtesting is the action of seeing how your investment strategy or model would have fared in a past scenario. 

This looks at how an investment strategy would have behaved under certain conditions and how specific events – such as breaking news or the reaction of investors to that breaking news – influenced the price of certain securities.

This is a great resource for any financial manager looking to understand how market forces work, but it’s important to remember that backtesting isn’t a flawless procedure. 

But this isn’t just because the market then wasn’t quite like the market now, but because the way most software conducts backtests is somewhat inaccurate. 

The Seven Factors That Negatively Impact Backtesting Results

1. Incomplete or Insufficient Data: Many strategies were designed years ago and are tested on daily data that has just four points: the daily open, high, low, and close. When you have just these four points to work with, you and the software must make assumptions about what happened in between. 
2. Book prices: While the bid-and-ask book provides managers with the market’s liquidity at every moment of the day, many backtests are conducted using only the filled price history. This means that many managers miss out on some crucial information about large spreads and low liquidity.
3. Inconsistent Software Across Live Trading And Backtesting: For software to run financial backtests, it needs to create an environment where it can run calculations against historical data. Nevertheless, the environment this takes place in will be different for users who trade using various types of software, which may or may not give you access to the same features, data, and so on. In other words, not running backtests and live trading on the same software can never return accurate results.
4. Execution Latency: When backtesting, the time between order generation and execution is all based on assumption because there aren’t the usual backlogs managers can experience during real-time trading. In other words, properly simulating the execution in backtesting requires consistent values from live trading.
5. Calculation Latency: Calculations in real-time may sometimes carry a different timestamp than those in the backtest. This may not seem like a problem to most, but it may actually affect the behaviour of investors and result in a difference in executed prices.
6. Trade Size: When backtesting, it is essential to test the system using trade sizes you’d afford in real life. This gives you a better idea of what to expect during live trading while also making you aware of potential slippage costs. (Read our Quick Guide To Performing A Slippage & Liquidity Analysis to find out more).
7. Risk Restriction: Suppose the backtesting environment isn’t capable of applying all the same checks found in the live setting. In that case, your risk pre-trade and post-trade checks will generate differently-sized exposures and potentially affect your performance when trading in real life.

So, should all financial managers just call it a day and never run another backtest again? Definitely not!

As our experience, many managers at global banks and world-leading financial institutions commission developers to code financial software that can conduct backtesting and live trading within the same environment. 

Here’s what this expensive, state-of-the-art software does differently… 

Here’s What Great Software Does To Minimise Differences Between Backtesting vs Real-Time Trading

• Data Time-Frame: Backtesting is a simulation of the live environment with accelerated data. This is particularly hard to create with only four data points, especially if you’re looking at how the market performed over a 48-hour period (i.e. with just eight points). Instead, the best software in this category collects the opening, high, low, and closing data of every minute of every trading hour. So, for markets that are open 10 hours a day, that would result in 4,800 price points over just two days. That naturally results in more accurate assumptions. 
• Synchronisation: To properly simulate live trading, backtesting must synchronise intra-day all the processes involved. In fact, such software can run portfolio evaluations, position sizings, and risk checks using the same processes applied to live trading. 
• Book Prices: By managing multiple price time series, such software can run backtests with accurate bid-and-ask prices. Once again, this helps it come up with more accurate execution price predictions. 
• Manage Timezone: Anyone who’s ever conducted backtests knows that managing the timezone on data can be a real headache, especially since different time zones have different daylight rules. This type of software can automatically handle all these differences to help you avoid making mistakes in your trading.
• Latency: How long does it take to process the data you’ve received, apply your specified rules to it, and then execute investment orders? Latency in this process can cause the price of a security to change, leading to negative or positive slippage. Either way, this will mean that your backtesting results will be incorrect, which is why the best backtesting software offers managers minimise latency.

Affordable Software That Reduces the Backtesting vs Real-Time Gap

As we have been discussing in this series of blogs related to Event Stream Processing, our collaboration with TIBCO™ has resulted in the creation of some fabulous software frameworks that can revolutionise how financial managers work. 

These are namely:

• CYBMIND, which is our strategy automation software;
• DEEPMODE, which is our complex data modelling software; and 
• NEXTVIEW, which is our predictive modelling software.

Together, these three frameworks give you the ability to 

• collect data from a multitude of sources, 
• analyse it, normalise it, and enrich it to your specifications, and
• automate your trading ideas.

This may sound simple, but it has helped revolutionise many SMEs’ financial strategies, especially since it removes the backtesting vs real-time trading divide almost completely.

The best part is that the way it was programmed allows for faster financial software development that costs much less than what large corporations pay. 

Called Event Stream Processing, this technology comes with numerous benefits for SMEs, which include multiple financial streaming adapters that can save you time and lead to better results when backtesting your strategy.

So, what are you waiting for? Get in touch with us to find out how to get your hands on our Event Stream Processing frameworks.