Backtesting accuracy

Desktop SDK - Strategies
1

Hello, I’m developing a 5/15 minute strategy and need to verify the exact execution times (down to the minute and second) during backtesting for specific candles. I have Tick Replay enabled to mimic real-time conditions. What is the best way to view these precise timestamps? Thanks in advance!

2

I’m not sure there is a way to get a specific tick time when using bar level resolution. TickReplay is not really designed to improve analyzer accuracy. In my experience, the best way to get this level of precision in an analyzer run is to manually add a tick data stream and perform your executions there.

TickReplayNote

Developing for tick replay

OnStateChange()
{
    if (State==State.Configure)
        AddDataSeries(BarsPeriodType.Tick, 1);
}
OnBarUpdate()
{
    if(BarsInProgress == 1)
    {
        if(wantEntry)
        {
            Print($"Fill tick time ={Time[0]}");
            EnterLong(BarsInProgress,quantity, signalName);
        }
    }
}
2 Likes
3

So given the documentation you have provided, it would be inaccurate to say - “Strategy Analyser is less accurate than replay” ?, it seems to me adding the tick data series is supposed to make strategy analyser as accurate as market replay.

4

I think accuracy is dependent on how a strategy is coded. While the method I posted works great for market orders, limit orders are still evaluated by the analyzer on a per bar basis. So if your strategy is placing all orders as market orders (including stops and targets) the analyzer can be extremely accurate. If you’re using limit orders for anything they will have the same inherent limitations the analyzer normally has. Market replay is closer to live and removes some of those limitations, but has flaws of it’s own. Ultimately, for perfect accuracy, nothing beats live testing in a sim account.

2 Likes
5

Maybe use the ALERT LOG. (select NEW and find the ALERTS LOG)
I use it manually in REPLAY MODE and get every signal and the exact time (hours, minutes and seconds) and the exact price for each trade.
You can later export the log file to excel.
So you see with the highest accuracy possible the exact moment and price.

1 Like
6

This is really good to know, I use EnterLong and EnterShort API to enter a trade, and ExitLong and ExitShort to close them, I configure a global stop loss and take profit in State.DataLoaded with a huge value, just as a security blanket incase my VPS crashes using API
SetStopLoss(CalculationMode.Ticks, staticTPSL);
SetProfitTarget(CalculationMode.Ticks, staticTPSL);

So in my case, what I have understood based on my limited knowledge is that if I backtest my strategy I will get similar or more accurate results using strategy analyser rather than market replay.

1 Like
7

Yes. Strategy Analyzer uses “virtual bars” to simulate more realistic intra-bar fills. You can update the order fill resolution to high or add a tick data series. For the tick data series, you will trigger your strategy on the primary data series [0], but enter in the tick data series. This will give you better intra-bar fills, but Market Replay is meant to simulate as if you are sequentially receiving each tick (depending on the speed you choose).

https://ninjatrader.com/support/helpGuides/nt8/NT%20HelpGuide%20English.html?backtest_a_strategy.htm

2 Likes
8

Thank you all for responding.
If I use tick replay it would not let me use order fill resolution.
With replay turned on, do I still need to use trading on tick series within the code? I will use Alert log to output the exact times and see if that is realistic. I do understand nothing comes close to simulated live env. Thank you again.

9

I’ve never used the order fill resolution myself, since I’ve always used tick replay with a tick data series. From my understanding, it is essentially doing the same thing, which is why they give you that error. I think it assumes that you have the tick data and included it in your code. You still need the actual tick data in the historical data. So if you download the minute data then it won’t work as you expect. The closes to the real live environment is to run Market Replay in 1x speed. :joy: Ideally, back test with the strategy analyzer, if it looks good then run it through Market Replay. Most of the time I’m ok with 6 to 10x speed. Some people have mentioned that it doesn’t matter, but I’ve noticed slippage for myself if I use 50x+ speed. It might just depend on your strategy too. I’m assuming they start to batch process rather than run each tick sequentially as you bump up the speed.

1 Like
10

In PLAYBACK, I use HISTORICAL DATA.
The database you use should be tick data.
If you would use a 5 min chart, the alert will be triggered when the signal for an order is generated on that chart. So it will be somewhere within some 5 min bar. In the ALERTS LOG file you will see the exact time when, within that 5 min bar, the signal was generated.
Put PRIORITY at HIGH for fastest reaction to the signal.

2 Likes