Several times I have observed that when running my orderflow indicators live the data differs from historical data and from market replay. While market replay and historical data is basically identical. A big issue that I noticed is that historical and market replay tick data is binned every 4ms. And most likely all ticks within 4ms may or may not be ordered correctly.
Has anyone ran into this problem?
I am using ninjtrader brokerage live data.
Example tick timestamps:
20260714 040000 0280000
20260714 040000 6200000
20260714 040001 0640000
20260714 040007 4560000
notice how the final part has the millisecond timestamps. There are no timestamps that are not divisible by 4. I checked in a python script and this is true for hundreds of thousands of ticks.
@Daniel_S Actually, this has long been the case. You might want to read this: Timestamp Issues With Kinetick Data - Live versus Historical - NinjaTrader Support Forum
The situation was reported, culminating in “We have the behavior reported to our development but we do not yet have a tracking ID or any further information at this time.” That was four years ago.
I figured one of the most plausible explanations is that on the back end they are storing a fractional second timestamp in a single byte to save storage space. 1/256 of a second is about 4 ms, which matches up with the observation. Space is cheap, but it probably wasn’t as much so when this decision was made years ago.
Because of the relationship with DTN IQFeed (Kinetick) it’s quite possible the quantization of timestamps is happening there before it arrives at NinjaTrader.
2 Likes
I see that would make sense. I found the forum post that you shared as well and I checked the ticket number 15299 which says it was fixed in this patch note:
So I figured something was wrong with my end but I guess that is not the case. Do you know if switching data providers to rithmic should solve this problem? Or if Ninja still adjusts their data anyway.
Rithmic does not have historical data - it would come from NinjaTrader’s servers.
I see, thats a bummer. Do you know of any way to deal with this issue then? I sent
them a support email and I will see what they say.
I would suspect it would involve buying the historical tick data elsewhere, but yes, let’s see what platform support says. Be sure to email platformsupport@ninjatrader.com about this if you haven’t already as they don’t respond back here on the community forums.
Yes I sent them an email and I will see what they say. Thanks for the help.
What did they say ? I’m a Rithmic user and looking for a solution too
They opened a ticket and are looking into it. So just waiting for now.
The 4 ms timestamp pattern is interesting, but I think the bigger issue for order-flow work is whether events that land inside the same timestamp bucket still preserve the sequence you need for the calculation.
If your indicator depends on the exact order of trades — especially for bid/ask classification, delta changes, or other event-by-event logic — historical data can look internally consistent while still not reproduce the same state you saw live.
One test I’d try is to capture a short live session yourself and keep the incoming events in their original sequence.
Then compare three runs over exactly the same period:
-
the values produced while the data was live;
-
the values rebuilt from your own captured event stream;
-
the values rebuilt later from NinjaTrader historical data.
If #1 and #2 match but #3 doesn’t, you’ve isolated the problem to the historical input rather than the indicator logic.
The impact also depends on the calculation. A profile that mainly accumulates volume-at-price may tolerate timestamp quantization better than logic that depends heavily on the exact sequence of bid/ask events.
Did NinjaTrader ever come back to you with anything useful on the ticket they opened? I’d be interested to know whether they confirmed where the 4 ms grouping is introduced.
Not yet I am tempted to reach out again. I am leaning towards the ordering can be messed up too based on some logic that I have that needs exact temporal trade ordering.
That would make me treat this as a sequencing problem rather than just a timestamp-resolution problem.
If your logic genuinely depends on exact trade ordering, then two or more events sharing the same 4 ms timestamp become problematic if the historical store no longer preserves the same order in which they were delivered live.
If you follow up with NinjaTrader, I’d try to give them one very small reproducible example rather than just the general observation.
Capture a short live window and log each event with:
Then reload that exact period from historical data and compare the events inside one of those 4 ms buckets.
The local sequence number obviously only proves the order NinjaTrader delivered to your code live — not some absolute exchange-side ordering — but if the same group of trades later comes back in a different order historically and that changes your calculation, that would be a very strong reproducible case.
The especially useful example would be one where changing the ordering actually changes your signal, delta calculation, or other output.
If they give you a tracking/ticket ID or explain where the 4 ms grouping is introduced, definitely post it here. I’d be interested in the answer too.
According to my research, the market makers use that data to coordinate the price action as its developing. Therefore, they want to make sure none of that data ends up in any way traceable. In other words, its disposable on purpose.