There is a gap in most algo traders' testing process, and it costs people evaluations. You backtest a strategy in the Strategy Analyzer, the equity curve looks clean, so you put it on a funded account and wait. Then the fills come in at prices the backtest never showed you, the strategy takes trades you did not expect, and you are down a payout cycle trying to work out what changed.
The step almost everyone skips sits between those two points. NinjaTrader 8 ships with a Playback connection that replays recorded market data tick by tick, and it will run your NinjaScript strategy against that data as if the session were happening right now. It is not a backtest and it is not live trading. It is the only place you get to watch your bot make decisions on real order flow before real money is involved.
We run NinjaTrader algos on prop accounts, so this is the test we actually rely on. Here is how Market Replay works, what it catches that a backtest cannot, how to set it up correctly, and where it still misleads you.
What Market Replay Actually Is
NinjaTrader's Playback connection replays market data that has already been recorded or downloaded. Per NinjaTrader's help guide, "Playback supports running on Market Replay data or Historical data," and the two are not equivalent. Market Replay is the better of the two because it "holds both level I and level II (market depth) data." Historical data gives you bars. Market Replay gives you the tape.
That distinction matters more for an algo than for a discretionary trader. If your strategy reads volume, depth, or intrabar price movement in any way, replaying bars will not exercise the logic you actually wrote. Replaying the tick stream will.
NinjaTrader describes the feature as a way to "rewind and retrade real futures sessions tick by tick," and calls it an "evaluation-safe way to rehearse setups without risking a funded account." In their own prop trading materials they note that "charts, order ladder, and market depth all move together, just like a live session," and that replay can "run at up to 1,000 times normal speed."
For a strategy developer, that last part is the whole point. You can push a month of sessions through your logic in an afternoon and watch every entry as it happens.
Why a Backtest Cannot Tell You This
The Strategy Analyzer is genuinely useful, but it is worth understanding what it is doing under the hood when it fills your orders. NinjaTrader documents this openly. Their historical fill algorithm "breaks each historical bar into three virtual bars to mimic the movement of price within each bar's timeframe."
Three virtual bars. That is the resolution of a standard backtest fill. If your strategy is on a 5 minute chart, NinjaTrader is approximating five minutes of order flow with three synthetic price movements, then deciding whether your stop or your target got hit first.
You can improve this. NinjaTrader's Order Fill Resolution setting has a Standard mode that uses "the same Open, High, Low, Close, Time values that are available to the strategy," and a High mode that lets you "set a secondary bar series to be used as the price data to fill your orders." High resolution with a 1 tick series is far closer to reality.
But there is a catch that hits a lot of real strategies. NinjaTrader states that Order Fill Resolution cannot be used with Multi Time Frame or Multi Instrument strategies, and it cannot be used with Tick Replay. If your algo pulls in a second data series for a higher timeframe filter, which many prop firm strategies do, the High setting is off the table and you are back to three virtual bars per candle. That is exactly the situation where Playback earns its place in the process.
A backtest tells you whether the idea has an edge. Playback tells you whether your code behaves the way you think it does. Those are two different questions, and passing the first one does not answer the second.
This is closely related to a problem we have written about before. If you have ever wondered why your NinjaTrader 8 backtest does not match live trading, fill resolution is usually one of the first three culprits.
Getting Market Replay Data
You cannot replay data you do not have, and there are two ways to get it.
Record it yourself. Go to Tools, then Options, then the Market Data category, and check "Enable market recording for playback." From that point NinjaTrader records data for "any instrument in any NinjaTrader window that is receiving live market data." One detail people miss: capturing Level II depth requires an open Level II, SuperDOM, or FX Pro window for that instrument. If you want depth in your replay, you have to have a window pulling it.
Download it from NinjaTrader. Open the Control Center, go to Tools, then Historical Data, and expand the "Get Market Replay data" section. Pick the instrument and date and press OK.
Two limits are worth knowing before you plan a testing block. NinjaTrader states that it "offers a limited amount of Market Replay data free to download," and that "only the most common instruments are currently available." NQ and ES are fine. Something thinner may not be there at all, in which case recording it yourself is the only route, and that means planning ahead of the sessions you want to test.
Three download gotchas that waste an evening
NinjaTrader's documentation is explicit about all three. First, "enable market recording for playback must be disabled from the Market Data category of the Options menu before downloading replay data." Second, you must disconnect from the Playback connection before you download. Third, closing the Historical Data window cancels any download that is still running. Start the download, leave the window open, and walk away.
Once you have data, it lives on disk. Per the help guide, "Market Replay data is recorded and stored in compressed files located in the Documents\NinjaTrader 8\db\replay directory." That folder is worth including in your backups, especially if you have spent weeks recording sessions for an instrument NinjaTrader does not distribute.
Running Your Strategy on the Playback Connection
The setup itself is short. The order of operations is what trips people up.
Start by closing your other connections. NinjaTrader's guide is direct about this: "you must close all other connections before connecting to Playback." Then, per the same guide, "the Playback Connection can be accessed by left mouse clicking on the Connection menu within the NinjaTrader Control Center and selecting the Playback menu item." The Historical Data window comes up so you can confirm what data you actually have before you start.
Next, open your chart and set the replay start point. The Playback control has a slide control that "selects a point in time to start replay." Be careful with it once you are running, because NinjaTrader warns that "sliding during playback will reset the Replay101 account trade history." Scrubbing mid session wipes the results you were collecting.
Now the single most important detail for strategy testing, quoted directly from NinjaTrader's Playback documentation: "Should you be using the Playback for testing a NinjaScript strategy, please be sure the chart you apply the strategy onto has bars populating it prior to the start time of your replay."
If your chart starts at the same moment your replay starts, your indicators have no history to calculate from. A 200 period moving average needs 200 bars before it means anything, and a strategy that guards on CurrentBar will sit there doing nothing while you watch and assume the logic is broken. It is not broken. It is starved. Load the chart back far enough that every series in your strategy is warm before the first replayed tick arrives.
Finally, set your speed. Each successive click on the speed control increases playback speed, and NinjaTrader notes that "playback of 'Max' will process data at fastest possible speed." Run at Max for coverage, then drop to normal speed to watch a specific entry in detail.
One thing that looks like a bug and is not: the interface "only visually updates in 1-second intervals for performance optimizations," but NinjaTrader confirms that "orders, indicators, and strategies will calculate just as they were running in real-time." The display is throttled. Your strategy is not.
The Three Testing Layers at a Glance
Backtest, playback, and live simulation are not competing options. They answer different questions, and a serious testing process uses all three in order.
| Factor | Strategy Analyzer backtest | Playback with Market Replay | Live sim on real-time data |
|---|---|---|---|
| Data granularity | Bars, filled via three virtual bars per bar | Recorded tick stream with Level I and Level II | Actual live tick stream |
| Speed | Fastest, runs years in minutes | Adjustable, up to Max | Real time only |
| Repeatable | Yes | Yes, the same session replays identically | No, every session is different |
| Network latency modeled | No | No, orders process immediately and synchronously | Partly, plus real feed behavior |
| Best for | Screening ideas and parameter ranges | Verifying that the code behaves correctly on real flow | Final soak test before funding |
What Playback Still Will Not Tell You
Playback is a better mirror than a backtest, but it is still a mirror. Two limits deserve your attention.
The first is execution realism. NinjaTrader states that orders submitted to the playback account "are processed immediately and synchronously," and that "simulated internet latency delay simulation is not present in playback." That is a deliberate design choice, because it makes strategy testing reproducible. It also means every fill you see in Playback is optimistic compared to what a real broker round trip will give you. If your strategy's edge depends on getting filled at a specific tick, Playback will flatter it. Budget for slippage on top of whatever Playback reports.
The second is you. NinjaTrader's own disclosure on the subject puts it plainly: "emotional and psychological factors of real money risk are not replicated," and "simulated trading does not represent actual trading and is based on hypothetical conditions." For a fully automated strategy that matters less than it does for a discretionary trader, but it still matters. Watching a bot take a four handle drawdown in replay and watching it take the same drawdown on a funded account are not the same experience, and traders who have only done the first one tend to intervene during the second.
A Practical Playback Routine for Prop Firm Algos
Here is how we sequence it when a strategy change needs validating before it touches a funded account.
Step one, backtest for direction. Use the Strategy Analyzer to confirm the change did not break the edge across a long sample. Do not read the fill prices as gospel. This step is a screen, not a verdict. If you are tuning parameters here, do it with walk-forward optimization rather than a single in-sample run, or you are just fitting the curve.
Step two, replay the ugly days. Do not replay a clean trending session and call it validated. Pull replay data for the days that would have hurt: a CPI morning, a chop day, a session with a violent open reversal. Those are the sessions where a logic bug shows up as a trade you never intended.
Step three, watch one session at normal speed. Max speed is for coverage. Once per change, sit and watch a full session play out at real speed with the strategy running. You will catch things no report shows you, like an entry firing twice, a stop that never moved, or a flatten that fires a second later than you assumed.
Step four, check the rule edges. If you are trading a funded account, replay a session that pushes your strategy toward the daily loss limit and confirm your protective logic actually engages. Prop firm accounts get closed for rule breaches, not just for losing money, and the rules vary by firm. Our breakdown of prop firms that allow automated trading covers which ones let a bot run at all.
Step five, sim on live data. Only after replay is clean do you put the strategy on a real-time simulated account and let it run for a stretch of live sessions. That is where feed hiccups, connection drops, and genuine latency show up.
Skip the Build, Keep the Testing
NQ Ultra is a NinjaTrader 8 strategy built for prop firm rules, with session controls and contract sizing ready out of the box. Load it on the Playback connection and watch it work before you ever fund an account.
Get NQ UltraThe Test That Pays for Itself
Market Replay does not make your strategy profitable. It makes your strategy honest. It is the only stage in the process where you can see your own code react to genuine order flow, repeat the exact same session as many times as you need, and fix what you find before an evaluation fee is on the line.
Most traders who blow an account on an algo did not have a bad idea. They had an untested implementation of a decent idea. An afternoon in Playback is cheaper than a reset, and far cheaper than a funded account.