Expert advisors: what a backtest can and cannot show

A robot’s backtest climbs smoothly for ten years without a losing year. The seller says it took two years to build. It could equally have taken an afternoon of adjusting settings until the past looked like that.

What a backtest is

An expert advisor (EA) is a program that trades by itself inside a platform such as MetaTrader. It follows fixed rules for entry, stop and exit. A backtest runs those rules over past prices and reports what would have happened, using whatever spread and costs were typed into the test. It is a simulation of the rules on that data.

In-sample and out-of-sample

In-sample data is the stretch of history used to build and tune the rules. Out-of-sample data is a later stretch the rules never saw while they were being tuned. Only the second tells you anything about the future.

With enough settings, any rules can be made to fit the past. That is curve fitting. Signs to look for:

• Many inputs, such as a dozen settings to adjust.
• Oddly precise values: RSI 13, a 37-pip stop, trading only on Tuesdays between 09:00 and 11:00.
• Results that collapse when one setting is nudged by a step.
• A test that starts and ends on convenient dates.

A sounder approach is walk-forward testing: tune on one period, test on the next, then roll both forward and repeat.

Try it

StrategyTrend (MA cross)BreakoutRange fadeRSI reversal

MarketTrendingRangingMixed

New prices

The rules

• Prices: 240 simulated closes starting at 1.0850, moving 8–12 pips a bar at random. Trending: a steady drift that changes direction once or twice. Ranging: always pulled back towards one level, staying within roughly 60 pips either side. Mixed: range, then trend, then range.
• ATR (a measure of normal movement): average size of the last 14 bar-to-bar moves × 1.6.
• Trend (MA cross): when the 10-bar average crosses the 40-bar average, enter in that direction. Stop 2 × ATR away. Exit (and reverse) on the next cross, or at the stop.
• Breakout: buy a close above the highest close of the last 20 bars, sell a close below the lowest. Stop at the middle of that 20-bar channel. Exit when price closes beyond the opposite 10-bar extreme, or at the stop.
• Range fade: buy a close below the lower Bollinger band (20-bar average − 2 standard deviations), sell above the upper. Target the 20-bar average. Stop 1.5 standard deviations beyond entry. Exit after 20 bars if neither is hit.
• RSI reversal: buy when RSI(14) drops below 30, sell when it rises above 70. Exit when RSI crosses 50, or at a stop 2 × ATR away.
• Every strategy: one trade at a time, entered at the close. Stops are checked on each close, so a loss can be slightly more than 1R. Each result is measured in R (the distance to the first stop) after a 1-pip cost.

Trades

Win rate

Avg win

Avg loss

Expectancy (per trade)

Total

Max drawdown

Simulated prices, not real data. The same moving-average rules on trending and ranging prices. Results are illustrative, not a forecast.

Treat the trending market as the robot’s in-sample world: the trend rules finish about +4.9R over 7 trades. Now tap “Ranging” without changing a rule. The same rules lose about 5R over 9 trades. Seven trades prove nothing either way, so press “New prices” a few times in each market before you draw a conclusion.

Costs the backtest leaves out

The smaller the target in pips, the larger the share that spread, commission and slippage take. A scalping robot’s backtest is only as good as its cost assumptions.

“99% modelling quality”

In MetaTrader 4’s strategy tester, modelling quality describes how completely the tester rebuilt price movement from the data it had. A high figure means detailed tick data covered the test period. It says nothing about whether the spread was realistic, whether commission and slippage were included, or whether the settings were curve fitted. A test can show 99% modelling quality and still be tuned to the past.

Ask a seller for out-of-sample results, the exact cost assumptions and several months of verified live trading at a realistic size. A backtest, however detailed, does not promise future results.

Quick check

Quick check

1. Question 1 of 4

   What is out-of-sample data?

   AA period of history the rules never saw while they were being tunedBThe period used to optimise the settingsCLive trading on a funded account

Go deeper: Testing a strategy · Trading Expectancy: The Number That Says If a Strategy Works