Introduction to Backtesting
Learning Objectives
● Understand the goals of the backtesting validation technique
● Identify the steps in creating a split-data backtesting model
● Identify the steps in designing a sliding-window backtesting
model
● Understand some of the weaknesses and biases that can affect
a backtest
● Distinguish between development and implementation
backtesting models
Agenda
What is a Backtest?
Key Objectives
Common Weaknesses and Biases
Design: Development vs
Implementation
Split Window Backtesting Data
A B
Split Window Backtest
Go Live or Paper Trade Strategy
Collect and Backtest
Partition (A∣B) Strategy on
Historical Data Set Data Set B
Design Strategy Using Data Set A
Split Window Backtest
Go Live or Paper Trade Strategy
Collect and Backtest
Partition (A∣B) Strategy on
Historical Data Set Data Set B
Design Strategy Using Data Set A
What is a Backtest?
● A simulation designed to
measure the performance of
your trading and risk
management rules applied to
historical data
● Quantifies the performance of
your strategy for comparison
with other strategies
● Predicts likely capital
requirements, trade frequency
and risk for your portfolio
Sliding Window Backtest
Validate Strategy Using Next Testing Window
Design Strategy Design
Using Training Strategy Using
Window Next Training
Window
Design
Strategy
Using Data
Set A
Validate Strategy Using Testing Window
How a Sliding Window helps in Back Testing
Agenda
What is a Backtest?
Key Objectives
Common Weaknesses and Biases
Design: Development vs
Implementation
Backtesting Objectives
● Filter out any strategies that
don’t meet your performance
criteria:
● Sharpe Ratio = Return / Risk
● Calmar Ratio = Return / Maximum
Drawdown
Backtesting Objectives
● Test and fine tune the
performance of new models
taking into account
microstructure issues:
● Transaction costs (spread and
market impact)
● Order routing
● Latency (data and orders)
● Liquidity
Agenda
What is a Backtest?
Key Objectives
Common Weaknesses and Biases
Design: Development vs
Implementation
Backtesting Objectives
● Optimize your strategy
● Increase its performance by modifying its associated quantity
or parameter values
Backtesting Weaknesses
● Dependence on transient
correlations
● Low-quality data
● Unrealistic handling of:
○ Slippage
○ Market impact costs
Backtesting Biases
● Optimization Bias - Overfitting
your model too closely to limited
data
● Look-ahead Bias - Unintended
introduction of future
information into past data
● Survivorship Bias - Only
including financial instruments
which still exist (positively
skewed sample)
Backtesting Biases
Drawdown Tolerance Bias
● Backtest data for longer-term momentum strategies often show
upwardly trending equity prices
● These strategies can show very attractive returns and good
Sharpe Ratios but can often have drawdowns of as much as 25%
with a duration of 3 or more months (think Fall 2018)
● When you are actually faced with this big a drawdown, it can be
difficult to sustain the strategy
Agenda
What is a Backtest?
Key Objectives
Common Weaknesses and Biases
Design: Development vs
Implementation
Development Backtesters
● Create preliminary models
quickly so many
strategy/parameter variations can
be tested quickly
● Identify statistical relationships
● Often give inflated performance
