StockFetcher Forums · General Discussion · Usefulness of backtesting debate<< >>Post Follow-up
psheridan050
52 posts
msg #64795
Ignore psheridan050
7/9/2008 12:02:11 AM

I just wanted to open up the debate on the usefulness of backtesting. Iím curious how many people put their faith in strategy backtest results, and how close actual trading compares to the results found within a backtest. Another question for backtesters is what criteria do you use to measure whether a strategy is worth pursuing or dumping; profit factor, max drawdown, etc. Another good question for people who believe in backtesting; to what lengths do you go to prove your strategy is valid? By this Iím asking about the methods you use. Examples: Do you take a 6 month backtest as confirmation your strategy is valid? Does your strategy have to have a PF of 3 or greater in SFís backtester since 2002 in order to keep using a strategy? Do you run your backtest results through another program such as Excel to verify results? Do you use a battery of statistics tests to ensure the distribution of profits has a fat right tail with an otherwise small standard deviation?

For those who donít believe backtesting is useful, how and when do you decide to use a particular strategy when trading? Along these same lines, in what way do you determine a strategy is working for you and how long does it take to make such a determination? I assume the only way to find this out is with live trades which can get expensive if itís a bad strategy.

I guess a debate such as this can be narrowed down to two main ideas:
* Do you believe that the past performance of stocks is indicative of future performance?
* Are you a mostly a mechanical trader or a discretionary trader?


cunparis
71 posts
msg #64797
Ignore cunparis
7/9/2008 6:30:18 AM

Let's say we backtest with a certain condition (crossover, candlestick pattern, volume, whatever) and we go long for 1 day hold. Quite simple. Let's say backtest says this works 65% of the time with a 1.5 reward/risk ratio.

Does that mean that tomorrow when this condition occurs, there is a 65% chance of the trade working? No. because the conditions for tomorrow are different. There could be an announcement, oil could hit a new high, etc.

But over time, if this pattern didn't work 65% of the time, then the backtest statistics would drop down.

let's say 2002-2004 65% 1.5 ratio.
in 2005 either we get more than 65%, 65%, or less than 65%. Either way, for 2005 the results will show what we observed.

So my opinion is that what happened in the past will continue to happen in the future. This principle is called "yesterday's weather". But one must acknowledge that at any given moment the results could change. so if using a system based on backtest, it must be monitored to see if it has changed.

The more complex the system (indicators!!) the less reliable in my opinion. The more basic, the more it relies on human nature which doesn't change.

For the SF backtests, I find the stop loss very limiting and I can't get any strategies profitable with stop losses. For example I'd like to put my stop below the low of the last 5 days. I can't do that with SF. That kind of ruins it for me. It's fun to test out ideas but it can only go so far.



chetron
2,817 posts
msg #64800
Ignore chetron
7/9/2008 6:38:08 AM

backtesting was a step towards learning to trade.
once i learned how to trade, i stopped backtesting.




StockFetcher Forums · General Discussion · Usefulness of backtesting debate<< >>Post Follow-up

*** Disclaimer *** StockFetcher.com does not endorse or suggest any of the securities which are returned in any of the searches or filters. They are provided purely for informational and research purposes. StockFetcher.com does not recommend particular securities. StockFetcher.com, Vestyl Software, L.L.C. and involved content providers shall not be liable for any errors or delays in the content, or for any actions taken based on the content.


Copyright 2016 - Vestyl Software L.L.C.Terms of Service | License | Questions or comments? Contact Us
EOD Data sources: DDFPlus & CSI Data Quotes delayed during active market hours. Delay times are at least 15 mins for NASDAQ, 20 mins for NYSE and Amex. Delayed intraday data provided by DDFPlus