StockFetcher Forums · Filter Exchange · BACK TESTING - do it right !<< >>Post Follow-up
xplorer
257 posts
msg #34012
Ignore xplorer
11/19/2004 8:08:32 PM

Backtesting is a necessity !

... but if it isn't done right it can skew reality.
So what is the most reliable and accurate way to back test our filters ?

Hopefully someone has the answer so we all can use a known "standard".

Should a simple line at the end of our filter work ?
OFFSET nn days (nn being an number to max)
like this :
Fetcher[
RSI(5) is greater than RSI(2)
and RSI(5) is less than 25
and price is between than .4 and 4
and Bollinger(20) is between than .01
and average volume(5) is greater than 52000
and offset 3 days
]


(but the results are always biased to the current sentiment)

Or should every command end with the
LAG nn DAYS
Fetcher[
RSI(5) is greater than RSI(2) lag 3 days
and RSI(5) is less than 25 lag 3 days
and price is between than .4 and 4 lag 3 days
and Bollinger(20) is between than .01 and 3 lag 3 days
and average volume(5) is greater than 52000 lag 3 days
]


(this won't give you the % gains ... you have to calculate it)
(but I think it probably is the only REAL way to do it !)
... there has to be a better way !

.. or a combo of both :
Fetcher[
RSI(5) is greater than RSI(2) lag 3 days
and RSI(5) is less than 25 lag 3 days
and price is between than .4 and 4 lag 3 days
and Bollinger(20) is between than .01 and 3 lag 3 days
and average volume(5) is greater than 52000 lag 3 days
and offset 3 days
]


(yikes ... get me out of here !)

..or maybe the PERFOMANCE listed on the SF results ?

.. or another way ?

Any opinions ... or info would be appreciated !

THANKS !

...by the way ...
this filter gives fantastic results,
if you backtest the "CORRECT" way.





jpistell
123 posts
msg #34015
Ignore jpistell
11/20/2004 12:26:04 AM

Backtesting needs to withstand the 4 seasons of trading.
SPRING, SUMMER, FALL & WINTER.

Summer = bulls rally
Winter = Bears rally
Spring & Summer join the 2 phases together.



jclaffee
81 posts
msg #34016
Ignore jclaffee
11/20/2004 3:27:52 AM

xplorer writes

"Backtesting is a necessity!
. . .but if it isn't done right it can skew reality.
So what is the most reliable and accurate way to backtest our filters?"

IMHO, this question goes like an icepick to ones survival as a trader and the answer lies beyond the range of any of the alternative answers proposed.

Consider this:

On Monday, 11-1, my filter gives me an entry signal on an equity which closed the session at $10 a share. . .on Tuesday, 11-2, it closes up 1.00 to $11.00. . .on Wednesday, 11-3, it gains another 1.00 to $12.00. Two days, 20% gain. . .right? One great filter. . .right? Maybe. . .maybe not.

Going a little further, let's say that on the morning of Tuesday, 11-2, my issue either

1) gapped up 0.75 to open at 10.75 and traded above 10.60 all day so -- since my trading discipline includes "don't chase price higher" -- I never had an entry. (Perhaps I'd be insightful enough to conclude that if the "pros" supported this advance in the final hour an entry at the higher level would not be "chasing" -- perhaps not.)

or

2) opened at 9.80, giving me an entry, then traded lower to 9.00, triggering my initial stop, before registering the advance to 11.00.

So, I had the right issue at the right time and, in alternative 1), failed to get on board and, in alternative 2), took a loss. In either event, a "backtest" of my filter would say that I had a 20% gain in two sessions.

Again, IMHO, meaningful backtesting has to be done by following the chart of an issue day by day according to ones own trading style. Anything else is backtesting a unreal world.

Jim























xplorer
257 posts
msg #34020
Ignore xplorer
11/20/2004 9:30:40 AM

Here is an accurate ...is burdeonsome, convoluted, and restricted
to short term trades... unless you are willing to accept the gaps with
the "weeks" testing...if that is your trading personality.

This is way to get an accurate backtest.

Assuming I have a filter in which the results will
return stocks that I want to hold less than 6 days....
a quick return.

for 18 days ago:
Fetcher[set{gainday6,close 12 days ago / close 18 days ago}
set{gainday5,close 13 days ago / close 18 days ago}
set{gainday4,close 14 days ago / close 18 days ago}
set{gainday3,close 15 days ago / close 18 days ago}
set{gainday2,close 16 days ago / close 18 days ago}
set{gainday1,close 17 days ago / close 18 days ago}
set{closeday0,close 18 days ago}
RSI(5) is greater than RSI(2) lag 18 days
and RSI(5) is less than 25 lag 18 days
and price is between than .4 and 4 lag 18 days
and Bollinger(20) is between than .01 and 3 lag 18 days
and average volume(5) is greater than 52000 lag 18 days
and add column CloseDay0
and add column gainday1
and add column gainday2
and add column gainday3
and add column gainday4
and add column gainday5
and add column gainday6
]




...or for 118 days ago:
Fetcher[set{gainday6,close 112 days ago / close 118 days ago}
set{gainday5,close 113 days ago / close 118 days ago}
set{gainday4,close 114 days ago / close 118 days ago}
set{gainday3,close 115 days ago / close 118 days ago}
set{gainday2,close 116 days ago / close 118 days ago}
set{gainday1,close 117 days ago / close 118 days ago}
set{closeday0,close 118 days ago}
RSI(5) is greater than RSI(2) lag 118 days
and RSI(5) is less than 25 lag 118 days
and price is between than .4 and 4 lag 118 days
and Bollinger(20) is between than .01 and 3 lag 118 days
and average volume(5) is greater than 52000 lag 118 days
and add column CloseDay0
and add column gainday1
and add column gainday2
and add column gainday3
and add column gainday4
and add column gainday5
and add column gainday6
]



... or for the 8 weeks:
(though this screen wasn't designed for this type of "hold")
Fetcher[set{gainday6,close 2 weeks ago / close 8 weeks ago}
set{gainday5,close 3 weeks ago / close 8 weeks ago}
set{gainday4,close 4 weeks ago / close 8 weeks ago}
set{gainday3,close 5 weeks ago / close 8 weeks ago}
set{gainday2,close 6 weeks ago / close 8 weeks ago}
set{gainday1,close 7 weeks ago / close 8 weeks ago}
set{closeday0,close 8 weeks ago}
RSI(5) is greater than RSI(2) lag 8 weeks
and RSI(5) is less than 25 lag 8 weeks
and price is between than .4 and 4 lag 8 weeks
and Bollinger(20) is between than .01 and 3 8 weeks
and average volume(5) is greater than 52000 lag 8 weeks
and add column CloseDay0
and add column gainday1
and add column gainday2
and add column gainday3
and add column gainday4
and add column gainday5
and add column gainday6
]



Your thoughts ?
Is there a better way ?


xplorer
257 posts
msg #34021
Ignore xplorer
11/20/2004 9:35:03 AM

Just to add clarity to the results ...
the "gain" columns are a ratio ...
less than 1.0,and you have a loss...i.e .94 is a 6% loss.
a 1.15 would be a 15% gain...


xplorer
257 posts
msg #34022
Ignore xplorer
11/20/2004 11:46:32 AM

Jim,

I agree with you ... backtesting is not a cut and dry science ... but within the limitation of SF, we need to have a way to evaluate our screens. I like the idea of this site do the number crunching... Of course there are anomalies, picks that will deviate from the norm ... noise is expected.

In the end, trading skills will seperate the true traders from the dreamers.


cegis
235 posts
msg #34165
Ignore cegis
11/28/2004 8:19:24 PM

My suggestion was posted in the thread "Technique: Calc your own returns - for any period" in the General Discussion forum. xplorer seems to be using some of that technique.

Basically, what I do now is use "days ago" (e.g., 10 days ago) to offset the conditions in the filter I'm backtesting, then using "future" days ago (e.g., 9 days ago, 8 days ago,...) to get (using "add column") open/high/low/close of the following days. I then download the results into a spreadsheet and analyze the results. There, I can "program" assumptions to most closely match the trading I'm contemplating with the filter, take commissions into account, and anything else I can think of.

HTH,

C


xplorer
257 posts
msg #34178
Ignore xplorer
11/29/2004 5:02:15 PM

Hey CEGIS ...
I took a look at you post you referenced .... and it is similar. I like the way you incorporated the HIGH and LOW within the last xx days ... nice ! Technical writing seems to be a gift of your ... nicely written.


cegis
235 posts
msg #34181
Ignore cegis
11/29/2004 7:51:58 PM

Gee, I blush! :-) Glad you liked it, and found it useful...

C


StockFetcher Forums · Filter Exchange · BACK TESTING - do it right !<< >>Post Follow-up

*** Disclaimer *** StockFetcher.com does not endorse or suggest any of the securities which are returned in any of the searches or filters. They are provided purely for informational and research purposes. StockFetcher.com does not recommend particular securities. StockFetcher.com, Vestyl Software, L.L.C. and involved content providers shall not be liable for any errors or delays in the content, or for any actions taken based on the content.


Copyright 2016 - Vestyl Software L.L.C.Terms of Service | License | Questions or comments? Contact Us
EOD Data sources: DDFPlus & CSI Data Quotes delayed during active market hours. Delay times are at least 15 mins for NASDAQ, 20 mins for NYSE and Amex. Delayed intraday data provided by DDFPlus