I have encountered a few issues when using backtesting, which can affect the backtests in a huge way. I am not able to recall the few issues i have found, so I am mentioning one below which i just encountered.
I will use this thread to post about any other issue that might cause a huge difference to backtest results.
I have created a simple strategy to enter into a short trade if the prev close of a 15 min candle crosses below the Supertrend (144,3)
I am using this on the GBPINR March Future script. As per the charts on kite, there is no sell signal from the supertrend on 27th Feb, 2019.
But, the transactions list shown in the backtest points out that there was an entry taken on the same date.
I have provided the screenshots below,
Could you please look into this. Please let me know if there is any issue on my end or if i have missed something.
I will update this when i am able to recall any other potential issues.
There is nothing particularly wrong with the backtest results. In your entry condition you have specified ‘Previous close crosses below Supertrend’ hence it will compare the close of previous 15 min candle with the current [Candle (0)] value of supertrend.
If we look at the charts, we can easily see that the close of 16:45 candle of 26th feb crossed below supertrend value of 9:00 A.M 27th feb candle.
@Renko_Jhunjunwala Okay, wow. I didn’t see it that way, so thank you.
Good to know that is how it works.
@Streak But ideally, I don’t think there would be much of use to compare a previous Nth candle close with current supertrend value. I really think it should be compared with that corresponding candle’s supertrend value, if it’s possible. Anyways, thank you.
@prash_u
Offset for supertrend indicator is needed to form this condition. We are glad to inform you that offsets for about 10 indicators are scheduled to be released this week including supertrend!
Now, i have noticed the below two issues with this:
The entry signal shown on 5-Feb-2019 is not included in the transactions of the backtest. I am not sure why this could be. It is highlighted as A in the chart
The entry as per the chart on 7-Feb-2019 must have been at 9:15 AM i.e. at point B1 at the price around 82.18.
But, the actual entry is taken on 7-Feb-2019 at 2:45 PM, i.e. at point B2 at a price of 81.77 when actually there was no signal as per the chart.
This could cause a huge difference in P/L in the backtests.
You are using supertrend with period as 144.
Now when you are backtesting from 1st Feb on 15min candle, that means each trading day has 32 candle.
Not when backtesting from 1st Feb, there will be a need of atleast 144 candles to be present before the supertrend can be calculated and few more candles for the supertrend signal to stabilise. Then the conditions can be validated.
That is why first few signals if any, will not get captured or the first trade can be captured incorrectly.
You can easily avoid lower the start date by 1week.
This problem in live deployment will not be encountered, because the system there automatically calculates the values on a abundance of candles, making sure that more than enough candles are present for technical calculations and the signals are precise.
Okay, thank you very much for the detailed explanation.
But, is there any way this can be rectified?
i.e. during backtesting also, can the calculations and signals be made precise by using enough/abundance of candles as necessary (just like when it is deployed)
Because, the backtesting results might vary by a huge margin because of these inaccurate trade calculations.
So, based on this backtesting we might actually think the strategy is good and deploy it. When actually, the strategy might not be that good.
@prash_u,
So we are looking into ways for doing this dynamically for backtesting, that’s why by default we have enabled 3months for backtesting in intraday so mostly no one face’s this.
Now as your period is huge, you should try to backtest for as long duration as possible and then just ignore the first trade, the pnl only for the first trade can discounted for and the rest remains the same.
Okay, thank you very much for the explanation and the workaround suggested for now.
Will try this when backtesting again and update the thread if any other clarifications needed.
Thank you for the workaround, this helps now. Hope this can be fixed in the coming days.
Also as a suggestion, could it be made possible to download the backtest data for all scripts at once, instead of having to download for each script separately.
maybe it can all be downloaded in one excel workbook, with a separate worksheet for each script
or zip all worksheets and downloadable at once
I have attached screenshot below, for what i am referring to.
Here is another very important issue that needs to be rectified at the earliest.
This is mainly when testing CNC orders because of the gap ups and gap downs.
Let’s say if i enter long into a trade place a SL exit at 0.2% on a stock, and it opens say 1% gap down,
then the exit price should be the current price at when exit is triggered.
But, currently it just takes 0.2% value of the price as exit value, even though that price has not been hit.
This is a major issue and influences the backtest results in a very wrong way. Below is a few screenshots to help.
Great work @Streak and thank you very much.
I checked the issues that i have mentioned earlier in this post and its resolved.
Saves me even more time now.
Will update if i find anything else or if any feedback.
Please share the strategy link for the strategy that you have created and also mention the date and time of the false signal that was generated. I will check and assist you further. Refer to the link to learn how to share the strategy link - How to share a backtested strategy - Streak Tech
