How to understand that the adviser is out of date?
Good afternoon, ladies and gentlemen, Forex traders!
I often hear questions from beginners about why you need a forward test during optimization, how to evaluate the results obtained during testing and optimization, how to compare the test results and the work of real-time advisers.
And the most important question: “How to understand that the adviser is out of date, has stopped working as it should, and it is time to withdraw it from the account?”
It is precisely these questions that I will try to answer today.
Backtests and real trading
In practice, it often happens that a system that has shown good results in back testing is not able to make a profit in real trading. This is due to the peculiarities of back-testing, conducted on historical data, to which the system adapts as a result of optimization.
Back-testing programs do not take into account some factors affecting the success of a trade, for example, the degree of liquidity of a tool or competition from other bidders. The test does not allow taking into account possible technical difficulties, which (if we leave aside the question of the speed of execution of applications) are most important for high-frequency trading, they are reduced to communication deficiencies, malfunctions of brokers and the exchange itself. The disadvantages of back testing are useful to consider in order to soberly assess the capabilities of your system.
One of the stages of testing a system is its optimization, during which the robot adapts to historical data in order to achieve a better result. By changing the values of various parameters of the algorithm, almost any trading system can be forced to show a plus on a previously known segment of the price history. Moreover, the number of parameters used matters: a simple system using a small number of variables is more difficult to optimize, but with the introduction of each additional condition and parameter, the trading result can be improved. However, the growing theoretical profitability does not mean that the system is getting better. The robot is simply better suited to historical data. Complicating the system, it is possible to achieve that in an open story it will work on all the possibilities, but in the market it will bring only losses.
The apotheosis of the complexity of trading systems are neural networks, which have a lot of parameters. A robot based on a neural network, having memorized a large amount of information, can adapt to historical data so much that it will simply use them in the future. Therefore, especially when it comes to complex systems with a large number of coefficients, the optimization process must be treated with caution.
A good system should show at least some positive result without optimization, if all profitability is achieved only at its expense, then the robot writer has reason to think. In the optimization process, in my opinion, it is not always worthwhile to dwell on the coefficients of the system at which the best profitability is achieved during back testing. If the parameter value is very close to the edge of the positive range (for example, 7 with positive results from 5 to 20), then it makes sense to shift the parameter value closer to the middle.
What is a forward test? This is a test of the system parameters obtained during optimization on a different historical data sample other than the optimization one. If we explain it on our fingers, first we adjust our settings to a piece of history, finding some better sets (this is called optimization), and then check on a later piece of history whether these settings are still suitable or they are no longer relevant and the adviser with them began to merge. Many beginners neglect forward testing. I hope I explained to you why you should not neglect this.
The best set of parameters on a particular piece of history does not have to be the best on a later piece - everything can change a hundred times. And, I think, once again before installing the adviser on real life, reinsurance and checking the performance of the parameters still does not hurt, even if you are the son of an Arab sheikh and use dollars instead of napkins.
Unlike established considerations, a forward test is not necessarily a check in the strategy tester. Forward can be carried out online on a demo or a small real account. This approach has both pros and cons. The main disadvantage is that such testing takes a very long time. A plus can serve as the fact that in this case the test readings are as close as possible to real indicators (in fact, these are real indicators). That is, you will definitely see how the strategy of this broker in this type of account behaves.
In general, the forward test has three main goals, and not one. The first two are essential for adhering to test technology. The latter provides important and unique information about the expectations of profit and risk for future trading on real accounts.
So, primary goal test - to understand whether the results obtained during optimization are real. The fact is that a trading system designed correctly should be profitable not only in the optimization area, but also subsequently (in the forward period and in real time). But this does not always happen - some trading systems simply cannot pass the forward test, starting to pour immediately after the optimization period. Such systems are simply initially inoperable, no matter how hard you try to optimize their parameters. Hence the rule - if you intend to use the system in real trading, it is simply obliged to pass forward testing.
Second goal test - to avoid over-optimization of system parameters. Quite often, if the optimization technology is not followed, even a good system can be re-optimized. This can happen due to the presence of too many rules and filters, an excessive number of optimized parameters, or too small a step for optimization.
The forward test provides a measure of effectiveness, called the forward performance indicator, which compares the annual rate of return for the forward period with the rate of profit obtained during optimization. It is easy to determine the re-optimization of parameters based on a comparison of annual rates of return - if they are very different, you are dealing with re-optimization.
And here we come to third goal forward testing - measuring the profitability and risk of a system to determine investment expectations. An ideally designed and optimized system will have the same profit and risk indicators for the forward period and subsequent trading as for the optimization period. If the rate of return on the forward test differs significantly from the profit on the forward or in real time, such settings are not suitable. I hope that now you have no questions about the feasibility of conducting forward testing.
Demo Tests and Live Account Tests
Many try to test advisors on demo accounts. Quite often you can hear: "I'm chasing this adviser on a demo." If you are really going to use this or that advisor on real accounts, you should not do this.
FirstlyThere are no performance problems on demo accounts. Most brokers configure their demo servers so that the trader does not experience problems with slippage and requotes. As a result, it often happens that the robot works fine on a demo, and when installed on a real account, it starts to lose money.
Secondly, quite often the broker provides the "best spread" on demo accounts. When you open a demo account, you will most likely receive quotes with the minimum possible spread for this broker. Thus, the influence of this factor can greatly affect the idea of the total profitability of the adviser.
AND thirdworking with demo accounts, you do not experience psychological pressure. This factor also has a positive effect on account profitability.
Therefore, it is best to test advisors on small real accounts - 100-200 dollars in most cases are enough. In addition, many brokers now provide cent accounts. And yet, ideally, it is worth trying to test the advisers precisely on the servers on which you intend to subsequently trade.
Key indicators of trading systems
Now we will analyze the main indicators of trading systems, think about whether to pay attention to certain indicators and how important they are.
Running any business is costly. Forex is the same business as everyone else, and the cost of trading profit is determined here by risk and margin.
Maximum drawdown is the value of the deepest decrease in the yield curve of a trading account before it reaches a new maximum. It is psychologically very difficult to withstand large drawdowns in real trading. When selecting sets, I usually do not consider results with a drawdown of more than 15-20%. When calculating money management, I try to achieve a maximum drawdown of one pair of no more than 10-15%. When working with large amounts, it is recommended that the maximum allowable drawdown in the account be no more than 5%. Judge for yourself how nice it is for a couple of months to drop a ten millionth account by 20%. Not every nerve is so strong and sometimes doubts arise in the used trading system. The higher the maximum drawdown, the higher the psychological pressure experienced by the trader.
Trading is needed in order to make money. A series of losing deals hits the ego of the trader, unsettling and forcing him to make rash decisions. This, of course, concerns manual trading more, but nevertheless, when trading robots, the algorithmic traders monitor the state of their accounts, so that it will not be possible to completely get rid of the psychological factor in algorithmic trading (although the influence of this factor is greatly leveled by eliminating the need for the trader make trading decisions). And yet, if the trader is not psychologically ready to accept such drawdowns, it is better to determine an acceptable framework in advance, even on the shore, by adjusting the advisor's lotness in the proper way.
Required capital - the minimum amount of money to trade an adviser. Knowing the maximum drawdown and properly setting up money management, we can determine the amount of minimum investment for trading an adviser. Our task is to allocate such an amount of money so that with a tuned money management we can withstand at least 1.5 maximum drawdowns quite calmly, without unnecessary nerves. Some professional traders are insured by accepting 3 maximum drawdowns.
Amount of deals
Each type of trading system has its own reliable number of transactions. For intraday trading, this is usually from 200 to several thousand transactions per year. For trading on D1, the number of transactions per year can reach as little as 10. In any case, it is worth remembering that the more transactions there are in statistics, the more reliable the result is. As a rule, the minimum number of transactions for evaluating system parameters is one hundred.
Average profit per trade
When developing a trading system, it is very important to pay attention to this indicator. The higher the average profit per trade, the better. Sometimes it happens that in tests when trading with lot 0.1, the system gives 3-5 dollars of profit per trade. Having evaluated all the other parameters, the trader decides to put the system on a real account. After some time, it turns out that the system is losing money.
When assessing the statistics of a real account, it turns out that the average profit per trade is in the negative zone. How did it happen? Just testing was carried out with a spread of two points and without taking into account reality. And the real spread turned out to be 3 points more than planned and as a result, only $ 2 was left from the average profit per trade. And they forgot to take into account the slippage, which "ate" two more points. Plus the commission, which was generally forgotten. And in the end, the average profit was minus one dollar.
The average profit per trade is an important parameter and in order not to frantically search for a broker on which “this” will work in plus, it is worthwhile to focus on values not lower than 10 points ($ 10 per trade with lot 0.1), or better - more .
I already wrote a lot about this, but I will repeat it again. Many professionals work with a percentage of profitable trades of 50, 40 or even less. But it is psychologically very difficult and not everyone can withstand such a trade. As a rule, a low percentage (below 50%) is explained by a high ratio of the average profitable transaction to the average unprofitable transaction - from 3 to 1 and above. Such a balance is characteristic of long-term trend trading systems. Scalper systems and intraday systems are usually characterized by a value of 60-70 percent or even higher, but the profit to loss there is usually not higher than 1 to 1. But such systems are already more comfortable trading, and their yield curves look smoother . Nevertheless, it is extremely difficult to find such a system, which is stable at the same time - sometimes trading with a percentage of profitable transactions of less than 50% is less time-consuming.
After you have compared the average annual profit of the trading system with other investment instruments you use and decided on the advisability of using the system in this vein, it is time to evaluate the TS regarding risk and required capital. An annual profit of $ 100K is cool, but if you need 2 million to deposit it, it is only 5% per annum, it’s not so cool. Moreover, if the maximum drawdown is 25%, which corresponds to $ 500K or a risk-to-risk ratio of 1 to 5 - this is generally nowhere worse.
And vice versa, if 100K is required for the same profit at a risk of 10% or 10K, then the ratio of profitability to risk is already 10: 1. And this is just a fabulous result.
Risk Adjusted Return - RAR
The examples above suggest the correct idea that the assessment of the profitability of the system should be carried out taking into account the risk necessary to obtain this profitability. The ratio of profitability to risk (reward to risk ratio) is just such an indicator. He compares the maximum annual return with the maximum drawdown that was allowed.
For example, an annual return of $ 25,000 with a drawdown of $ 5,000 will give a reward to risk ratio of 5. As a rule, the higher this ratio, the better. Many trading systems have this indicator from 5 to 10.
Return on equity
In general, profit should be considered as return on investment. It is calculated simply - it is enough to divide the annual profit by the minimum required capital.
Here is an example: we have a system that gives a maximum drawdown of $ 10,000 and an annual profit of $ 40,000. The profitability / risk ratio is quite good, it is equal to four. For example, we are ready for drawdowns of 20% and then we will accept the drawdown on the test for 10% (double margin). Then the minimum capital for trading with us will be $ 100,000. Having invested $ 100,000, we will get $ 40,000 per year or a yield equal to 40% per annum.
The advantage of considering annualized returns is the ease of comparison. This is a generally accepted standard, which also facilitates the comparison of a particular trading system with others.
This method of comparing the profits of different systems considers the profits from the system in the context of the capabilities of the current market. In different periods, markets have greater or lesser potential for profit and it would be logical to pay attention to this when comparing indicators.
Here it is worth introducing another definition.Potential market profit is the profit that can be obtained by buying each bottom and selling each vertex for the considered period of time (usually a year).
Naturally, no trading system is capable of squeezing everything out of the market. Therefore, you can enter a special coefficient - the efficiency of the trading system. This is the efficiency of how effectively the system converts the potential profits offered by the market into real trading profits in the trader's account. For example, suppose the net profit of the system is $ 25,000, and the potential $ 300,000. Then the system efficiency is (25/300) = 8.33%. This is quite good performance. On average, fairly good trading strategies have a ratio of 5% or higher.
The efficiency indicator makes it easy to compare systems for different markets and at different periods. Markets are constantly changing and those indicators that the system had in the past may never be repeated. At the same time, the system efficiency is a fairly reliable indicator. Efficiency, which remains at a consistently high level from year to year, is an indicator of the stability and high quality of the trading system and suggests that no matter how the market changes, the system continues to profit from it on an ongoing and stable basis.
Instead of judging the system by the average annual profit, it is more convenient to consider such a parameter as the profit factor. In fact, this coefficient is another attempt to measure the effectiveness of the trading system. Profit factor is the quotient of dividing total profit by total loss. For example, a profit factor of 1.5 may indicate that the system loses 2 (3/2 = 1.5) on average for every 3 dollars of profit. A value above 1 indicates that the system can make money. The higher this unit is, the more effective it is. It is advisable not to consider systems with a profit factor below 1.3 and ideally strive for a value of 1.6.
Trade stability is the most essential characteristic of a trading system. The more stable the trading system in all respects, the better. And vice versa, the more messy and unstable the system, the more dangerous it is and, therefore, should cause great doubts. Agree, when the results are very erratic and it is difficult to guess whether you will get a profit of 80% this year or whether the system will lose everything in the last month of the year and go into minus - not the best state of things. Let's see how you can measure the stability of trade and what indicators are used for this.
Profit and loss sharing
The uniformity of profit and loss distribution in the test and forward samples is the most important indicator of stability. The mere net profit brought by the system does not say anything about its stability. After all, all the profits could be made in just one month a year, while the rest of the time the system leaked money. It is the distribution of profits and losses over time that gives a good idea of how much you will have to worry when using the system.
Suppose a system has a profit of $ 50,000 over a five-year period with a drawdown of $ 10,000. For example, as shown in the table:
|2013||50 000||5 000|
|2014||30 000||6 000|
|2015||10 000||7 000|
|2016||- 15 000||9 000|
|2017||- 25 000||10 000|
The greatest profit was in the very first year, the largest loss - in the last. In addition, if you build a profitability chart, we will see a falling curve. Even worse, annual drawdowns are growing. This trading system flourished in the first two years, after which it has clearly merged capital for several years now.
Or another example:
|2013||-15 000||5 000|
|2014||110 000||10 000|
|2015||-15 000||7 000|
|2016||-15 000||6 000|
|2017||-15 000||4 000|
Even a quick look at the table shows that all the profit was made in 2014. The rest of the time, the system was steadily losing money. This alone is enough to abandon such a strategy.
And a third example:
|2013||10 000||7 000|
|2014||5 000||10 000|
|2015||10 000||6 000|
|2016||10 000||5 000|
|2017||15 000||4 000|
In all three examples, the resulting profit was $ 50,000 with a maximum drawdown of $ 10,000. But pay attention to the uniformity of the result in the latter case. In addition, the system shows a pleasant upward direction and a decrease in drawdown. Moreover, the system had a maximum drawdown in the most distant period of operation. All this indicates a completely satisfactory stability of the trading system.
Here is another example:
|2013||- 25 000||10 000|
|2014||- 15 000||9 000|
|2015||10 000||7 000|
|2016||30 000||6 000|
|2017||50 000||5 000|
This example is the inverse of the first. This option, despite the unequivocal improvement in indicators, is also most likely unsuitable. But if there is a great desire to still use it in trade, then first of all it is necessary to find the answer to the question: why did it work so poorly before and work so well now? Perhaps this is a temporary factor, and some time after the start of the system, it will already exhaust itself.
In general, you should be guided by the following rule - the more uniform the results, the better. If profitability has a tendency, it is necessary that it be moderately upward, and not vice versa. Moreover, any trend should be justified.
The distribution of transactions is usually calculated in exactly the same way as the distribution of profits and losses for a given period of time. The more uniform the distribution, the better, of course.
The best strategy is one in which profits and losses are evenly distributed throughout the period. But you will never achieve perfect uniformity. Therefore, it is important to ensure that at least the main profit is not received as a result of one or more series of successful transactions.
By the way, a series of winning and losing trades should also be distributed evenly throughout the site. The smaller the standard deviation, the more predictable and stable the result of trading.
So, a stable trading system has the following properties:
- the most uniform distribution of profits and losses;
- the most even distribution of wins and losses;
- the most even distribution of series of wins and losses.
Maximum drawdown plays a crucial role in assessing the risk of the trading system. It should be evaluated relative to other losing series generated by the trading system. A-priory maximum drawdown - This is the largest losing series of deals, but it is also important to know how much this series of deals is larger than others. For example, if the maximum drawdown is only 20-40% more than all other drawdown periods, this may serve as additional evidence of the stability of the system.
If your maximum drawdown is 300% of the average drawdown, then this is a pretty bad sign. Unless, of course, this is caused by objective reasons such as a stock market crash or other force majeure events. Such events are almost impossible to predict and often they lead to significant losses, so it’s worth trying to protect yourself from force majeure. This is done by introducing special algorithms in the strategy that limit the maximum losses.
The biggest winning streak
It should be evaluated in the same way as the largest loss-making series. It should be compared with the average winning streak. In addition, the largest winning streak should not provide a proportionately large share of the total profit of the strategy.
Additional statistical tools for evaluating trade performance
To evaluate the effectiveness of trading systems, various factors are often used. They allow you to look at the trading result in the context of various factors. Many of these coefficients have already been considered, so I will give only a brief description of them. Some have not yet been considered, and I will dwell on them a little more.
Investment performance is often measured in terms of income dispersion. One such indicator is the Sharpe Ratio. This coefficient shows how the arithmetic mean AHPR, reduced by the risk-free rate, and the standard deviation of SD from the HPR series are related. The value of the risk-free rate RFR (Risk Free Rate) is usually taken equal to the interest rate on income on deposit at the bank or the rate of income on treasury bonds.
You can learn more about this coefficient here.
Profit over Hold Time (HPR)
In his book “Mathematics of Money Management”, Ralph Vince uses the concept of HPR (holding period returns) - profit for the time the transaction is held. A deal that brought 10% profit corresponds to HPR = 1 + 0.10 = 1.10. A deal that brought a loss of 10% corresponds to HPR = 1-0. 10 = 0.90. In another way, the HPR value for a transaction can be obtained by dividing the balance value after closing the transaction (Balance Close) by the balance value at the time of opening the transaction (Balance Open):
HPR = BalanceClose / BalanceOpen
Thus, each position corresponds not only to the transaction result in monetary terms, but also to HPR. This allows you to compare systems, regardless of the money management used in each case. One of the indicators of such a comparison is the arithmetic average - AHPR (average holding period returns).
Along with the arithmetic mean, Ralph Vince introduces the concept of geometric mean, which we designated as GHPR (geometric holding period returns), which is almost always less than the arithmetic mean AHPR.
The system with the highest geometric mean will bring the greatest profit if you trade based on reinvestment. A geometric mean less than one means that the system will lose money if you trade based on reinvestment.
The average value can be calculated not only for the sample, but for a random variable, if its distribution is known. In this case, the special meaning of the mean value is mathematical expectation. Mathematical expectation characterizes the "central" or average value of a random variable.
For profitable trading, the mathematical expectation should be higher than zero, plus all the transaction overhead (such as spread, swaps, slippages, commissions, and so on). More information about the mathematical expectation can be found here.
We already discussed the standard deviation above when we talked about the stability of the trading system. This value shows the range of values relative to the average value. The smaller the standard deviation value, the more stable the result, the higher the value - the less likely you will get a yield close to the average value. Now that we understand what a standard deviation is, let's move on to a more detailed consideration of this characteristic.
For trading accounts, three averages are often used: average profitability for a certain period, average profit and average loss. Then it is logical to calculate three standard deviations for each average: standard deviation for average profitability, standard deviation for average profit and standard deviation for average loss.
Average profitability is defined as the sum of profits and losses divided by their number, this value shows the most likely value of profitability that an account can bring for a certain period of time. The standard deviation for average returns summarizes gains and losses. If we assume that the distribution of the system’s profitability is subordinate to the normal distribution, then with a probability of 95% the value of the potential yield will be in the range of two standard deviations from the average result.
By analyzing the standard deviation from the average profit, you can know what the spread of profit relative to the average value. The smaller the standard deviation, the closer the expected result to the average value, the more stable it is.
Parameters MAE and MFE
Looking at the final result of trading, which presents the outcome of trading operations, we can not draw any conclusions about the presence of protective stops (Stop Loss) or about the effectiveness of profit taking. We see only the position opening date, closing date and the final result - profit or loss.
Having no information about floating profit during the life of each trading position and about all positions in the aggregate, we cannot make judgments about the nature of the trading system. How risky is it, how was profit achieved, were paper profits lost? The answers to these questions can be given to us sufficiently by the parameters MAE (Maximum Adverse Excursion) and MFE (Maximum Favorable Excursion).
Each open position until the time of closure is constantly experiencing fluctuations in profit. Each transaction in the period between opening and closing reached maximum profit and maximum loss. MFE shows the maximum price movement in a favorable direction. Accordingly, MAE shows the most unfavorable price movement. It would be logical to measure both indicators in points, but if the trade was conducted on different currency pairs, then to bring to a common denominator, you can use the monetary expression.
Each closed transaction corresponds to the result of this transaction and two indicators - MFE and MAE. If the transaction made a profit of $ 100, but at the same time the MAE (maximum floating loss during the life of the position) reached - $ 1000, then this is not the best way to characterize this transaction. The presence of many transactions with a positive result, but with large negative MAE values for each transaction, tells us that the system outstrips unprofitable positions, and sooner or later such trading is doomed.
Similarly, you can get information from the MFE values. If the position was opened in the right direction, the MFE (uncommitted maximum profit) on the transaction reached $ 3,000, but as a result the transaction was closed with a result of plus $ 500, it can be said that it would be nice to refine the system of protection of uncommitted profit. This may be some kind of floating stop (Trailing Stop), which we can pull up at the price with a favorable movement in our direction. If the shortfall in profits is systematic, then the trading system can be significantly improved. MFE will tell us about it.
Van Tharp Estimation Technique
The quality of the evaluated system, Van Tharp proposes to measure as the ratio of the mathematical expectation to the standard deviation of the transaction results:
where M (x) is the mathematical expectation,
σ is the standard deviation.
The resulting value of R is classified as follows:
less than 0.16 - very poor quality,
from 0.16 to 0.20 - low,
from 0.20 to 0.25 - average,
from 0.25 to 0.30 - good,
from 0.30 to 0.50 - excellent,
from 0.50 to 0.70 - excellent,
from 0.70 and more - grail.
Thus, the greater the mathematical expectation of the system and the smaller its standard deviation, the higher the quality of the system.
In our case, the mathematical expectation is a simple average of all transactions:
where xi is the result of the i-th transaction,
n is the number of transactions made by the trading system.
The standard deviation is the square root of the variance:
Well, how to find the variance, we have already analyzed a hundred times.
The technique of calculating estimates by the coefficient of Sortino
Sortino coefficient is the ratio of the mathematical expectation to the standard deviation of the results of transactions with negative returns:
where M (x) is the mathematical expectation,
σ 'is the standard deviation with a negative return.
The resulting value is classified as follows:
less than 0.24 - very poor quality,
from 0.24 to 0.30 - low,
from 0.30 to 0.38 - average,
from 0.38 to 0.45 - good,
from 0.45 to 0.75 - excellent,
from 0.75 to 1.00 - excellent,
from 1.00 and more - grail.
The negative standard deviation standard deviation is the square root of the average value of the sum of the squares of loss-making trades:
When calculating the deviation value, an important point should be taken into account: positive transactions are not excluded from the calculation, their values are replaced by zeros. This affects the number of trades (n) in the formula.
So how do you know if a system is no longer working?
Understanding that the system no longer works is very important for the trader.What to do if the account experiences a prolonged drawdown? The system stopped working, and it's time to remove it? Or is the drawdown period about to end? Many traders, especially beginners, have absolutely no action plan for such a case.
Many experienced traders will say that they plan to wait a certain period of time, and if the system does not begin to exit the drawdown, then it will be decided that it no longer works. Another popular approach is to wait for the double drawdown shown in the tests. But how correct are both of these approaches? Are they backed by statistics? How much time does it take to wait? Why is it a double drawdown, and not, say, a triple? There are no statistics on this subject, it’s just customary to do so.
And yet, how to understand that the system is broken? With the help of old, not verified statistically methods, or is it still worth a little thought? Let's try the second option for a change.
And we will begin to think about what it is all about when the system stops working. Well, this means that it no longer works, does not work as it was laid in. All we need to do is just compare whether the system works the same way as in the tests.
But what parameter to compare? We examined quite a few characteristics, but to evaluate the system’s performance, it’s worth highlighting one of the most important ones - the distribution of our transactions on a real account in comparison with this distribution on the test. All that needs to be checked is whether the selection of real trading deals is part of the selection from the tests. If you can refute the hypothesis that this is so with a certain level of confidence (usually 95%), then your system no longer trades as it was intended. So, she broke down and you can safely withdraw it from the account.
Using this criterion is a very powerful tool. After all, in this case you rely on such a science as statistics. And she is the best friend of the trader. And you do not need to wait another 1.5 months, wondering if the system will come out of the drawdown. No need to wait for two test drawdowns, losing money. Simple calculations in Excel (which we literally performed in 10 minutes in one of the lessons of the ExcelTrader course) - and you already have a clear, statistically confirmed, scientifically sound decision about whether to remove the system from the account.
So what do you need for this?
First, you need to do a system test on historical data. To do this, you can simply use any strategy tester, and then transfer the tester data, for example, to excel for further analysis. The most important thing is to get the results of the transactions themselves - profits and losses.
Secondly, you need data that you will verify. Many of you probably have account statistics on myfxbook. In the upper right corner of the account growth chart there is an “export" button. By clicking on it, you will see a drop-down menu where you need to select the format of the stored data. For our purposes, the csv format will fit:
Open the resulting file, prepare the data:
Clicking on “Text by columns”, we get into the text breakdown wizard. Choose a variant with a separator:
Myfxbook uses a comma as a separator:
Next, we bring the wizard to the end and convert the resulting data into a table:
Now we can sort the data by values and hide the rows we do not need. Find the column "action" and set the checkmarks in the filter as follows:
Now find the “gain” column, copy it and transfer it to a new sheet:
We no longer need the sheet with the statement, delete it:
Now open the file with the test results from the MetaTrader terminal:
Press ctrl + a and ctrl + c to select all lines of the file, then ctrl + v on a new sheet in excel:
Delete the header, leaving only the lines with deals:
Convert the data to a table:
In the filter of the “Profit” column, uncheck the empty lines:
Copy the “Profit” column:
And transfer it to the sheet on which the prepared column with transactions from the real account is already located:
So, we will compare these two columns. First, calculate the average value for the test values:
And for real ones:
We also calculate the number of transactions:
Then we define the standard deviation for the population (what was obtained on the test):
And the standard deviation for the sample (from a real account):
Next, we need to calculate the standard error:
And the last - z conversion:
Find the standard distribution calculator on the network or use the site shown in the screenshot:
Fill in the value of the z transformation, as shown in the figure above, and click calculate. As a result, we get the value of P:
In our example, P is 0.5054. This is much more than 0.05, so we can’t say that the data from the account is different from the data from the tests. Therefore, we can conclude that the strategy works within the framework of the conceived algorithm.
You can also watch a video where this technique is described (one of the lessons of the ExcelTrader course):
As you can see, the idea is very simple. We have a long piece of data obtained during the test of the trading strategy and there is a small piece obtained during the work on a real account. And there is a technique that allows you to determine whether this small piece is similar to the data obtained during the test. Of course, with a certain degree of probability - 95%. But in most cases this is enough.
So when to withdraw the system from the account?
For someone, a signal to check the system will be a drawdown of half the account. Someone will simply ignore the scientific approach to this issue and continue to use the "grandfather" methods. And especially the faint of heart will begin to test their systems with every drawdown of more than 10%.
In fact, the fact that the system is currently trading in profit does not mean at all that it is trading exactly as intended. In other words, a system trading in profit can also trade not as it was laid down in its creation. Most likely the reaction to this idea will be like this - if you earn money, you don’t need to touch anything. But a system that works "differently" also carries a danger, because it will just the same day begin to work in the red. In my opinion, it is worth checking the systems once a month, and if the vehicle losing money does not pass the test, it is ruthless to withdraw it from the account. So far, the systems that earn money can simply be taken on a pencil and monitored more carefully.
Trading is, first of all, statistics. And today we have analyzed a lot of indicators that somehow characterize the quality of the trading system. Of course, there are also a huge number of different coefficients such as the coefficient of Squid, Jensen, Sterling, Sortino, which were also invented to evaluate trade. However, the characteristics that are given in this article are quite sufficient for evaluation in most cases.
Separately, I would like to highlight a method for evaluating the performance of trading systems, which I mentioned above and which is discussed in the video course about Excel. This is a really powerful, scientifically sound way of making decisions about withdrawing a system from an account or continuing trading. I hope he helps you more than once.