The assumptions that will dominate the processes described in Part 2 are:

1. That price is an effect and not a cause, and
2. That the financial markets are non stationary.

The market price reflects not only the differing value opinions of many orthodox security appraisers but also all the hopes and fears and guesses and moods, rational and irrational, of hundreds of potential buyers and sellers, as well as their needs and their resources -- in total, factors which defy analysis and for which no statistics are obtainable, but which are nevertheless all synthesized, weighed and finally expressed in the one precise figure at which a buyer and seller get together and make a deal. This is the only figure that counts. [emphasis added]

In no way should this discussion be construed as anti-technical analysis. There are more trend following systems in use today, making more money, than any other type of system. That is precisely why there is such a great opportunity for non trend followers. The issue at this web site is Risk Management By Being DIFFERENT. Adding non trend following, non technical analysis, non price based, non stationary, non linear equity curves to an existing portfolio can help smooth the overall portfolio's equity curve. One issue in Risk Management must be diversification. And diversification can be implemented at all levels: indicator, model, instrument, and technology.

Not only is technical analysis concerned mostly with the movement of price, but it also fails to address the systematic changes which are attendant to price movement. The financial markets are non stationary. That is, the very underpinning structure of a particular market is evolving over time. This hidden movement is in addition to the observable movement of price as it reacts to various market forces. It follows that if the structure itself is moving around, then the usefulness of any parameterization must be expected to have a half-life. If the usefulness of a parameter set is expected to decay, then the parameters must be made to adapt. Consequently, what one builds must be more sophisticated than a "black box." Some quantitative analysts conclude that there cannot be a closed-form solution to the markets.

Only a "process" can handle this problem. An indicator cannot. A model cannot. Parameter set maintenance must be often enough to track parameter sets as they migrate in parameter space. Indicators can be constructed such that they adapt automatically to short-term market changes. The following discussion will show how.

The above graph will be used to illustrate visually what is meant by non stationarity. It depicts a surface of Sharpe Ratios³. Think of it as a contour map, a landscape including mountains and valleys. Each point on the surface is an altitude which reflects the goodness of an indicator (Z-axis) whose two principal components are the variables noted on the X- and Y- axes. Along the X-axis are parameters of a proprietary statistic, while the Y-axis depicts parameters used in filtering noise from input data⁴. The non-stationary market assumption may be envisioned as a Sharpe Ratio surface which is moving about over time. Today, a set of parameters x_i and y_j may produce Sharpe Ratio z_i,j, but a month from now the same parameters will produce a different Sharpe Ratio z'_i,j.

Second, a univariate filtering transformation is used on the raw indicator time series to remove varying amounts of noise. Third, a statistical calculation is performed on the filtered time series producing the final time series. The final time series is an indicator in its own right which oscillates about zero, generating buy and sell signals as it crosses through zero. These signals are used in a trade simulator which calculates the Sharpe Ratio of the resultant equity curve for each parameter pair. For present purposes each of these indicators will be called a sub-indicator, reserving the designation of indicator for a more robust time series made up of several sub-indicators combined in some useful way.

The non stationary market assumption infers that the Sharpe Ratio surface in the above graph is moving about over time. A point which is downhill from the peak today may climb up the hill (see footnote 1 in Part 1) and become the new peak next week.

The innovation being described here involves how the set of sub-indicators, represented by parameter pairs, are used in the construction of a single more robust indicator. First of all, the whole surface must be inspected to determine that there are not any sharp peaks or deep chasms. It would not be wise risk management to use a family of parameter pairs which describe such a risky surface. Crises happen by themselves; there is no sense planning them. A common mistake made by novice model builders is called "telephone pole optimization." There frequently are very small groups of parameter sets which are vastly superior to their immediate neighbors. It would appear as a very high peak or telephone pole on our Sharpe Ratio surface. It is tempting to put those indicators into the market. It is wise to put them into the trash heap. Since the markets are non stationary, indicators constructed with such parameters are bound to fail.

Next, a robust⁶ area of the surface is located. Then a lead sub-indicator is chosen to represent that area. To accompany the leading sub-indicator, a group of five to ten other candidate sub-indicators is chosen, with sufficient care given to assure some variety. Only a few of the sub-indicators should be chosen from inside and near the snow-white peaks in the graph. This selection is made weekly, which minimizes the computer intensive optimization activity. An extremely valuable side benefit is not having to be involved in any computationally intensive activities between the time the regular trading sessions close and the Globex opens.

The variety is necessary to increase the probability that as the lead sub-indicator migrates downhill a neighboring sub-indicator will migrate up the hill, replacing it as lead-indicator for tomorrow. Stated another way, the variety provides diversification at the indicator level.

The weighted sum of sub-indicators should have several good characteristics. It should reasonably minimize the co-variance among those chosen. Those chosen should not be chosen simply to fit the objective function better at the expense of predictivity, however. A good curve-fit is not the ultimate objective — a predictive indicator is.

Consider that the sub-indicators have been carefully selected as varietal samples from a robust Sharpe Ratio surface. The points on the surface represented positive Sharpe Ratios — not negative. Consequently, let us define them as right-side-up sub-indicators. It might seem useful to consider using ordinary linear regression to synthesize the sub-indicators into an indicator, fitting the objective function in a least-squares sense. But, linear regression has no conscience. It has one task to perform — minimize the sum of the squares of the residuals. In order to do that it will freely assign negative coefficients. But, if we allow regression to use negative coefficients, we also allow regression to turn our sub-indicators up-side-down. While this may give a better curve-fit, it will do so at the expense of predictivity, subverting our whole process.

Also, linear regression does not understand that our task is to emulate near-perfect market trading, represented by an oscillator which fires off a buy or sell signal as it crosses through zero. Again, linear regression's task is to minimize the sum of the squares of the residuals. Regression is insensitive to where the resulting curve crosses through zero. And crossing through zero is all that we are concerned with in the financial markets. This might be less true if our trading frequency is very low, but that may not be the case. Statistical validity may require that we trade seven trades each month. Certainly, the more degrees of freedom we allow in our indicators, the more frequently we must trade, and the more sensitive our synthesis of indicators must be to when the resultant indicator crosses through zero. In the extreme case, a very fast trading indicator which is otherwise perfect, but one day off, may lose every trade.

Furthermore, linear regression does not consider that recent market action might be more important than distant market action. Regression will dutifully seek to minimize the sum of the squares of the residuals. If there are large residuals toward the beginning of your chosen indicator construction time horizon, regression will do everything in its power to minimize them at the expense of recent market activity.

And fourth, regression will allow one stubborn (large) residual to lock your indicator into a LONG or SHORT position until enough time has elapsed for that residual to be dropped off the front-end of your construction horizon. Regression will cease being influenced, not by present market activity, not by good judgment regarding being LONG or SHORT, but by the data in question falling off the front end of the chosen time period. And, these are only four of linear regression's faults. Consequently, linear regression is not a good algorithm for synthesis of indicators. The algorithm chosen to synthesize sub-indicators must, at the least:

Preserve the integrity of the inputs,

Optimize the synthesis of inputs based on trade timing issues,

Add value (lower the covariance) with every additional input,

Weight recent market activity more heavily than activity in the distant past, and

Avoid being locked into a position for the wrong reasons.

[1] Richard W. Schabacker, Technical Analysis and Stock Market Profits.
[2] Robert D. Edwards and John Magee, Technical Analysis of Stock Trends, p. 5.
[3] The Sharpe Ratio is a measure of goodness in which rewards are adjusted by the risk involved in achieving those rewards . It is the annualized ratio of the mean of periodic market gains (or losses) and the standard deviation of those gains (or losses).
[4] If one were choosing buy/sell signals based on the crossing of two moving averages, the x and y values could have been the numbers of days used in the moving average calculations.
[5] It is assumed that the discontinuities due to option expiry had been removed prior to computing this ratio.
[6] Robustness involves many issues, including high Sharpe Ratio, fairly flat surface, and sufficient trading decisions to be considered statistically valid considering the degrees of freedom in the optimization process. Here, assuming five or more trades / month, each year of data should have 60 or more trading decisions, which is 30 decisions for each of two DOF.
For further discussion, see Statistical Validity.

The Importance of this Website to Your Business – As the markets become more volatile, you would do well to train your quants to protect your portfolio against the ill effects of nonstationarity.

Exogenous Data Based Models – The good and bad characteristics of Exogenous Data. (Don’t miss the interesting visualization of some SPX Index Option data.)

Visualization of Exogenous Data – In case you missed it above.

Quantitative Analysis Platform – A user-friendly modeling platform for improving the productivity of quantitative analysts.

Overview – Advanced Automated trading Systems.

Consulting Services – Helping your quantitative analysts deliver a better product for your clients. 

Trading Model Building Services – Continuous and Discrete Models, using Price or Exogenous Data. 

Quantitative Analysis Training Seminars – Topics covered in typical training seminars. 

Model Validation – A Catch-22 in the struggle between the Central Limit Theorem and the “Law” of Requisite Variety.

Non Trend-Following,  Non Technical Analysis Methods – The difference that a non-price market view can make in your portfolio’s success.

Back to Home Page

© 1997-2004 Thomas W. Wright. All Rights Reserved