Using a Bayesian Probability Framework in Investing

Recency bias is a very real problem in the investment decision making process as it causes widespread bad behavior in investors and traders. This happens to individuals and professionals alike – nobody is really immune to it. 
Current news, price movements and other information are routinely judged to be much more important and reliable than they actually are. This phenomenon tends to become more pronounced the closer we watch the market and the stronger the news sentiment of the moment is.

The momentary climate is a good example: a confusing barrage of news causes many investors to overweight current uncertainties, assume the worst and react accordingly (we can see elevated flows into money market funds and put buying) – all while US equity markets chop around within a few percentage points of all-time highs.

In investors´ minds long-term probabilities are swamped by this recency obsession. This is problematic, because the long-term probability distributions for the return of different asset classes are much more reliable than medium- or short-term interpretations of news flow or price action. Rather than keeping these long-term risk premia at the forefront of the decision making process, we tend to assign too much certainty to our current predictions. Portfolios influenced by this tend to underperform considerably as they continually buy high and sell low. 

In the investing realm, dominated by uncertainty and randomness, it can help to use tools to keep the big picture as the basis of decisions to avoid being swamped by short-term noise. Knowledge of a behavioral bias, like the recency bias, will not make you immune to it as these tendencies are very much ingrained in the make-up of the human mind. We need to use a deliberate mechanism to force ourselves to weight all the available information equally and not just use the most recent news or data in a good investment process.

Bayes’ theorem 

A great tool one can employ, wether you are a long-term value investor, a short-term trader or any other type of investor, is a decision chain based on probabilities using a Bayesian process.

The basic idea is quite simple: an evidenced based prior probability establishes the base-rate for the probability of an event to occur. For example, we can use the available stock market data of the past 70 years (or more) to calculate the probability that we will see higher stock prices one year from now – this is about 73% for the S&P 500.
We then use current information, that influences our interpretation of probabilities for the event to occur, to update our prior probability. This step is continually repeated for each new piece of information we judge to be relevant.

The math for using the theorem can be found on Wikipedia, or a deep dive into the application into gambling and investing in this post by Philosophical Economics. But I would argue that the naive approach of simply using the baseline probability and then averaging in our current market interpretation to reach an approximate new probability will get us 99% of the way to reach a good decision.
That is because our edge – the ability to accurately judge wether probabilities are skewed one way or the other – is likely to be very small and uncertain with randomness being the dominant factor. Assigning correct probabilities to plug into a formula for a better result is next to impossible.

The tools for assigning current probabilities are individually diverse – they are what makes up every investor´s process. I will give a concrete example, based on some of my own ideas and processes, to illustrate how this can be implemented advantageously without using complex mathematical formulas.

I use a probability based decision tree to always keep the big picture in mind. From long-term probabilities I filter down through the different time frames to reach my current portfolio decision: exactly how long- or short- biased should my equity allocation be at any given time?

The process

I will illustrate the approach going through the time frames that I judge to be influenced by the long-term base rate – all the way down to the short-term (1-8 weeks). Depending on your process the approach can be cut short at any step – in fact it may be wise to stick to the longer-term as the reliability of probability judgments tends to deteriorate rapidly when moving down the chain.

  1. The long-term base rate is the equity risk premium which I judge to be very reliable. The prior probability is reached by the statistical analysis of 70 years of stock market returns. This is the base case for the buy-and-hold index investor.
  2. The first adjustment is made by judging current long-term probabilities. For this I use a systematic process, I call the Meta Strategy, that in my opinion can add a lot of value. My default is to be exposed to the equity risk premium as the basic probability for good returns is very high. A combination of fundamental and technical indicators informs a view of current market conditions, that leads to a probability distribution of stock market returns over the next year or so. This is the basis for an active portfolio, that adjusts its equity allocation over the major economic cycles.
  3. The next step is to look at medium-term factors that influence my outlook over the next 3-6 months. Here I use a range of market studies and indicators – often from good outside sources. Averaging across studies I create medium-term probabilities which help to judge how serious corrections are likely to be or how sustainable or prone to pull-backs market advances are. These are used to adjust the probabilities I derived in step two to take advantage of intermediate, large market fluctuations.
  4. The last area I look at is the short-term, 1-8 week, time frame where a lot of mean-reverting market swings around the medium-term trend take place – the way prices move away from their 50-day moving average, come back to it and often cross below towards the long-term trend. I use this quite a lot in practice to increasingly hedge my portfolio in rising markets and to add to risky strategies during pull-backs. Price movement over his time-frame is highly random and judgements are error prone, nonetheless I often find my intuitive convictions to be very high. This is exactly the type of irrational behavior, that I try to correct through this process by forcing myself to consider the probabilities derived from steps 1 to 3 to provide a check to irrationally confident predictions.

The current market as an example

The main point here is to illustrate how to use Bayes’ theorem in spirit.
By using fairly approximate rules of thumb and skipping a lot of the math, because our judgments are bound to be inaccurate in any case, it is still possible to gain a lot of practical value from the process.
How to arrive at the exact probabilities at each step is entirely up to your own judgment and you may well disagree with my assessment of current market conditions – simply plug in your own predictions as long as they are probabilistic.
An individual process may stop after step 1 or 2 or continue all the way down into intraday strategies. The same decision making chain can be used in every case.

Step 1 – Establishing the base rate

The historic probability of a positive return in the S&P 500 over different time periods. The equity risk premium, though varying greatly over shorter time periods, has been remarkably consistent over rolling 20-year periods. These are very high probabilities for strong returns (around 9% annually) over longer time periods and the influence of the equity risk premium is considerable even down to day-to-day market returns.

The main use of this table is to put a basic year-over-year probability for a rising market at 70% to 75% and for a 3 to 6 month period at  60% to 70% as the starting point for all following calculations in the decision making process.

How to translate these basic probabilities into an actual portfolio allocation is largely a matter of individual risk preferences. To me the equity risk premium is large and probable enough to consider a 100% equity allocation my base case even with the possibility of 50%+  drawdowns – all the tilts in the following steps derive from this (high) risk preference and are not advising anyone to do the same.
Other investor may consider a diversified global asset allocation portfolio or simply a larger bond allocation (e.g. the classic 60/40 portfolio) more appropriate as their base case.

Step 2 – Long-term probability adjustments

The actual yearly return of the S&P 500 has varied greatly with a higher likelihood to see outsized negative returns around economic contractions and during downtrends.
As a rule of thumb signs that we are in an economic expansion and a long-term uptrend will skew probabilities for positive returns up to tilt my portfolio towards a leveraged equity allocation.
Vice versa increased danger signals for a economic contraction and a long-term downtrend in the S&P will change my portfolio allocation in several steps to 0% equities and 100% in safe or alternative asset classes.

This fairly radical tactical asset allocation is the core of my systematic Meta Strategy philosophy: leveraged equity exposure in good times and a complete switch to safe assets in dangerous times to protect my capital.

In the current monthly newsletter “The Meta Strategy ETF Portfolio“ my indicator analysis on October 5th reaches the following conclusion:

With only one of the six leading economic indicators used in the strategy signaling a warning and all technical levels signaling a solid uptrend my current verdict is „all good“, despite the headlines screaming recession warnings at us on a daily basis.

I use this analysis to adjust my probability assessment for a positive S&P 500 return in the next 6 – 12 months upwards to 80%. This is simply an average of the base assumption of 70% to 75% and the Meta Strategy assessment of a 85% to 90% probability for positive returns.

Step 3 – Medium-term probability adjustments

Over the last month a slew of news has turned investor sentiment down and uncertainty seems to be on the rise.
Analyzing a number of indicators and data points by studying their market impact when comparing current conditions to similar historic occurrences has shown largely positive returns from this set up (I look at the analysis of many different investors that I have come to trust and pay close attention to the studies that sentimentrader.com publishes on a daily basis for paying subscribers – as well as integrating my own ideas and backtests).

Here are some things I pay attention to to reach an overall bullish conclusion:

  • High small trader put buying (> 25% of volume)
  • Rising market breadth and breadth thrusts throughout the year
  • Hard economic data rebound
  • VIX falls back to a low reading of around 13 in mid September – bull market tops are usually accompanied by rising volatility levels with lows above 15

Overall this leads me to a bullish assessment with a 75% probability; higher than the medium-term base case of 60% – 70%. This is close enough to the long-term probability assessment to leave my long-term equity portfolio unchanged and to provide a strong bullish bias to the short-term analysis in the next step.

Step 4 – Short-term probabilities

Here things become more and more difficult to analyze with probabilities clustered closely around a random outcome and myriads of technical analysis methods (many of which are quite useless) and interpretations to choose from – it absolutely is a rational decision to ignore these fluctuations altogether.

I have decided for myself to allocate 20% to 30% of my portfolio to deviate from my core exposure which I implement via broad index ETF (leveraged, plain vanilla or inverse depending on my allocation decision). On the one side I hedge my portfolio with some short exposure, on the other I use short volatility strategies to further enhance equity market returns temporarily. The exposure to this alternative strategy overlay may vary day-to-day or stay stable over the course of several weeks depending on the market.

The strongest edges, I found, center around the mean-reversion tendencies the market displays around a 1-month time window.
The easiest way to implement this is to simply add exposure during pull-backs, that I judge to likely be temporary (in steps 2 and 3) and to take profits and increase hedges during a rise to new highs when pullbacks become increasingly probable. To fine-tune entry and exit points I pay attention to price behavior in many different ways using ideas of support and resistance areas (especially old highs and lows), the 60-day and 275-day moving averages and many other „voodoo“ ideas.
To be honest, I’m still figuring out if this actually adds any value to the simplistic approach of adjusting exposure at set intervals without any regard for technical levels.

Recently, for example, I used these inflection points to change my short-term probability outlook:

  • Breakdown from a small top formation near all-time highs (double top) on September 24th and again October 1st 
  • Resistance areas around the August lows as possible targets where the drop might stop
  • High likelihood to retest lows after the first rebound on October 3rd and 4th
  • and many other ideas

Here are two key assessments that caused me to change my short-term exposure considerably on October 4th and 8th. These examples illustrate the continuous process, that concentrates on overall probabilities filtered down all the way from my base rate assumption – rather than individual signals:

On October 4th we saw a first bounce from a considerable market drop which my backtests indicate to have an uncommonly strong probability to retest lower levels which I put at 30% for a partial retrace of the bounce; 25% to hit the last low and 15% to go to the August low or lower = a 70% overall probability on the short side.

Averaging this high probability to see a falling market over the short-term with the bullish medium-term indication from step 3, I hedged my portfolio to neutral exposure (instead of going 70% short as my over-confident intuition would tell me) and waited for the next level at either 2900 or at a close above 2950/60 where I saw the next inflection points to strengthen the bullish case.

On October 8th the first of the lower levels was reached at 2900 and I changed my probability assessment accordingly:
My first lower target at 2900 was reached which reduced the probability for a further falling market to the sum of the remaining probabilities to reach 2855 or 2800 (or lower) = 40%.

Short exposure was reduced and long exposure added to match the new probability (the average between medium-term bullish bias and short-term neutral to bullish bias) of 70% long / 30% short.

Further adjustments will be made when lower levels are hit or if we see a close above the 60-day moving average. When full long exposure is reached things will likely quiet down for a while until either the all-time high is reached or the long term trend at the 275-day moving average is broken.

This is my detailed thought process for a continuously adjusted portfolio exposure through the markets fluctuations over different time frames. Using effective strategies that get paid for hedging (writing covered call options) and that add a strong risk premium (the volatility risk premium for short volatility strategies) has led to an outperformance over the tactical long-term Meta Strategy Portfolio by over 30% this year.

I realize that my market assessment is very unlikely to be the same as many other investor´s, but you can use this illustrated process to plug in your own probability estimates – whatever they may be.

The framework should help to avoid recency bias, the short-termism and over-confidence, that are a big problem to overcome for many investors and traders. It also enables you to better juggle several competing scenarios for the future in your head at once – an unnatural way to think which reflects the realities of investing quite well. I found a quick visual sketch to be helpful to keep the process clear in my mind – each sketch represents one step in the process and is based on the previous assessment.

Best of luck!

David

2 comments

Leave a Reply