Overfitting or opportunity set: Alternative risk premia performance
Investor demand for new opportunities, lower-cost products and better liquidity is often outstripped by providers willingly feeding such demand. Alternative risk premia (ARP) is no exception. With a healthy flow of capital gravitating towards ARP products in recent years, investment managers and investment banks have come in droves, introducing new funds, carve-outs of existing strategies and, in the case of banks, a seemingly endless list of ARP strategy indices.
Investors have rightly questioned the volume of strategies and the speed at which they are produced. Most have very short live track records, but are accompanied by impressive backtested returns. Interestingly, investors are much more skeptical of track records marketed by banks than they are of investment managers’ simulated returns. This skepticism stems from an inherent distrust of banks, but is supported in many cases by disappointing realized performance relative to backtests.
Investors that have experienced these poorer returns have understandably been led to the assumption that bank strategies must be overfitted – a data-mining process whereby models’ rules are excessively altered and complicated such that it produces the most attractive return stream looking back, but will most likely be a poor predictor of future observations.
Adding fuel to the fire, investment managers whose own businesses are under threat from bank products are keen to highlight the various potential conflicts of interest that exist with bank indices and abuses of overfitting. As investors in bank indices and allocators to investment manager strategies, we find that in most cases, these claims are unsubstantiated due to a lack of sufficient knowledge and due diligence.
We were therefore keen to offer an objective view of live versus backtested performance from bank ARP strategies. But we also wanted to go a step further and explore whether different environments during backtested and live periods had contributed to differences in risk-adjusted performance.
Live versus backtested performance
Using our database of bank ARP, we analyzed daily data of 747 ARP indices offered by 11 global banks across five different asset classes: commodities, credit, equities, foreign currency and interest rates. Within this dataset, the average backtest length and average live track record is 11.6 years and 3.9 years, respectively. We found a median deterioration of risk-adjusted return (RAR) of 83% (mean deterioration 75%). This is consistent with Suhonen et al.  , which finds a median deterioration of 73% over 215 strategies. However, through individual inspection of each index and with the help of some simple statistical analysis, we find that many of these “flagship” strategies are in fact close derivatives of other strategies. For example, one bank may offer a Global Equity Value strategy, and a Global Equity ex Japan Value strategy. By filtering out these “duplicates,” reducing our universe to 228, we were able to concentrate on a more representative set of top-level ARP strategies. Performing the same analysis, we then found a median deterioration in RAR of 62% and mean deterioration of 48%. We can also state the deterioration in absolute terms, because percentage change is impacted by the starting RAR level. Here we find the average reduction in RAR to be 0.51, as shown in Exhibit 1.
Exhibit 1: Risk-adjusted return changes
Soucre: Aberdeen Standard Investments, April 30, 2017.
We are not shocked to see some deterioration, as live model performance rarely surpasses simulated performance; this is true of both investment manager and bank strategies. We note that 27% of the strategies actually produced better live results, but overall it is understandable that investors feel aggrieved by live performance.
But how much of this is due to overfitting? Is the relatively short live period enough to draw this conclusion, or are there other factors negatively impacting ARP performance in the more recent years?
Effects of economic cycles
Upon investigation of RAR using one-year and three-year rolling windows, we observe that RAR is not constant over time and exhibits a cyclical behavior. Can we link this behavior to economic cycles? Using business cycle data published by the Economic Cycle Research Institute (ECRI), we calculate the backtest RAR in the expansion and contraction periods separately. We adjust backtested RAR to account for the length of time the live period has been in a state of expansion or contraction, making the backtest and live datasets more comparable. We saw a marked improvement in the RAR difference from backtest to live periods, from an average of -0.51 in our original result to -0.36 (see Exhibit 1).
Next, we investigated the effects of the federal funds rate on the performance of ARP strategies. In the analysis, we look at performance according to three states in the interest-rate cycle: “rate hiking,” “rate cutting” and “rate stable.” As shown in Exhibit 1, we see that the average difference in RAR between backtest and live periods is reduced when adjusted by the federal fund rate cycles, from -0.51 in the original result down to -0.25, suggesting a significant effect of interest-rate cycles on the performance of ARP strategies.
We investigated the median RAR of ARP strategies broken down by asset classes and investment styles in the three interest-rate environments. The results are striking in that nearly all ARP strategies perform far better in “rate cutting” periods and, to a lesser extent, “rate stable” periods. This observation provides a partial explanation for the deterioration in RAR in live periods for ARP strategies, as the interest-rate hiking environment (and expectations of a continuation of tightening) in recent years has not been supportive of these strategies. We can see that the point of tightening can be detrimental to most ARP, although we appreciate that an absolute higher level of interest rates may be a more favorable environment.
We repeated the analysis using inflation rising/falling cycles and U.S. high/low unemployment cycles. The adjusted mean deterioration in RAR is -0.33 for the former case and -0.31 for the latter, as shown in Exhibit 1. Overall, there is reasonable support for our view that the short live period provides insufficient data to conclude overfitting is the sole cause of poorer live performance. Moreover, we find that basic regime states can have a meaningful impact on ARP performance.
With that in mind, should investors still be excited about ARP, and is its recent growth justified?
The benefits of diversification
While we may have experienced lower RARs across the majority of ARP strategies in recent years, for many these RARs were still positive. We are familiar with the old adage of diversification and a free lunch, so we compare the pairwise correlations between indices in backtest and live periods, which confirm that the benefits of diversification remain stable. We do see a slight increase in pairwise correlations, which is expected since the live periods are generally much shorter than the backtest periods. In both periods, the ARP indices exhibit diversifying behaviors.
Finally, we construct a portfolio of the 228 ARP strategies with a simple equal-weighted allocation to illustrate the performance of diversification. Exhibit 2 shows the rolling one-year RAR of the portfolio since August 1990. While highly cyclical, the RAR over the whole period is 2.71. Even when considering the period after the sample’s median live date of August 4, 2014, we still achieve a respectable RAR of 1.22 for a diversified portfolio.
Exhibit 2: Rolling 1 Year RAR of a portfolio with equally weighted risk premia strategies
Soucre: Aberdeen Standard Investments, April 30, 2017.
There can be no arguments that live performance of bank ARP strategies has been lower than backtested results suggest. While we cannot rule out the presence of overfitting, we see encouraging evidence that economic environments and interest-rate cycles may have contributed to lower returns. Regardless of the drivers of weaker performance, investors are disappointed that results have been well below expectations set by strategy providers. We believe that investors should continue to be optimistic about the opportunity set in ARP, particularly when accessing them through a cross-asset multi-strategy approach. There is clear evidence of the existence and persistence of diversification between these strategies, leading to attractive RARs when combined.
Furthermore, we see considerable cyclicality in RAR across ARP strategies, meaning that nearer-term underperformance could reasonably be expected to show some mean reversion. In an environment where hedge fund returns and volatility remain low, investors continue to search for yield, and fees and liquidity are paramount, we believe that recent interest in ARP is more than just a craze, and rather an indication that investors are taking meaningful steps to improve diversification and efficiencies in their portfolios.
Full analysis can be found in the working paper: Yip, W and Moir, D. “Overfitting or Opportunity Set? Exploring Alternative Risk Premia Strategy Performance,” 2017.
Simulated backtested performance and actual live performance are not indicators of future actual results.
The hypothetical portfolio is provided for illustrative purposes only and assumes monthly rebalancing back to the original target equal weights. In a real environment, these weights will drift based on the performance of the underlying indices and will not necessarily be rebalanced to the target weights. The simulated performance for the hypothetical portfolio is gross of all fees and expenses. Had these fees and expenses been incorporated the returns shown would have been lower. Indexes are unmanaged and have been provided for comparison purposes only. No fees or expenses are reflected. You cannot invest directly in an index.
Results are calculated by the retroactive application of a model built upon assumptions regarding liquidity, market factors, global events, and adherence to the model. These assumptions have been made for modeling purposes and are unlikely to be realized. Changes in these assumptions may have a material impact on the returns presented. No representations and warranties are made as to the reasonableness of the assumptions. No live portfolio has achieved the results shown nor should it be inferred that any portfolio will perform in a manner outlined in this article. When interpreting the results, an investor should take into consideration the limitations of the model applied, including that the model was prepared with the benefit of hindsight, hypothetical trading does not involve financial risk, and no hypothetical analysis can completely account for the impact of financial risk in actual trading. Further, simulated modeling allows for the security selection methodology to be adjusted until past returns are maximized. The hypothetical portfolio shown is not currently available for investment.
Please note: This document was originally written in 2017, and is being provided as a demonstration of Aberdeen’s track record of research. Any analysis performed and/or data in this document is representative of that timeframe and should not be relied upon to be illustrative of the current environment.
Alternative investments involve specific risks that may be greater than those associated with traditional investments; are not suitable for all clients; and intended for experienced and sophisticated investors who meet specific suitability requirements and are willing to bear the high economic risks of the investment. Investments of this type may engage in speculative investment practices; carry additional risk of loss, including possibility of partial or total loss of invested capital, due to the nature and volatility of the underlying investments; and are generally considered to be illiquid due to restrictive repurchase procedures. These investments may also involve different regulatory and reporting requirements, complex tax structures, and delays in distributing important tax information.
A version of this article was previously published in EQ Derivatives in October 2017.