By Victor Haghani, James White and Jerry Bell Estimated reading time: 10 min.
Introduction: Back to the Future
In the 1989 blockbuster Back to the Future II, time travel enables Michael J. Fox’s nemesis, Biff, to become a gazillionaire by bringing an almanac with sports match outcomes back from the future. We thought it might be instructive, and certainly entertaining, to make a less fanciful version of this dream a reality – for a few lucky people.
In November 2023, we ran an in-person, proctored experiment involving 118 young adults trained in finance. We called the experiment “The Crystal Ball Challenge.” We gave each participant $50 and the opportunity to grow that stake by trading in the S&P 500 index and 30-year US Treasury bonds with the information on the front page of the Wall Street Journal (WSJ) one day in advance, but with stock and bond price data blacked out. The game covered 15 days, one day for each year from 2008 to 2022.
You can play this game for yourself here: Crystal Ball Trading Challenge – though without the pecuniary component. As of the time of writing, over 1,500 people have tested their skill and luck by playing the game on our website.
Summary of results
The players in the proctored experiment did not do very well, despite having the front page of the newspaper 36 hours ahead of time. About half of them lost money, and one in six actually went bust. The average payout was just $51.62 (a gain of just 3.2%), which is statistically indistinguishable from breaking even. The poor results were a product of: 1) not guessing the direction of stocks and bonds very well, and 2) poor trade-sizing. The players guessed the direction of stocks and bonds correctly on just 51.5% of the roughly 2,000 trades they made. They guessed the direction of bonds correctly 56% of the time, but bet less of their capital on bonds than on stocks (if you’re planning a career as a proprietary macro trader, consider putting your focus on bonds).
Perhaps the front page of the WSJ isn’t a particularly clear crystal ball, or our players weren’t very adept at reading it. As former Goldman Sachs CEO Lloyd Blankfein reminded us in a widely-circulated tweet, sometimes the markets don’t react to the news as even seasoned experts expect – an important lesson all successful traders learn, eventually.
It didn’t help that the players also did not seem to know how to size their bets well. On eight of the 30 trading opportunities,
These players all finished with gains. On average, they grew their starting wealth by 130%, with a median gain of 60%. All of the players were selective and highly variable in their trade-sizing. They did not bet at all on about 1/3 of the trading opportunities, but bet big on days when they presumably felt confident in the impact of the news on stock or bond prices.Betting on a Biased Coin.” We recently wrote a book, The Missing Billionaires: A Guide to Better Financial Decisions, that is focused on investment sizing.
Over 90% of the participants were in graduate programs in finance or MBA programs with finance modules at four east coast US universities with low admission rates. The participants were not told in advance that they’d be invited to participate in this experiment. Any who did not want to participate were allowed to leave (though no one did). Here’s how we explained the rules of the game: We are giving you $50 to play our “Crystal Ball” game. The object is to see how well you can do trading stocks and bonds if you know the news from the front page of the WSJ one day in advance. In other words, you’ll be in that dreamed-of position of being a trader who “knows the future.” For example, you will be shown the front page of the WSJ for a Wednesday, and be able to take a long or short position in the stock market and in the bond market at prices prevailing at Monday’s close (that is, two days earlier). Your trades will be liquidated at Tuesday’s closing prices. Note, on each front page we’ve blacked out anything that tells you explicitly what market prices actually did that day – leaving that information in would make this game too easy and no fun at all! You will be trading the S&P 500 stock market index and a 30-year US Treasury bond futures contract, and you can use as much leverage as you’d like to, up to 50x. Use the sliders to choose the positions you want and then click the “Trade” button. Remember that for the 30-year Treasury bonds, prices go down when yields go up, and you are trading on price. You are starting off with $50 of bankroll, and we will pay you however much this has grown to, or shrunk to, with a maximum payout of $100. You will have 45 minutes to play the game. We have not chosen these days to try to trick you – they are randomly chosen. You will be able to trade on 15 different days, once per year over the past 15 years. The days will be presented to you in a randomized order. We’ve chosen these days randomly from a set of days where one third of them are days of employment reports, one third from days of Fed announcements, and the other third purely randomly, all taken from days that are in the top half of days ranked by overall market volatility. You can use the “Skip” button to skip any day you don’t feel like trading, and you can trade stocks, bonds, or both, each day. You can use the “Finish” button if you want to stop before being presented with all 15 days. Leverage of 1x means your position size is equal to your capital size. There are no transaction costs or overnight financing costs or rebates on your trades. Good luck, and have fun – you may never have this opportunity again! Below are two pictures of the screens that players engaged with. In the first screen, the player can expand the picture of the front page of the WSJ on the left to be able to read it more clearly, then revert to the trading page. Then, after clicking “Trade” the result of the trade is revealed, showing the market move in stocks and bonds, and the resultant profit or loss. The player’s bankroll is expressed based off of a starting value of $1 million, but the players understood that their payout would be $50 times the ending wealth divided by $1 million, with a minimum of $0 and a maximum of $100. All the front pages can be seen here, in chronological order: https://elmwealth.com/crystal-ball-gallery/ “He who lives by the crystal ball will eat shattered glass.” — Ray Dalio Was Taleb correct in his conjecture that “If you give an investor the next day’s news 24 hours in advance, he would go bust in less than a year”? While our experiment didn’t test his statement precisely – we only gave players 15 days of front pages, players were risking just $100 in the game, etc. – by and large we think Taleb is right. His counterintuitive proposition is both insightful and instructive. The financial industry is replete with individuals and organizations constantly working to develop their own proprietary crystal balls. We hope that the experiment and results described herein convince crystal ball makers that sensible investment-sizing is essential to realizing the value of what they are trying to build. The poor aggregate showing of our 118 financially-trained participants highlights the importance of educating young, aspiring finance industry professionals in decision-making under uncertainty, and particularly the theory and art of investment-sizing. We hope our Crystal Ball game will be a helpful tool – or a prototype for a better one – that educators and financial firms can use to teach these concepts and skills. Perhaps it may even become part of the hedge fund boot camp training programs at Citadel, Point72, Balyasny, and Jane Street that have been in the news recently.“Hedge Fund Talent Schools Are Looking for the Perfect Trader.” Bloomberg. Perhaps Matt Levine foresaw the results of our experiment with his article titled: “Knowing the Future Isn’t That Helpful.”“Knowing the Future Isn’t That Helpful.” Money Stuff. Bloomberg. Below is a chart showing the distribution of payouts to the players. The average payout was $51.62 per player, representing a weighted average return across all the players of 3.2%. The players forecast the correct direction of stocks and bonds 51.5% of the time. With stocks, they got the correct direction on 48.2% of their trades, and on 56.2% of the bond trades. Notice there were four days where more than 70% of the players forecast the correct direction of the market, and 10 days when more than 60% were correct. However, the players placed 40% more trades in stocks than bonds, which is unfortunate given the players had a better realized edge with bonds than stocks. The table below describes each of the 15 trading opportunities and shows how many trades the players placed on each of the days and what percentage of the trades were placed in the correct direction: long when the market went up and short when it went down. The next table puts the focus on trade-sizing. It shows the average leverage used for each trading day for stock and bond trades, and also the averages conditional on being higher than 5x leverage. Average leverage used in stock and bond trades was 13x and 10x respectively, and two times as much – 22x and 20x – for trades where leverage was greater than 5x. We calculated the correlation of leverage (i.e. trade size) for each day versus the win percent for each day, and found a zero correlation in the case of stock trades and a -0.1 correlation for bond trades, along with a +0.2 and -0.1 conditional on leverage used being greater than 5x. It seems that our players on average did not follow a strategy of placing bigger trades on those that they had a higher probability of getting right. Perhaps this is due to them not knowing which ones they had a higher probability of getting right, or perhaps they were not following a disciplined sizing strategy. The players traded 2,067 times, for an average of about 18 trades per player. The maximum number of trades each player could have made is 30, which would entail doing a stock and bond trade for every front page. Some of the 40% shortfall versus the maximum number of trades is due to 16% of the players going bust, but most of the shortfall is from players abstaining from trading opportunities. Players were more apt to take long positions in stock and bonds; they traded stocks 62.5% of the time as a long position, and 59.6% of the time for bonds. About 10% and 8% of the players were long stocks and bonds, respectively, for every trade they made.The Game
Conclusion
“>5 The uniformly positive results of the five experienced macro traders we invited to play the game suggest that there are teachable skills involved in successful discretionary investing.
“>6 He describes a delightful academic study that analyzed the trading results of a cartel of investors with an excellent, albeit illicit, crystal ball.Crystal Ball Trading Challenge
Appendix I: Detailed Analysis of Player Results
However, even with their weak ability to read the tea leaves, the players could have done quite a bit better if they applied a sensible and constant amount of leverage to all their trades. It would have been reasonable for the players to have estimated the daily standard deviation of stocks and bonds, given our description of how we chose the 15 days, at around 1.5% – 2% for stocks and 1% – 1.5% for bonds. They might have then considered that there could easily be a two or three sigma event in the sample, and so the maximum amount of leverage they could use with a low likelihood of being wiped out might have been 8x for equities and 12x for bonds, if betting on both at the same time.
With that maximum leverage in mind, the next step would be to find the optimal size, subject to the maximum constraint above, given their view of the expected return and risk of the trades.
However, a much bigger improvement could have been attained from the players doing a better job discerning when they had a more accurate reading of the future. If they’d scaled their trades bigger when they were more likely to be right, they’d have done much better, but it’s hard to know if, on the days when a high percentage of players put on the correct trades, if they really did have a stronger conviction that they were going to be right. It was clear which days were employment and Fed announcement days, and it turns out that the players were more accurate in their readings for bond movements on those days with a 58% hit ratio, while on the other third of the days they only had a 50% hit ratio.
The players also could have done much better if they had based their forecasts on a simple set of rules around the news that was on those front pages. They’d have been correct about 60% of the time if they had shorted bonds whenever the balance of news items was in the direction of a stronger economy, higher inflation, higher energy prices, a stronger Euro or a less accommodating Fed, and vice versa for the opposite news. For stocks, players also would have been correct about 60% of the time if they went long stocks when the balance of news items was in the direction of stronger economy, lower inflation, higher energy prices, a stronger Euro or a more accommodating Fed, and if they went short stocks when the balance of news items was in the opposite direction. There were several days when there was no news of the above variety or there were an equal number of news items on each side of the ledger. In those cases, abstaining from trading would have been a sensible decision.
The table below shows the results from applying the simple trade decision rules described above for direction and sizing. This approach had a success rate of 58% for stocks and 64% for bonds, resulting in an average 6.1% return on each trading day and a 2.4x growth of the bankroll.
As a further test of our hypothesis that this Crystal Ball would bear fruit for players with more experience in connecting news to markets and in sensible trade-sizing, we had five senior bank and hedge fund traders play this game.
