10/28/22

Even with all of our wisdom, creativity, dedication, computing power, and trillions of dollars at stake, it is still (thus far) proving impossible to accurately or consistently predict the future price movements of public equities. What gives? In conversation the other day, I mentioned that the reason for our shortcomings is that the underlying dynamics that drive changes in prices are incomprehensibly complex and probably fundamentally chaotic. The implication was that no matter how much creativity and computing power we throw at the problem, we probably won't ever get much closer to a solution. Those statement can appear dismissive and defeatist, so I want to take some time to explain what I mean and explore the implications.

Let's build up a model for forecasting stock prices. To start, we can try breaking down a stock's movement into component factors that might be easier to forecast; so rather than going directly at forecasting stock prices, we'll try forecasting things like dividend yield, earnings growth, valuation ratios, and currency fluctuations (for foreign companies) instead. These will all have an effect on stock price, but individually we may find that these characteristics have more manageable distributions and more predictable cyclicality than stock prices. Each of these components can be represented by a pendulum. For this simulation, the pendulums have massless, rigid rods connecting masses on one end to pivot points on the other. There is no friction or air resistance, so the pendulums will swing back and forth forever. We'll track the position of the mass first by tracing out its path, then by plotting its vertical position over to the side (dashed line for clarity):

(You may need to wait a couple seconds for GIFs to load)

The animation above represents one of our components of stock prices. Let's simulate some more components:

We have pendulums of shorter and longer length, representing the magnitude of the values our modeled components can take on. Tuning the masses gives control over the period- length of time before a cycle repeats. For the sake of this illustration, let's assume that the individual components (dividend yield, earnings growth, P/E, etc.) can be perfectly modeled by their pendulum.

This initially seems like a huge success- if we could perfectly model every individual component that makes up a stock price, surely we could accurately forecast the stock price. That would be a true statement if every component we came up with was completely independent of all other components. Annoyingly, this is not the case in the real world. Currency fluctuations might affect earnings, which might in turn affect components like dividends and valuation ratios (which might wrap back around to affect borrowing costs and earnings again). None of the component parts that we've hypothesized make up the stock price exist on an island- changes in each will affect all the other components in likely unpredictable ways (both in magnitude and direction). We'll switch up our pendulum simulation to show these complex interactions. Rather than each pendulum swinging independent from one another, we attach the pivot point of one pendulum to the mass of another, forming a chain. First with just two components:

Above is the classic "double-pendulum" illustration of chaotic motion. The paths to the left that were before drawing smooth and predictable curves, are now warped. Predicting the next data point is more difficult- the two pendulums sometimes align forces and reinforce the direction of motion, driving the system faster and higher, and sometimes they counteract each other and mute the response. Let's add a third component:

Stranger... more complex. We add our final components:

Though we haven't changed anything about the pendulums' properties or the physics of the simulation- the component parts are still simple and understood- the resulting system becomes incredibly complex. The path traced out is wild, abruptly snapping from one behavior to the next. Apparent patterns appear and vanish before they can be understood. The simulation is mesmerizing, and I urge you to try out the code for yourself and observe the system for longer than the limitations of GIF quality and website hosting I cheaped-out on can allow (link below).

I don't think it's much of a coincidence that the left chart (position over time) begins to resemble something like a stock's price chart. If stock prices really behave like our simulation - the sum of complex interactions between component parts, like we hypothesized - then we have some fundamental problems and limitations to consider:

The number of component parts -

We showed that the jump from 2 to 3 to 5 linked pendulums resulted in wildly different traced paths. Adding or removing the incremental component to the system doesn't increase or decrease the accuracy of the model, it completely changes its nature. If the phenomenon we're trying to predict is truly a 10-pendulum system, for example, observing a 9-pendulum model gives us no more information than an 8 or a 3 or a 20-pendulum model. We have no way of discerning how close or far our model is from reality. If you miss just one underlying component of the system, you don't just miss the field goal by a couple inches... you're playing an entirely different sport on a different planet.

The nature of component parts -

We started the simulation by assuming that not only do we know how many pendulums to use, we know their exact properties. We know with certainty the rod length, mass, and laws of physics that govern their motions. That's a pretty bold assumption, and it's one that's necessary for accurate modeling. Try covering the right (pendulum) side of the above graphic with your hand, only observing the scrolling graph on the left. Using only that graph, can you conclude anything about the properties of pendulums or laws of physics that generated it? Given only stock price data, can I state anything about the nature of component parts with certainty? I don't think so. And if I can't know the properties of the underlying components with certainty, I know nothing about the future of the system.

The measurement of component parts -

Here's the final, probably impassable filter. Let's pretend that we have absolutely perfect knowledge of the number and nature of components that interact to create a stock's price. We know the laws that govern the interactions and we can create a model that will perfectly predict the future given that we start with the correct inputs. How accurately can we measure the inputs?

The simulation that generated the swinging pendulums is completely deterministic, meaning that the same output will result from the same input every single time... in theory. However, the chaotic nature of the system we've created means that for extremely small changes in input values, we get wildly different results. Thus enters the problem of the accuracy of our measurements. A double-precision floating point number, which is what the simulation is using to calculate the angular acceleration and position of the components, can represent numbers greater than one accurately to about 16 decimals places. If I let the simulation run for 10,000 frames (around 2.5 minutes), and check in on the position of the last pendulum, I get the coordinates (554.337, 390.053). Then I tell the simulation that instead of calculating figures using all 16 decimal places, round the calculations to 10 decimal places. After 10,000 frames I now get the coordinates (689.426, 442.948). Same simulation again, 10 decimals places, results in coordinates (580.832, 488.936). Wildly different outputs with the same initial inputs, but the accuracy of intermediate calculations was changed. The same experiment performed at 20,000 frames, exact same initial inputs, resulted in coordinates (513.859, 423.449), (394.667, 357.075), (625.419, 365.284).

How can we rely on a model, even when we know the components are perfectly understood and the interactions are deterministic, if the results are so chaotic that microscopic changes to inputs or calculation accuracy produce wildly different results? If our hypothetical perfect model for predicting stock prices relies on components like GDP or inflation or trade flows, how accurate are those numbers? To the dollar? To the nearest $1,000? Accurate to $0.0000000000000001? Our model will output unpredictably different results depending on that accuracy. In our messy real world, it just doesn't work. Chaotic systems, like the pendulums or maybe future stock prices, simply cannot be tamed. That's what keeps it fun.

Sources:

Equations of motion for the double pendulum, myPhysicsLab - https://www.myphysicslab.com/pendulum/double-pendulum-en.html

Equations of motion for n pendulums, Yesilyurt, Boran - "Equations of Motion Formulation of a Pendulum Containing N-point Masses" - https://arxiv.org/pdf/1910.12610.pdf

Further reading:

Recent article on economic prediction from Howard Marks, "The Illusion of Knowledge" - https://www.oaktreecapital.com/insights/memo/the-illusion-of-knowledge

Mandelbrot, Benoit & Hudson, Richard - "The (Mis)Behavior of Markets: A Fractal View of Risk, Ruin, and Reward"

Gleik, James - "Chaos: The Amazing Science of the Unpredictable"

Waldrop, M. Mitchell - "Complexity: The Emerging Science at the Edge of Order and Chaos"

Code:

Run it yourself at p5.js! - https://editor.p5js.org/loganprob/sketches/xUT0XqZED

(I can't promise it won't eventually blow up)