Tuesday, November 30, 2010

Are complex models the answer?

There seems to be consensus that conventional economic models did a poor job predicting the magnitude of the last crisis and that we need to do better. In today's Wall Street Journal, we see the beginnings of one response:
In short, a physicist, a psychoanalyst and an economist believe that they can build a bigger model that captures the complexities of the real world and does a better job of forecasting the future. Good luck with that! While I wish them well, my response is that this will go nowhere or worse, go somewhere bad.

To those who believe that complex models with more variables are the answer to uncertainty, my response is a paper by Ed Lorenz in 1972, entitled Predictability: Does the flap of a butterfly's wing in Brazil set off a tornado in Texas?, credited with creating an entire discipline: chaos theory. In the paper, Lorenz noted that very small changes in the initial conditions of a complex models created very large effects on the final forecasted values. Lorenz, a meteorologist, came to this recognition by accident. One day in 1961, Lorenz inputted a number into a weather prediction model; he entered 0.506 as the input instead of 0.506127, expecting little or no change in the output from the model. What he found instead was a dramatic shift in the output, giving rise to a Eureka moment and the butterfly effect. (One of my favorite books on the topic of Chaos is by James Gleick. It is an easy read and well worth the time.. for investors and economists)

Complex models work best with inputs that behave in thoroughly predictable ways: software and engineering models come to mind.  They break down when the inputs are noisy and the relationships are unstable: macro economic models are perfect lab experiments for chaos. The subjects (human beings) belong in strange and unpredictable ways, the variables that matter keep shifting and the relationships between them change over time. In fact, I will wager that the models that worked worst during the last crisis were the most complex models with dozens of inputs and cross relationships.

So, what is the solution? My experience in valuation suggests that you should go in the other direction. When faced with more uncertainty, strip the model down to only the basic inputs, minimize the complexity and build the simplest model you can. Take out all but the key variables and reduce detail. I use this principle when valuing companies. The more uncertainty I face,  the less detail I have in my valuation, recognizing that my capacity to forecast diminishes with uncertainty and that errors I make on these inputs will magnify as they percolate through the valuation. More good news: if I am going to screw up, at least I will do so with a lot less work!!


Unknown said...

What is the name of the book by James Gleick?

Anonymous said...

Thank you! One of the most wanted post I've seen for last year. The only thing I would like to add - from the educational view point: for people who have just been involved in analysis creating big and complex models can help to understand and "to feel on a fingertips" the limitations of their work. It gives somewhat birdseye picture that helps later not to tresspass the border of rationality and redundancy.

Aswath Damodaran said...

Chaos is the book by Gleick.

Anil Bains said...

Thanks for such a good post!!!I am sort of delighted to read your post as it will work as a testimony of my reasoning that we must reduce the complexity of the model when we are not aware of the details..
Recently this has happened in Manganese Ore India Ltd (MOIL's) IPO, where some of the information about the appropriations and restatements by the company were not available publicly, so I was not able to project it well and thought of simply leaving it instead of projecting it wrong...

Johnny said...

Can we use monte carlo to solve complex situation with uncertainty?

Is it worth to spend time on it?

As i believe, many ppl outside finance field (physicians, scientists and etc.) has used this method for their experiements that of course have many elements of uncertainties.

Aswath Damodaran said...

Leaving out a factor or expense is not an option, since you are making an assumption then: that the expense will be zero. Make your best estimate and move on.

Aswath Damodaran said...

I will put up a post on Monte Carlo simulations soon. I think they are a great device but it is garbage in, garbage out.

Anonymous said...

to Johnny:
You can try by using the software like CristallBall or Mathlab simulate simpliest models for NPV search. Result will surprise you - even when if you have a few inputs with pretty small variances, the variance of result mostly is high enough to make your analysis almost useless. When we speak about big models with lot of variable inputs and long time-horizon - the outcome is even worse. I'm far from saying that MC is useless itself, but it has limited practical application and needs cautious approach.

Anil Bains said...

I have prepared valuation models for a few Indian IPOs/FPOs (recently CIL,PowerGrid etc. to name a few) and have come across one basic problem in all these cases and that is:
The data about appropriations, restatements and adjustments are either not very transparent or is difficult to interpret/link at least for a common investor who has access only to public data such as DRHP and the annual reports of the company.
Shouldn't these companies or the regulators pay some attention to this transparency problem?
If this issue gets resolved, I think a common investor's knowledge would certainly go up by one notch.
There are a few other turn-offs as well that limits the horizon to understand the company/issue well.

Johnny said...

the variance shows the upper and lower bound which set the range and confidence level for your analysis.

Why you said that variance of result mostly is high enough to make your analysis almost useless?

Anonymous said...

to Johnny:
Not sure I can understand your question correctly. The resulting "range" is much "wider" then any of your inputs to model, therefore the estimate - more blurred.

Unknown said...

Do any of you need a 127mb-large model for valuing credit default swaps on a pool of commercial properties in California?

Moody's ex-employee

zaerro said...

To Milos
It would be very interesting to take a look at this monster unless it prohibited by NDA or Moody's policies.

Unknown said...

I was being sarcastic...

Accounting Tutor said...

Simplicity is key when confusion reigns.

I first thought the $6 Billion price tag for Groupon was crazy. Here is a simple back of the envelope calculation.


Your thoughts would be great.

Manish Poddar said...

Hi Sir,

I know this question is not with this article being discussed, but I have a question
Why is Effecient Frontier curved??

Aswath Damodaran said...

The efficient frontier is curved, if all your assets are risky because expected returns will not change linearly with standard deviations. When you introduce a riskfree asset, the efficient frontier is linear. The capital asset pricing model is based on that efficient frontier.

Accounting Tutor said...

You probably read about Dells offer to buy Compellent Technologies - http://dealbook.nytimes.com/2010/12/09/dell-in-talks-to-buy-data-storage-company/?nl=todaysheadlines&emc=a26

Why is Dell offering a price lower than the market price of Compellent Technology to buy it?

doctorgeeta said...

As a layman, the only reason I can think of for someone to give a complex model are:

- to show how smart they are

- to demand a hefty packet

- to obsfuscate/befool

- to confuse

and finally, if all these fail to convince why things didn't work the way they were expected to work as per the model, to take recourse to any of these factors to explain.

simple things are the easiest to understand.

But then, who will pay the doctor couple of hundred bucks for telling the patient to simply gargle with warm water instead of all those fancy cough syrups for the cold?

Hemanth Manda said...

The richest firm today is worth $380 billion. So even if Facebook were to surprass that number in the next 10 yrs, it amounts to an annual return of around 20% - Certainly not worth the hoopla. If I were sane enough to invest in Apple or Amazon during the downturn, I would have made this return ( ofcourse the risk of investing in Facebook is a lot higher and I am betting that its not the one for me - Its a different matter that I said the same during Google's IPO ).