[quote=Arraya]The big question is why did academics fail at forecasting this? THAT is the question and how that applies to society overall?[/quote]
Probably because the immediate incentive structure did not benefit more than small actors and constituencies.
That is a recurring problem with treating Adam Smith and Milton Friedman as if they were Muhammed.
The explanatory utility of small actors working as an invisible hand gets wildly goofy when applied to big companies.
You have lots of little actors with incentives that benefit them instead of their company or that benefit their division instead of the firm or that benefit their company’s p/l this year but alienate clients moving forward.
For example, WaMu made lots of loans when the lending side needed better market share. They took resources away from other priorities.
When the loans started to go bad (inevitably) they did not have a functioning loss mitigation department (it had been folded into collections).
So when a distressed homeowner called them, they had aggressive credit card collectors bully the borrower.
This was a failure of overall strategy due its sacrifice at the alter of tactics.
A more obvious example is how Sony who had the largest catalog of songs and the best portable music player failed utterly to capitalize on digital music.
However, Apple, who had just come back from being within 90 days of bankruptcy ate their fucking lunch. Unbelievable.
The answer is that Sony set up divisions to compete with each other.
So the music division hated the electronics and vice versa and the tech side couldn’t get anyone to talk to them.
Each small actor there (the division bosses) lived in fear of being upstaged by or having talent poached by, the other division bosses.
Ever wonder why there are like 3 distinct Sony companies that make video games?
The bottom line is that academics assume that private actors know how to operate in their own best interest.
And that is often not true.
If those giant actors (banks or conglomerates) have strategies and incentive structures that are too focused on small constituent actors (like short term options and bonuses based on short term localized success), then it is often to the detriment of the overall mission of the company or big actor.