“Diversification is the only free lunch in finance” is a cliché which is easy to dismiss but one which captures an important truth: in a world where it is difficult to predict returns, lowering volatility without reducing return is extremely valuable.
The distinction between diversification (where all ideas are expected to make a positive return) and hedging (where ideas are deliberately offsetting both in risk and return) is extremely important. Here we are referring to genuine diversification: good investment ideas that have low correlation with each other.
Investors can sometimes be dismissive of this – they worry that diversification lowers volatility levels and imply through a naïve information ratio type analysis that this must lead to lower return potential. At its heart this is based on the false premise that market returns are truly independent through time (I.I.D. as it is referred to in statistics).
We know that in the real world returns trend and so investment ideas can have a negative correlation, but both return positively. An obvious real-world example of this can be seen in US Treasuries and the S&P500 over the past 20 years; they have a negative correlation over most time periods, but both have made substantial positive returns.