Lombard Equities News

The Hidden Cost of Diversification

Written by Arie van Gemeren, CFA | Feb 15, 2025 3:00:00 PM

You’ve been told your whole life to diversify. Spread your risk. Own a little bit of everything.

But the wealthiest people don’t build their fortunes that way.

They do the opposite: They bet big.

When I worked on Wall Street, we had a saying: You concentrate investments to build wealth and diversify to preserve it.

This fundamental observation was based on the experience of many advisors who have worked with countless incredibly wealthy families. Here’s the harsh truth: The “really wealthy” 50-year-old most likely didn’t build their $10 million+ fortune by diversifying into a broad cross-section of the S&P 500 and bonds.

Just for fun, I ran an analysis of a 25-year-old who started investing $100,000 per year into a diversified portfolio yielding 7% annualized returns, per year, until they are 50. The outcome for this person is $7.35 million in ultimate portfolio value on $2.6 million invested. The result is quite good - you won’t find me arguing that investing and compounding over time isn’t excellent. But the analysis is also somewhat unlikely. Here’s why.

  1. You must invest $100,000 annually, net of taxes, into your investment portfolio. At a minimum, if you’re a high earner, and let’s assume your tax rate is 35%, you need to be at least making $153,000 per year.

  2. This doesn’t count living expenses, which, of course, must also come out of the post-tax earnings. So, to allocate $100,000, your pre-tax earnings would have to be even higher.

  3. And lastly, the analysis assumes you are a man or woman of iron and don’t sell out at an inopportune time. Remember - long-term average returns are not the actual annual returns. When you hold your portfolio, you’re likely to experience dramatic up and devastating down years. But let’s assume you have the intestinal fortitude not to sell.

Here’s the real problem: Diversification is a great way to get rich... slowly. But it’s a terrible way to get rich fast.

It took this person 26 years to reach this outcome. If you’re 25, do you really want to wait until 50 to enjoy financial freedom? If you’re 50, do you have 26 years to build wealth? Probably not.

This is why you can’t start with diversification. You have to build wealth first—then protect it.

What if, instead of slow diversification, I invest $100K per year for just 6 years, earning 30% returns before pivoting to a diversification strategy?

The answer—below—is that you wind up with $6.4 million, but you only ever invested $600k, not $2.6 million. So, it was nearly the same outcome, but you invested a fraction of the money. Concentrate, then diversify. In this example, we assume that you never invest a single dollar again, nor do you rinse and repeat your high-return strategy that got you there. As you can imagine, the actual outcome of this person would be way in excess of the diversification example.

 

Anyway, let’s dive into an analysis of why concentration can pay off.

I’ll lead with a personal example. When we bought our first home, we weren’t just making an investment but betting everything. We emptied our savings, drained our investment accounts, and put everything into this house. It wasn’t just a place to live—it was our first big swing.

We bought it for $515,000. We got an 80% LTV loan and invested $103,000 to acquire that house. We invested another $40 - 50,000 into the property to improve it. Nine years later, we sold the home for $1.1 million. That is a CAGR, approximately, of about 22% over 9 years on our initial invested capital. However, the case study becomes more interesting because, during that initial investment, we took a cash-out refinance and used that money to acquire two new apartment buildings for an additional equity gain on those new investments of about $300,000. This means our CAGR on a 9-year investment of $140,000 to start is about 29.7%, or 9-year annualized returns at that figure.

The point of this story is not to illustrate my genius (which is nonexistent) but rather to show that, in a “relatively” simple example, it’s possible to achieve high returns with a consistent strategy and a high degree of concentration. Real estate does provide the opportunity to earn outsized returns in a more controllable format.

But don’t just take my word for it.

Some true investing greats are big believers in concentration and investing in what you know. Buffett doesn’t just believe in concentration—he warns against diversification. His words:

"Diversification is protection against ignorance."

Charlie Munger, who has always been my favorite of the Berkshire Hathaway duo, was even more direct:

"If you know what you’re doing, four or five investments are enough"

Both of these guys made a veritable fortune from concentration investing. The basic premise of their strategy was to buy what they knew and never to let it go. The issue with true diversification is that you are both hedging against risk (of one position blowing up) and diversifying away any potential for outsized returns.

Also, short of your position going to zero, the other issue with diversification is that it doesn’t provide that much protection in a downside scenario. Ask anybody about their diversified portfolio’s performance during the GFC. I was in college then, but my dad invested with a financial advisor, and I remember him telling me his portfolio was down 50%. Down 50%! In a diversified portfolio.

I think a better approach is to concentrate on building and diversifying to preserve. However, people get into trouble when they concentrate on building, never drop their positions, and keep concentrating forever. Holders of Bear Stearns or Lehman Brothers stock can attest that that strategy also didn’t work.

If you’re still in wealth-building mode, you don’t need less risk—you need more return. The biggest risk isn’t concentrating. It’s wasting time.

So ask yourself: Are you playing to win—or just playing not to lose?