When it comes to picking stocks, why is it hard to beat the indexes like S&P 500 (USA) or BSE SENSEX 30 (India)?

Pratyush Malviya
5 min readSep 5, 2021

In empirical research of 15-year returns in the US, over 95% of fund managers failed to outperform the index. Even in India, over the past few years, actively managed funds have significantly underperformed the index.

Like anything about the markets, there are no definitive answers as it’s an emergent phenomenon. Still, the first clue comes from this long-range study that suggests that most of the index returns come from a minority of stocks. Just 4 percent of all US stocks are the reason behind long-term equity out-performance over risk-free deposits (FDs) as per this study. A similar trend holds true in the near term. For example, as a supporting anecdote, only 3 out of 50 stocks in Nifty 50 were responsible for over half of its total returns. Stocks follow a power law in terms of their returns: the majority of gains come from a minority of stocks. How does this impact building portfolio by picking stocks?

Power law: An example power-law graph that demonstrates ranking of popularity. To the right is the long tail, and to the left are the few that dominate (also known as the 80–20 rule).

Most people give equal weights to stocks that they pick, e.g. if they invest in 25 stocks, each gets 4% weightage. In comparison, indexes usually weight companies by their (free-floating) market capitalization. This seemingly small difference has a significant effect. If your best performing (minority) stocks have a 4% allocation, the total returns of your portfolio will be dominated by the majority of non-performing stocks, dragging down your portfolio returns. So, the trick is to give higher weightage to the best-performing stocks in your portfolio. This is what an index does automatically!

The stocks listed in an index get weights depending on their market capitalization, and market capitalization is a function of historical business performance and expectations of business performance. As either performance or expectations grow, an index automatically gives a higher weight to that particular stock. So in a way, index investing is like betting the winners and cutting out the losers (which is a classic investing maxim).

Another reason the index outperforms is that companies that have usually demonstrated growth for a long time grow big enough to become a part of those indexes. A company big enough to be part of the index usually has good fundamentals, which take time to change. This is in contrast to stock picking, where you have to guess and study management, market prospects, moats, etc. A random company included in the index is likely to have better prospects than a random company on the market. When you buy an index fund that tracks the entire stock market, you are trusting the judgment of millions of consumers and the magic of the market in allocating the right resources to the suitable needs.

From Warren Buffet

credit: fool.com

By investing in the index, you’re effectively outsourcing research to other investors who take bets on non-indexed companies. Most of them turn out to be lemons, the few ones that succeed enter the index, And once they enter the index, you automatically buy good businesses. Imagine indexes as a filter to provide you with companies that have managed to grow from 0 to really big, and you’re backing only those companies. Rather than trying to guess future winners by stock picking, you’re betting on past winners (which is less risky).

Photo by Karolina Grabowska from Pexels

These indexes are less risky than active stock picking is a relevant bit because most of us do not want excessive risk with our savings. It’s the reason behind pension schemes, and even individual investors to prefer large, established stocks to allocate the majority of their assets marked for equity, So the money keeps flowing into equity markets, and most of it goes to large-cap stocks (which make up the index). Especially after low interest rates across the world (including India), net new money in equity mutual funds isn’t stopping, and more money is chasing the same 50 stocks in Nifty 50.

So, imagine a situation where people aren’t getting enough interest in FDs but don’t want the excessive risk. Where do they invest? You guessed it right — large-cap, blue-chip stocks.

Money is literally chasing few stocks. This means when you trade a stock that makes up an index, the buying decision is already made because of new cash coming into the markets. Still, when you trade a stock, not in the index, you’re trading against a sophisticated investor who might be selling it to you because they know something you don’t. In contrast, a seller might be selling stock in an index fund because of liquidity or rebalancing needs.

Does it mean that indexes are a bubble in the making? Some people claim that, but it’s hard to say it’s a bubble because nobody knows what a bubble is? It’s (almost by definition) known later when prices drop.

Indexing is like a type of insurance. Instead of investing in particular companies with idiosyncratic risks, you pool money with other investors and invest in a universe of companies. It’s just like life insurance: the probability of death is low but fatal.

--

--