That makes total sense. The guy wanting to buy needs a loan because he wants to make a house purchase, which most people can’t do without a mortgage.
The high net-worth individual who owns a house outright wants a loan to do what? Gamble the money? Throw into the stock market? What exactly is that guy going to do with that money, if he already is well-to-do? It seems riskier to me.