Flu, I recall reading that there may be minimum seasoning requirements for IRA accounts (required by law? as a matter of course by custodians?) that you must meet before withdrawing from an account without penalty. Therefore you may need somewhere between 21 and 35 such accounts (and per Diego, you’d then only be able to borrow 1/21 to 1/35 of your IRA savings).
Also, for those non-self-employed 401k borrowing scenarios, separation from your employer usually requires you to repay the loan (within 60 days?) or it is considered a distribution. I’ve usually joked that loss of one’s job could qualify one for a hardship withdrawal, but that would only waive the penalty.
Other than that hitch, I’ve been sorely tempted to borrow from my 401k. I look at it as investing in a fixed rate instrument with very low risk of default: me. In some market climates, that could be quite a good thing.