My 401(k) allows me to borrow up to half for a first-time home purchase. I would receive 5% interest when I pay back the loan to my account.
So check what the interest rate is on the loan. The only drawback is that the amount you take out will reduce your overall 401 (k) balance, lessening the effect of compound interest toward your future retirement goals. Probably not an issue as long as returns are as abysmal as right now!