Yeah, I would reject it, unless they are willing to pay A LOT for the rent back to make it worthwhile for you.
Also, I would recommend doing the final inspection/walk-through after they have moved out. This insures that they have properly moved out and didn’t damage/destroy anything during the move. In our case, we did our final inspection before the seller completely moved out, and they damaged the house during the move (and we didn’t find out until after close-of-escrow). It is now a pain to get them to repair the damages. After escrow closes, and the seller gets the money, buyers have no leverage left, short of threatening to take them to small-claims court.