I didn’t see you mention a wife, OP. If you’re presently single and unattached, good for you, BUT…
I wouldn’t buy a house now assuming that three years from now you’ll meet the girl of your dreams and she’ll gladly move into the home you built for yourself, to your needs. She’s going to have her own ideas and desires.
If you’re a sworn bachelor for life by all means execute your plan.
I would definitely approach this more as an investment than “I’m gonna buy a house and never move again”. Since you’re prepared to put in sweat equity your plan is far from crazy. Just understand you may end up selling down the road.
