I was “gifted”, but to me 4 sounds late to start teaching basics of counting, numbers and addition/subtraction. Addition and subtraction are very concrete subjects that can be taught with marbles, raisins, etc.
The “University of Google” taught me that before age 3 children are pretty limited, understanding only the concepts of “one” “two” and everything greater may as well be 100. But right around 36 months for most children something changes, and counting can be done. So really I’d say “the sooner the better”.