That is a big paradox of the modern age. We humans (specifically, residents of developed countries) are way too productive. We don’t need many people to provide for all our needs. 3% of the adult population is enough to feed everyone, additional 3% is enough to clothe everyone, to build furniture, electronics… What’s the largest capital expense in for most people, after the house? Probably the automobile. Major auto manufacturers such as Toyota and GM produce an average of 30-40 vehicles per employee per year. (That includes salespeople and paper pushers as well as assembly line workers.) If an average car serves 10 years before hitting the junk yard, that means we humans can satisfy world demand for cars by employing 0.2-0.3% of adult population in auto manufacturing.

So what this all comes down to is that we don’t really need many people to do “REAL work”. That was true long before “real workers” had their jobs shipped overseas.

Of course, this situation is self-reinforcing. Increases in productivity only appear when there are enough “idle”, preferably college-educated people around.