Here are the basic facts. In 2011, 562,000 people left California, and 468,000 came, according to the Census’s American Community Survey. That means 120 people moved out of California for every 100 people who moved in. Out-migration reached its peak in 2005, when 160 people moved out of California for every 100 people who moved in. The California exodus rose with the housing bubble and subsided in the recession.
Who leads the charge out of California? Even though California’s richer residents face high tax rates, lower-income households are more likely to leave. From 2005 to 2011, California lost 158 people with household incomes under $20,000 for every 100 who arrived, and 165 for every 100 people with household incomes between $20,000 and $40,000. In contrast, just slightly more people with household incomes in the $100,000-$200,000 range left than came to California (103 out per 100 in), and California actually gained a hair more people in the $200,000+ range than it lost (99 out per 100 in). The rich aren’t leaving California, but the poor and the middle class are.
(emphasis added)
There’s some interesting graphs and charts, so that link is worth a look. The conclusion, at least based on statistical evidence is that people move mostly because of high real estate prices, not jobs and not taxes.