The reason Whites stayed home is cause they believe the country is finished, they're economically despondent, and it no longer matters to them who's elected as with our globalized economy their skills are no longer market relevant and at their age they're unemployable.
They've quit. It's why 8 million left the workforce. My bet is that's the people who didn't vote. Romney ran an excellent campaign. It's the citizens, not him.
Remember, Obama got 7 million fewer votes, himself.