Bingo! If I remember correctly, after the 2000 election and all the hubbub in Florida, someone checked out the felons who illegally voted in FL that year and they voted something like 85% for Algore. I was actually surprised that the number wasn't higher.
My point is that Zero's voters are far more likely to be crooks than people who voted for Zero's opponent, just as you say. It isn't just an insult; it's a matter of fact.