A couple of problems with that approach:
- Everybody wants to throw the bums out, except for the bum from "my" district, he's doing just fine.
- Everybody is discouraged by the lack of choices on the General ballot, but can't be bothered to show up for the Primary. They might not even realize there is a Primary because the local news sure won't say much about it.
It's even worse with local elections. They have the districts so gerrymandered at this point, it's not even worth showing up at the Primary unless you happen to be a member of the party who is bothering to run a candidate at all. There were about 50 races on the ballot last time I voted, of the non-Federal races, 90+% of them had only one candidate listed.