Besides Rudy's extraordinary bi-partisian popularity, he would give Hillary supporters a big reason to vote for McCain. I don't agree with his pro-choice position, but McCain is staunchly pro-life and Rudy's position would bring a ton of Democrats to the dance.
I know it's not conventional wisdom these days -- but John McCain could do a whole lot worse than picking Rudy.