"He would not have been my pastor," Clinton said. "You don't choose your family, but you choose what church you want to attend."