Barney Frank is not backing down an inch from the comment he made on '365 Gay' over the weekend.