It's called the slippery slope fallacy.
Homosexuals will find your comment as confusing as you would if they say that if we write the law to prohibit one group from marriage, then what is to prevent further prohibitions.
We know that the KKK will, and probably already is asking for racial distinctions for marriage. But what if anyone gets it into their head that appropriate nationality should be a qualifying point, or perhaps wealth?
The fear of more than two participants being involved in marriage is made more illogical by the fact that nations which advocate marriages of more than two people are among those with the harshest penalties for homosexuality, up to and in some cases, including death.