By Krisztina Than and Gergely Szakacs
BUDAPEST (Reuters) - Almost all Hungarians who voted in Sunday's referendum rejected the European Union's migrant quotas but turnout was too low to make the poll valid, frustrating Prime Minister Viktor Orban's hopes of a clear victory with which to challenge Brussels.
Hungary's maverick right-wing leader, whose hardline policy on migration has been criticized by human rights groups but is popular at home, nevertheless said EU policymakers should heed the "outstanding" referendum outcome.
Orban said more Hungarians had rejected the migrant quotas than had voted for European Union membership in a referendum ahead of Hungary's 2004 accession to the bloc. Some 3.249 million votes were cast rejecting the quotas, compared with 2003's 3.056 million votes in favor of joining the EU.
"Thirteen years after a large majority of Hungarians voted at a referendum to join the European Union, today Hungarians made their voices heard again in a European issue," Orban said.
"We have achieved an outstanding result, because we have surpassed the outcome of the accession referendum," he told a news conference at which he did not take questions.
The National Election Office said on its website that 98.3 percent of those who voted had rejected the quotas with 99.97 percent of votes counted. Just 40 percent of around 8.26 million eligible people had cast a valid vote, however, less than the 50 percent needed to legitimize the result. Final results are expected next week.
Along with other ex-Communist countries in Eastern Europe, Hungary opposes a policy that would require all EU countries to take in some of the hundreds of thousands of people seeking asylum in the bloc after arriving last year.
Orban, who responded to the influx by sealing Hungary's southern borders with a razor-wire fence and thousands of army and police, says deciding whether to accept migrants is a matter of national sovereignty. He says Hungary -- with its Christian roots -- does not want to take in Muslims in large numbers as they pose a security risk.
In power since 2010 and with his Fidesz party still firmly ahead in opinion polls, Orban will use the referendum to keep the issue of migration on the political agenda in the run-up to 2018 elections.
Some opposition parties seized on the fact that turnout had fallen short of the threshold needed to validate the vote, with radical nationalist Jobbik calling the referendum "a fiasco" and calling on Orban to quit. Leftist opposition party DK also said Orban should step down.
Early on Sunday, after he cast his vote in Budapest, Orban had played down the importance of voter turnout, saying what mattered was that "No" votes rejecting the quotas should exceed the number of "Yes" votes.
Later, he said he would submit an amendment to Hungary's constitution to put the result of the plebiscite into law.
But the invalid vote could blunt Orban's efforts to force Brussels to change its migration policies, some analysts said.
"Obviously the government tries to sell this as a success, but this is not success: it shows Fidesz could not mobilize more voters than its own voter base plus the Jobbik voters," said Republikon Institute analyst Csaba Toth.
Toth said the government would keep the issue of migration on the agenda both at home and abroad as "the more the political discourse was about migrants, the better for Fidesz".
Orban has been at the forefront of opposition in Europe to the position taken by German Chancellor Angela Merkel, who has said EU countries have an obligation to take in refugees. But his go-it-alone approach to migration is not welcomed by all.
Italian Foreign Minister Paolo Gentiloni was among the first to react to the referendum, tweeting: "Nice to see a quorum was not reached."
Earlier Gentiloni told RAI TV: "The Orban government is sending out the wrong message with this referendum. We must realize that immigration is a permanent phenomenon. You can't leave this to single countries to sort out, it has to be confronted by all of Europe."
While Budapest says immigration policy should be a domestic matter, human rights groups have criticized the government for stoking fears and xenophobia with an aggressive campaign ahead of the referendum, and for mistreating refugees on the border.
Last year, hundreds of thousands of migrants fleeing war and poverty in the Middle East and beyond crossed Hungary on their way to richer countries in Western Europe. This year Hungary has recorded around 18,000 illegal border crossings.
Erzsebet Virag, voting on Sunday near Budapest's eastern railway station, where a year ago thousands of migrants camped outside waiting to get on trains toward Vienna, said:
"I voted (No) because there are a lot of poor people in our country too and if more poor people come in we will be even poorer and have to work even more."
(Reporting by Krisztina Than; Additional reporting by Marton Dunai and Crispian Balmer; Editing by Catherine Evans)