One of the first things Hitler did after he was "elected" king of Austria was to have a gun registration program. Shortly after completing that, the Nazis decided, after s supposed breakout of gun related deaths, was to confiscate all the guns. Of course, only law abiding citizens had registered their guns, so only law breakers retained fire arms. Also, by this time, there were no armed citizens to form a militia to prevent the Nazis from bringing the wrath of Germany upon the Jewish population of Austria.
You are right, the Jews were not armed, so unable to defend themselves. Hitler had already convinced a large portion of the ignorant population that Jews were the reason for their problems, so Jews were hated.