I believe most atheists couldn't care less about monuments and memorials. The sight of a cross on a group's logo does not make them quake with rage. It's historical. It's part of the culture. They see such things as simply a part of society which doesn't necessarily apply to them.
Then you have the militant atheists. These fanatics are no different than any other extreme group. They are fixated on a single goal; removing the tiniest mention of religion, esp. Christianity, from even the most loosely affiliated government activity. Why would someone who doesn't believe in something be so worried that other people might? It isn't as if Christians are assaulting atheists in the streets and making their lives a living hell.