Really depends on the ecosystem in question, not every ant does the same damage in every place.
Hawaii made such a hype about Wasmannia auropunctata because the species could have made farming literally impossible. They love to climb on trees but are really [censored] climbers, so every time you try to pick some fruit you get showered by thousands of tiny stinging ants.
One of the species i haven't seen mentioned here yet is Anoplolepis gracilipes (yellow crazy ants), they are on the top 100 worst invasive organisms list, they wipe out entire island ecosystems particularly in the pacific, where they not just eliminiate breeding bird colonies in their entirety but are also doing a really job making a lot of endemic animals like christmas island's famous red crabs go extinct.