It is unacceptable to talk about tiny, not significantly privileged minorities “invading” spaces or “stealing resources,” ever. It is unacceptable when fascists do it. It is unacceptable when nativists do it. It is unacceptable when terfs do it. It is unacceptable when anyone does it.
It does not matter how righteous your cause is or how great the threat seems in your estimation. That way of thinking is poison. To paint an entire minority class of people as an all-powerful, invading enemy is always unacceptable. You stand with the worst villains of history when you do it.