There certainly are offensive ways of stereotyping people or appropriating cultures, regardless of whether or not one intends to be offensive. Blackface is one, as is describing Jewish people as greedy, or Scots as miserly, or “immigrants” as lazy, and so forth.
Other stereotypes cause less offense, for example regarding lawyers as predatory or accountants as boring. Clearly, there’s are difference between these sets of examples, in that ethnic slurs – particularly those directed at marginalised groups – are likely to hurt more.