Used and Abused?

Is there really a difference between the words? Not when they are applied to people. To use someone is wrong, and to abuse someone is to be used wrongly, so what’s the difference?