But surely everyone is a manipulator in one way or another!

Possibly, but manipulation in its worst form is carried out by people who can always see what others should be doing but are incapable of seeing what can be done by themselves; it is also carried out by self-centred people. Sometimes, of course, people, particularly children, have to be manipulated for their own good. It is the motivation behind the manipulation that is important and the extent to which it infringes on the rights and freewill of the manipulated.