You can also use some of your own subtle energy to counter the effects of an energy vampire. Visualize a protective white light around you: an energy shield. You can still hear the person who's yelling at you or blaming you or pushing herself into your sphere, but she won't cut into you so viscerally anymore. You've created a buffer zone, where her negative influences can dissipate.
Setting boundaries is another way of protecting yourself; you draw a line saying, for instance, "This is what I can do for you, and this is what I can't." You don't have to convince the vampire of the rightness of your stance. Getting defensive simply adds to the negative charge of the encounter. You want to remain neutral. When someone starts pushing your buttons, and you start sizzling inside, you've got to make the decision not to react.
I also suggest you step back and think about what type of people aggravate you, because I believe that one law of energy is that we attract what we haven't yet worked out in ourselves. If I'm a very angry person, I'll find myself surrounded by angry people. By paying attention to the people who seem so draining, you might discover something you need to address. It has been my experience that once you've worked through a particular issue, you're no longer worn out by that kind of energy vampire. And the vampires, robbed of a source, move on to more easily drainable audiences.