The most evil literary character ever created might by the demon-possessed Weston in C.S. Lewis’s “Perelandra.” It can be useful to remember the ‘banality of evil’ but Lewis points out its emptiness. Weston — who is truly Satan — is utterly empty.
Evil, of itself, Lewis shows to have absolutely nothing attractive about it. It can make itself so only by borrowing from that which is good, twisting it to its own purposes. Weston is filled with emptiness,* so to speak, a nihilistic hatred of all that exists. One could not create a truly human character like that and have it seem plausible. No one alive is quite that far gone.
But there is something of it in every villain. Every hero too, for that matter — let’s not get into that right now. This is where that banality stems from; there is no grandeur to evil, of itself. It may clothe itself in many ways, trick us into mistaking it for the good, but there is nothing underneath. It spoils what is, makes it mean, tawdry, ugly.
This is, I believe, what we must remember in creating our characters. If there is anything attractive about an ‘evil’ character, a villain, it comes from the good in him or her. This is why Weston is totally unattractive. I have written some fairly despicable sorts into my books but there is always at least a glimmer of some redeeming quality. Lord Radal (in the ‘Donzalo’ novels) might serve the dark gods but he loves his daughter, and many of his ill-considered actions stem from that. His henchman Sojel is about as bad as they get, a sadist, a murderer, a rapist, but he is loyal and has a certain pride in his ‘work.’
I suppose the ‘Wizard-Lord’ introduced in my upcoming fantasy novel ‘The Ways of Wizardry’ (out in Jan 2018) is about as far as one can go with an evil individual. I have made sure to point out that he is quite insane, so there is a little more leeway in dealing with his humanity (what is left of it). That is, incidentally, a bit of a trade hazard for wizards, who are sometimes overwhelmed by the vast infiniverse they are capable of glimpsing. Yeah, that’s a useful bit of world-building I’ve employed more than once in plot creation.
Obviously, I believe in good and evil. Not every antagonist is evil, of course, or no more than the rest of us. Some simply want something different from the protagonist. The antagonist can even be morally superior to our hero. But evil exists and so do evil individuals. Recognizing this and the nature of evil will aid us in creating our characters.
Finally, recognize that there is dynamic of good and evil. People change, succeeding at times, failing at others. Some repent, some don’t. Some are redeemed, others are lost. They remain human. Unlike Weston.
*I’ve used that phrase in one of my own books. Maybe more than one.