Saying “no” or “I’m not interested” is enough of a response.
Explanations are scarcely required nor owed. 

Sometimes a person will ask, “Why not?”
We get to decide how much we’ll engage.
After offering a personal truth, the other might try to persuade or argue.

Most people do not enjoy rejection.
Understand that, only if taken personally, hearing a simple “no” or “I’m not interested” grows into that feeling of rejection.
Then a person might feel tempted to compromise, subconsciously hoping to avoid it with a promise of change or “Hey, you’ve got me all wrong.” 
If we compromise ourselves in response, then those feelings will continue to grow.
It’s not always easy to see when immersed.
Perhaps from a desire to be forgiving; in the spirit of second chances; or wanting to believe a fantasy is real.

It begins with a fear of rejection when instead of saying no, we save face by telling that person, “Yes, but I can’t because x, y, and z.”

If interested or curious, although with looming obligations, then yes, say no, but offer an alternative.

Saying yes, but really no, without an alternative, is just a no. 

“No” is a refusal to engage.

And if someone has offered an alternative, recognize yes – that’s the desire to engage.