I agree with trueblue.
I agree with true blue about communication as well - Communication is great but it doesn't make people who you want them to be.
You have a lot of very important nonverbal communication from him that you're ignoring. He's showing you very plainly that he doesn't feel passionately for you. He doesn't really want to hug or kiss or initiate sex with you. If he wanted to, he would. You say he feels terrible, and that's true, but what he really feels terrible about is the fact that he doesn't WANT what you want.
It's time to end the relationship. You're long past the end of the "honeymoon phase", but things should just not be this bad at this point. And I know you wouldn't say they are "bad" because they are good enough to stay, but there is too much unhappiness.