Two years ago this month, I ended it with xAP.
I'm thinking that you're a bit too fixated on a man who has no time for you. The only reason I am with my AP is because he gives me attention, affection and sex, otherwise why would I want him in my life ? What is this man giving you ? He's just someone you can talk to ?
Try not to concentrate on whether you matter to him. You matter when he needs someone to talk to, apparently when HE needs it only. And when it's like that it's better to turn your attention somewhere else.
Whenever I felt insecure with my AP, I would turn to my friends. I have great friends who listen and don't judge and I would feel that I "need" him less. I would think that if your M is not going well and you need someone to talk to, turn to your friends or family. Or is there anyway you could meet new people (not possible APs), for example by participating in meetup activities or something like that ? Having a more interesting life and building self-esteem would distract you from a man who's not giving you much anyway.
If I were you I do think I'd let him come to me. He is spending litle time communicating with you, yet knows you will be patient and continue to wait for him. And all the while, you are wishing it was more. I agree with sireanita, you have to keep busy and try to focus less on him. Its gonna be hard, but right now you and him are stuck in this situation you aren't really liking. There is this good post on EAS is the healing library, the zen of doing nothing. Its a good read and hopefully it will help you fight the urge to reach out to him yet again.