Erectile dysfunction is very frustrating in a relationship. I can see if you are already in a relationship and are in love with a man, then you just have to deal with the situation as best you can, but if you have just met this man, and obviously are not in love with him yet, then I wouldn't begin a relationship with him.
You're starting off in a relationship whereas he won't tell you the whole truth behind his problem, and will have a major problem to deal with going into this relationship. I know how frustrating it is. When I was with my ex, he had ED because of type 2 diabetes, and he broke his hand one day when he couldn't get hard. He was too embarassed to seek a doctor's advice, so we both had to suffer.
You only have one life on this planet and you have to look out for number 1. It's your duty to yourself. You are not a martyr, nor should you be.
Go with your gut girl!...and it sounds like you're going to.