I think you are probably worrying prematurely. It sounds like you are a young healthy couple and by what your doctor says you have nothing to worry about ;)
Thanks so much for the response.
I think its natural worry. I have 3 kids and I still worry about my fertility. I think ... what if I cant get pregnant? and we just assume that when we are ready to have children that it will happen and happen right away... so I dont think your fears are unfounded, especially if having a family is very important to you. I know its hard but try not to worry too much about. Jump the hurdles as they come ;)