Trust me I understand the desire to stay in denial. But here's the hard cold facts.
You are not unique. Your story has been told at least 60-100 times in the past two years. My own story is reminiscent of yours - He was my first love, we traveled all the time to be together, everyone knew - our spouses, our kids, our families. We worked in the same field and even to this past summer he has tried to get me back working with him. He still to this day has my favorite song as his ringtone. You are not unique.
This great love story - he didn't choose you. He chose to stay in his marriage REGARDLESS of the reason he didn't choose you. Regardless of the fact that he at one point asked for a divorce HE DIDN'T CHOOSE YOU. By your own admission in your first post you were unsure of leaving your own marriage for him - trust me this is no great love story. Neither of you chose to commit to the other fully. And now he's ended it.
About a year after my A and M ended (by my choice), I realized that there was no way to ever get back to just friends - that's the truly messy part of all this - you will try. You will blame yourself - you will blame him, you will keep a stiff upper lip because his situation is so hard...you will cry and hold on to the "love".. but let's take a look at you. WHY do you want to be a part of this friendship? Is he the only possible provider of feeling beautiful and sexy? Is he the only one on the planet that understands you? NO! It doesn't mean it doesn't hurt, that what he tells you doesn't impact you - but again, HE DIDN't CHOOSE to keep you in his life. He will take full advantage of the fact that you stick around tho - he will pour on the "at least we can be friends", he will use this - and you will use him until one of you stops being willing to take crumbs..
As for people at work - who cares what they think? Trust me that you are not so important that they'll all run gaggling and ogling the fact that you are not friends - WHO CARES. We all had some kind of fallout whether at work or in our social circles or whereever - we ALL thought it would be more awful - but it wasn't.
You sound afraid to be out of this - and it is scary. It is hard to hold your head up and not let those old emotions of love, lust, friendship - whatever - rule you. It is hard to do things that are new in an environment that is comfortable. Moving on means doing hard things to make a difference for yourself. Not for him. Low Contact can help but you have to embrace the mentality that this is all really over. Play out the scenarios - stay friends.....keep hoping he'll come back to a non-commited life with you (since you aren't sure if you could leave your family to be with him either). To what end will this become a fairy tale come true? You will always be the woman he had an A with.
I'm sorry if this all comes off harsh, but there is danger in denial. Denial leads us to think we can still be "his special someone", that you'll have a 'someday" and you miss living out all the normal days with your spouse and kids (do you have kids? - I know he does) waiting for the elusive happiness that lives just outside your reach. Denial takes away real love and replaces it with many other destructive feelings. Don't accept denial.