I need your help. I dont know what to do.

Two months ago I plucked up the courage to accept the fact that I was not happy in my current relationship which I had been in for 6 years. I knew it had to be over because I felt attracted to someone else. Once I had broken up with my partner, I then instantly was drawn to the this other person, it started with text messages and then grew from that to meeting up and talking on the phone for hours and hours on end (we had so much in common and are sooo attracted to eachother).

This other person is married and is 14 years older than me and has left his wife (of 20 + years) and set us both up in a flat. We are living in secret and we cannot be seen anywhere together.

The other day he had a sudden change of heart and burst into tears and has not stopped, he has since met up with his wife and said he still doesn't feel anything for her but is still confused because he says he loves me but doesnt want to give up 20+ years of marriage.

I dont know what to think or what to do? I have told him I think I should leave and he gets upset at the mention of it, I am not sure if he is crying because he realises that I will be alone and he will feel guilty for leaving or whether he doesnt want me to leave?