Affair with controlling man

I am unhappily married for two years to an emotionally abusive man been having an affair for a year with a married man I used to work with and lately he's been real controlling saying he wants me to obey him and he wants to own me, he has kids and married doesn't seem to be leaving in the future I'm thinking about getting a divorce soon and he says when I'm alone things will be different between me and him but just wondering is he Using me since we don't see much of each other, we normally just sext and send pictures and one possible meet up but never in public .

I'm trying to get them out of my head but I am in love with them even though he says he loves me, I'm not sure what his real feelings are. Feel Like a fool that I even started this. I've blocked him from my phone but I know that he's wondering what happened and I wonder if I should tell him or just not even talk to him anymore ?