Weird issue with prospective relationships

Hello,

None of my prospective relationships move forward beyond a point and for the same reason each time and I simply cannot figure out why. I should add that I am only looking for insight, not moral judgment:

So, either at work or outside, I will meet a dude who is clearly interested in me for more than a platonic friendship. If I find him attractive, I respond in kind (usually it is not particularly clear who has shown interest first) and we try to move it forward. We may meet for lunch or in the gym or some other social activity to get to know each other. These are almost always something I have to initiate - maybe everytime. At best, the other guy might just plant the seed of an idea but will not take any further steps. He may broadly hint at lunch or show interest when I talk about going to the gym but will not suggest that we do the activity together. Then we hang out together for some activity on my initiative. I usually do something like appear to be looking for lunch company at the last minute so that it appears completely accidental and not a pre planned action on my part. Make it appear casual.

Anyway we become friends and the mutual interest is obvious but nothing moves forward because I am always the one who has to initiate and I get tired of it. It is like I am playing with a doll!

Few things to add:

I am not very experienced in gay relationships. The last I had was many years ago in my country. I am an immigrant and non-white. I am in a majority white country. No, I am not suggesting racism on their part.

There is no language barrier at all. This is an English speaking country.

What is going on here? Why does the other individual not ever take any initiative but really, really wants me to do all the lifting? There have been situations where we were very close to making something happen but it did not because neither of us took any action.

This might sound nuts but if anyone has any insights, I would be grateful.

I can confirm that this person had originally admitted to being married and that a good few of the men he met, were also married.

I suspect that's the reason he said he wasn't looking for judgement.