I also drink soy milk instead of cow's milk. I've lost a few pounds and inches doing so. I feel much better. I never was fond of cow's milk.
My son was on soy formula due to a milk allergy as a baby. That was back in 1997. Now they try babies on a formula without the milk protein instead of soy due to the hormone factor. My littlest one was on a formula for milk protein allergies for a couple of months due to prematurity. Unless you consume a bunch of soy, the hormones should not be a factor, man or woman. We all have female hormones, like it or not fellows.