I am not a Dr but from my reading of studies and articles, the cancer link to T is not established. If anything it seems like having low T is worse for your health. Like others have said, watch your PSA and don't get your T levels too high.
.
For the ball shrinkage, I don't know how old you are and your kid plans, but my wife likes a tighter package. I don't know why anyone would think an older guy with a large saggy hanging scrotum is good - your junk be hanging out your shorts. I am past kids and don't mind that my package is tighter and smaller. Besides, from a perspective standpoint, a smaller scrotum might make my johnson look bigger.
.
It was always been pretty easy for me to not gain weight, but you might look at your T levels and take some T. I was able to drop weight much faster when I started T injections - I was also working out and eating right, but still easier with the T due to its affect on body and my motivation / energy level.
.
Finally, regarding your lady, she would be jazzed if you continue to lose weight and stop smoking, particularly if you say it is for her. Most women I know would probably care more about that than intercourse and many women like other types of sex just as much if not more than intercourse.
Good luck man. Stay active and eat healthy !!