The best advice I can offer is, if you are thinking seriously about TRT, is to consult with a doctor that specializes in TRT rather than your GP. (unless your GP happens to be a TRT expert - possible, but unlikely) I recommend Defy Medical (I have no affiliation with them) as they know what they are doing, their rates are very good and they are easy to work with.
Many if not most of us here started TRT with a doc that had no idea how to do TRT properly and then after months of frustration finally got a doc that specializes in TRT. Starting with the wrong doc causes many of the common issues with libido, E2, etc...
At your age, I would lean towards treatment (or at least giving it serious consideration) as you sound like you are already doing all of the correct the lifestyle things, so if all of those things are still leaving you with a total T as low as yours, treatment starts to sound like a good option. You could explore thyroid and cortisol issues first to rule them out as well. Get the tests on the cheap at www.discountedlabs.com