Yes, I agree I am not a strict believer in following the blood work vs. the subjective symptoms but my quick opinion was based on your numbers which are very good. But the HCG monotherapy is still not a natural pattern of hormone production, is it? thus it is possible that the pattern of T output is less than ideal? It can be an issue with TRT (roller-coaster) as well. Since you are already diagnosed secondary hypo perhaps you could try and see how you do. I certainly don't see much risk in it (unless you have another issue with your health that is being overlooked).
Another problem, hard to diagnose, is stress related CNS exhaustion (more vulnerable as we age - similar to chronic over-training), which too much T can aggravate. I know you said you looked into adrenal function, I'm not sure it always shows up there - I would look at resting heart rate, grip strength, and HRV as indicators. Certainly sex drive is also affected negatively.
As young men we seem able to adapt to high volumes of exercise and or hormones with mostly good results but in mid-life and older too much can tip us over the edge much easier than when we were young.