There’s a specific group of older men that can present with what appears to be normal T status in the morning hours, however mid to late day the pituitary becomes dysfunctional and T is anything but normal.
Feeling tired mid afternoon nine times out of ten is testosterone or thyroid related. The latter can be ruled out.
You have all the classic symptoms of low-T.
What is your SHBG?