I'm wondering if you are susceptible to the placebo effect. 40% of any clinical trial is influenced by the placebo effect, and I have to think that men, especially those who workout, watch their diet, and take supplements are even more susceptible. I'm also wondering if you are a troll. The manner in which your doctor is proceeding, combined with your journaling, can lead to over thinking and overreacting to subtle changes in your body.
You also should be working out. Too many guys go on TRT and wait for some magic event. Not working out, and working out hard, is a dumb move and a waste of the opportunity for health and strength that TRT provides. Are you working out? It would be better to be hyper focused on the positive results of exercise than any negative side effects that might come up.
Post your pre and post numbers. This anecdote doesn't give us any information. It's easy to blame TRT for any issues you are having, especially when just starting. For example, I got dehydrated during a workout the first week I was on, and I thought I was having a reaction of some sort. My son, who is on an HGH protocol went through a difficult time and believed his numbers were off. A blood test revealed it was psychological. Also, many guys overthink the supposed sexual side effects. They can be real, but I think too often guys are reacting with confirmation bias and getting the result that they think they are going to get. The more powerful the reputation of the medication, the more likely the placebo effect.
Get some bloodwork, contact your doctor, if you are new to TRT, he's not going to be surprised, and go from there.
And, if you are a troll, stir the pot somewhere else.