Don't fear arimidex/anastrozole, at least not based on the experiences described above. Your earlier symptoms sound like estrogen crash, not anastrozole sensitivity.
Talk to your doctor about their overall treatment methodology and plan or find another practicioner who can do more than skim...