Note: I'm not a doctor, information below is based on the fact that I've done a lot of supplementation for working out, and have done a good amount of research into those supplements. I would recommend following the advise of a doctor before anything I say, and fact check anything you read!
Caffeine is a vasoconstrictor. In general, nervous tension / anxiety has a side effect/symptom/cause of high blood pressure. Couple high blood pressure with a drug that makes your blood vessels more narrow, and you have even higher blood pressure. While under normal circumstances, caffeine might slightly elevate blood pressure, if you are already in a situation where it is high, it may cause more serious issues. In addition to this, caffeine does increase energy, makes you more jittery.. essentially increases tension! So long story short, removing caffeine may reduce symptoms of your tension, and may also net you positive health benefits from removing it.
As for how to get off caffeine. Personally, I do this whenever I start working out in the gym and treat caffeine consumption as a cycling supplement. This basically retains the affects of the drug without your system becoming too used to it. It isn't very easy, but typically I stop drinking coffee completely, and when I start to get a headache, I have a 'cheat' meal and a cup of coffee. Cheat meal meaning something reasonably high in fat. What I've found is when I get a headache, many times a heavy dose of fat / carbs / protein can correct it very quickly. What you will find when doing this is that you are providing an optimal condition to recover from the caffeine headache, but your body will quickly become used to not having caffeine so the headaches will go away in about a week. That week is no fun at all, but you have to step back and think.. it's only 7 days.
Lastly, your ask about safe levels of caffeine. This is basically impossible to determine based on the fact that 'safe levels' is entirely dependent on the individual. Unfortunately, we probably need to refer you back to your doctor on this. Ask questions of him! Why no caffeine? He may have a specific reason for it. I won't go so far as to make a recommendation other than follow your doctors advice.