Caffeine is a fiend ! Notorious for it's ability to lift you up higher...only in order to throw you down all the harder !
The bounce it gives to your blood sugar is a truly evil effect...which makes you crave sugary foods...which stresses your pancreas...which affects your adrenals and your gut....
Personally, I am on a diet that avoids sugar ( you can't eliminate it) and salt (ditto), gluten and dairy products (I have food intolerances - you may not) and additives in general. As you might guess from the latter, it's also an organic diet, high in vegetables and nuts, seeds, grains. I am also taking probiotics and Aloe Vera, to help my damaged gut (I have been diagnosed with CD), and Udo's Oil, which is an Omega 3,6,9 balanced oil designed to improve my Essential Fatty Acid status.
I usually find that eating well is the key to avoiding fatigue...junk food is a total no-no. As in, no cost = no energy. There are days I've missed a cooked meal, and the fatigue kicks in within half an hour. I've come to think of the fatigue as "instant starvation".
Supplements - particularly vitamins - only work when other more basic biochemical infrastructure components such as EFAs are already in place to help your body make use of them. If your cell membranes lack the right lipids, they cannot process anything properly. Ditto for minerals. And to make things worse, some vitamins and minerals are "antagonists" - they mop each other up, preventing uptake.
In terms of behaviour, I find that treating exercise like I'm weight lifting is a good idea. If you remember that feeling tired is like a pain warning from your muscles, a little pushes your boundaries but too much is damaging, - then you will not overdo things.
I don't know if this is any consolation to you...but in my experience it takes years to get used to throttling back and taking it easy when circumstances other than your body's warnings say otherwise.