Hello GZ,
Wish I had more time to respond, but only have a few minutes...so I'll cut to the chase. Years ago I had an extreme panic attack on a long isolated road. Result was a phobia of driving on isolated roads, miles from anything or anybody.
These days I make an effort to travel these types of roads. I take my G/F with me for support and once I start down a road that is isolated...my mind set is I will feel some anxiety but not to fight it and worse case scenario..I may have a panic attack...but if I do, I know it will pass and I will be OK.
I have yet to panic doing this and I think it's cause I except the anxiety without resisting in anyway. I echo what SC said about people who have overcome diving issues that were extreme. Keep your hope that you can do this and don't tell yourself you can't or find reasons that you can't. It's all about small baby steps to make progress.
Best wishes to you that you make some progress and know there are many that struggle like you. Take care.....WM