I think your best bet is to call your doctor and get advice.
I doubt if you are going through a withdrawl if you never took your benzo daily, but everyone is different.
Here is a good website as well that may give you some information on this subject.
http://theroadback.org/Did you try an OTC for sleeping yet...I think that you mentioned in a previous post, wanting to give this option a go?
Wishing you luck in whatever you ultimately decide to do. I am sure with your prescribing doc's help you will get it worked out.:)
S.C.