You really do need to find a new primary care physician. Any of you do, who find your current doctor unwilling to talk with you, or uninterested in your health needs. That's what they are THERE for, after all!
I actually interview primary care doctors when I need a new one (like, when I moved to my home now 10 years ago). I'm bound by the list of doctors my health insurance will cover, of course -- but I did interview (at my own cost) 4 of them on the list before I found my current doctor. I had one in the first year who I thought was going to work out, but I "fired" him as soon as I could, and then started the process again, to find my current doctor.
I call the office, tell the receptionist (or nurse or whoever it is that answers the phone) that I want a 20 minute - half hour appointment, at my expense, at which time I am going to share with the doctor my medical situation as I know it, ask him/her for her opinion, discuss my aversion to "alternative / complementary" medicine and be sure the doctor is OK with that, and other matters related to my care. No exam necessary, just an interview. I've never been refused. I go prepared with questions and concerns and see what the doctor's response is. I ask about how I'd reach her in an emergency; if there are others in the office who I should talk with (like a nurse-practioner, or the nurse that works exclusively with my doctor) if the doctor is not available; who she uses as back-ups when she is not available; how she feels about dealing with the fact that I use a wheelchair (needing assistance, for example, to get on the exam table, etc.) And then I try to get a feel for her (or his) attitude towards me. And if I don't like it...I tell them up front that I won't be back, thanks for their time, and I'm outta there.
I really like my current doctor -- who calls me directly with test results, calls me (or my husband, who is also her patient) if we've seen her with health problems, just to see how we're doing. Last year I happened to have my annual physical on my birthday -- I got home to find a call from her, wishing me a happy birthday -- it came before my appointment, but we'd been out all day shopping and then to the doctor so missed it. I'm sure she wondered why I didn't say anything about it at the appointment! :)
So don't be afraid to look for a new doctor -- then simply call and ask the office to transfer your records. You don't really owe the "old" doctor any sort of explanation, and can just move to someone new, who is more acccepting and understanding.