SethpBig breath and try not to overthink this issue, in fact consider it a good thing that you are able to get an apointment with a Psychiatrist as this is the person who is really trained in prescribing medication and different treatment options.
In practical terms, psychiatrists tend to focus on diagnosing various mental disorders and then treating them with medications or other types of medical interventions. Your Dr. may have been referring to the psychiatrist being more experienced in diagnosising as well as in therapy options that would fit best for you.
I have a really awesome Psychiatrist and we have an good relationship once we got to know each other well. She is a huge help to me now and always respectfully listens to ideas I may have re my anxiety/depression issues.
Remember it is rare to nearly nil for someone to not have the right to say No to medications. However, a lot of thought and discussion with another person experienced in the use of medications would be wise before saying no just on principal.
Be well and remember you are never alone,
Kitt