Yes, report him and also let your neuro know about your experience. I'm sure he won't want to send his patients there, either, plus he might also report him.
I had a similar experience, but worse, with an Internist when I was quite young. I didn't know to report him and, when I had enough maturity to know what he was REALLY doing, he was no longer in the area. If he were, I would have reported him then. I really regret that I didn't know enough to do that in the first place because I wonder how many other women he assaulted.
Sherrine