It's pretty common for a lesion to affect your inner sense of balance -- the vertigo and unsteadiness on your feet are common outward manifestations of that, but so is the sense of movement around you -- what you are describing when in a crowd. It's your brain trying to keep you upright and steady, and your eyes seeing the movement around you. There's a delay between eyes and brain, which causes the brain to work harder to keep you balanced...and back and forth.
It can be real disconcerting. I find even things like the back-and-forth movement of a vacuum can cause me to be dizzy and disoriented, and sometimes even lead to nausea. Or the back-and-forth movement of a mop, or watching someone shovel snow with a repetitive movement.
I do my best to stay out of crowds :)