Joint damage & range of motion will dictate and guide you as to what sort of exercises one is capable of.
Swimming yes is wonderful. Builds muscles plus is the safest for joints. Even treading through the water and water walking is THE BEST and burns a lot of calories while being "anti-gravity" lol.
My best thing I'm working on is just plain old walking. Best posture I can keep with my back, and working on dostance rather than speed.
I have great walking shoes and like you...the knees,back, hips and shoulders KILL after some walks, even short ones.
I think when it comes to exertion, some days will be better than others...and sometimes ya just can't predict it neither.
I would absolutely pick up a few pilates moves! Especially for the arms and hips. Maintains great range of motiona nd joint function. And pilates moves one can do without equipment too. I think probably you can get some quick online video clips of certain moves if ya Google pilates. I do some that I saw in magazines and while I was a "gym member". <<<sigh>>> I miss the ELLIPTICAL trainer.