I have equally dismal memories of school physical education programs. I got much more exercise by going roller skating and playing softball, even though I was crap at catching. I now know that my eyes don't focus together, which for some reason they couldn't figure out back in the dark ages. Of course, a lot of my activities started getting curtailed early on thanks to repeated knee injuries.

Fencing sounds like fun, but I would imagine it would be difficult with dodgy knees.